2501-2550 of 10000 results (66ms)
2019-08-23 §
17:39 <robh@cumin1001> START - Cookbook sre.hosts.decommission [production]
12:24 <_joe_> pooling mw1270 temporarily, debugging performance issues [production]
12:15 <_joe_> depooling mw1270 temporarily, performance issues [production]
11:13 <marostegui> Upgrade db1114 from 10.3.16 to 10.3.17 [production]
10:06 <dcausse> elastic: reindexing wikis with old mappings in eqiad & codfw (T230990) [production]
05:52 <moritzm> installing squid3 security updates [production]
05:11 <marostegui> Stop MySQL on db2066 for decommission T230885 [production]
05:08 <marostegui> Remove db2066 from tendril and zarcillo T230885 [production]
2019-08-22 §
23:41 <urbanecm@deploy1001> Synchronized php-1.34.0-wmf.19/extensions/WikibaseLexeme: SWAT: e4a5457: Fix Lexemes RDF generation (T230974) (duration: 00m 49s) [production]
23:32 <urbanecm@deploy1001> Synchronized wmf-config/abusefilter.php: SWAT: eb1c4ea: Rename globals and rights in AbuseFilter config (duration: 00m 47s) [production]
23:24 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 66b719d: General cleanup of `groupOverrides` (T231041) (duration: 00m 47s) [production]
23:15 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 872f4b0: Change language code for punjabiwikimedia, resyncing, got broken pipe at the end (T230680) (duration: 00m 47s) [production]
23:13 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 872f4b0: Change language code for punjabiwikimedia (T230680) (duration: 00m 48s) [production]
23:10 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: a5917e4: Clean up `wgRateLimits` to remove unneeded entries (T231040) (duration: 00m 48s) [production]
22:07 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Update MachineVision Beta config (duration: 00m 47s) [production]
21:19 <eileen> tools revision changed from 5c080bac63 to c0f4e7a379 [production]
20:35 <ejegg> updated payments-wiki from 85dce8f79f to 231b7b0850 [production]
17:14 <elukey> remove analytics-tool1002 from ganeti - T231021 [production]
17:12 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
17:12 <elukey@cumin1001> START - Cookbook sre.hosts.decommission [production]
14:37 <_joe_> restarting php-fpm on mw1348 to observe the effect on the slowdown, T231011 [production]
13:47 <elukey> update puppet compiler's facts [production]
13:42 <jijiki> Restart php-fpm on mw1348 and mw1347 [production]
13:41 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.34.0-wmf.19 [production]
13:36 <joal@deploy1001> Finished deploy [analytics/refinery@a9b99e9]: Regular weekly analytics deployment train (1 day late) (duration: 18m 57s) [production]
13:32 <jiji@deploy1001> Synchronized wmf-config/CommonSettings.php: Reverting PHP7 traffic back to 20% - T219150 (duration: 00m 57s) [production]
13:27 <tarrow@deploy1001> Synchronized php-1.34.0-wmf.19/extensions/Wikibase/client/: [[gerrit:531677|Revert "Use the backwards-compatible HTML ID for the wikidata item link" (T230958, T66315)]] (duration: 00m 58s) [production]
13:18 <joal@deploy1001> Started deploy [analytics/refinery@a9b99e9]: Regular weekly analytics deployment train (1 day late) [production]
12:56 <_joe_> restarting mw1270 with slowlog disabled [production]
12:41 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:40 <elukey@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
12:39 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:39 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:38 <_joe_> disabled slowlog on mw1348, repooling after reload [production]
12:37 <jijiki> Pooling mv1347 not mw1247 [production]
12:35 <jijiki> Pooling mw1247 [production]
12:16 <moritzm> upgrading mariadb (packaged Debian version) on matomo1001 [production]
12:15 <elukey@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
12:13 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:07 <jijiki> Depooling mw1347 and mw1348 [production]
10:55 <jiji@deploy1001> Synchronized wmf-config/CommonSettings.php: Push PHP7 traffic to 33.3% - T219150 (duration: 01m 01s) [production]
09:09 <ema> rolling ats-backend-restart to enable @debug system call family [production]
09:02 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:00 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:17 <moritzm> restarting oozie on an-coord1001 [production]
07:54 <tarrow@deploy1001> Synchronized php-1.34.0-wmf.19/extensions/Wikibase/repo/: Backport for UBN [[gerrit:531527|Hack to avoid trying to termbox render page before save (T230937)]] (duration: 00m 56s) [production]
07:46 <marostegui> Deploy grants on labsdb1009-labsdb1012 to allow connections for haproxy from dbproxy1019 - T202367 [production]
06:52 <moritzm> installing mariadb-10.1 updates from Stretch 9.9 point release (unrelated to wmf-mariadb, mostly client-side clients/libraries as shipped in Debian) [production]
06:37 <moritzm> installing python-pip updates from Stretch 9.9 point release [production]
05:41 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db2066 from config T230885 (duration: 00m 54s) [production]