2451-2500 of 10000 results (51ms)
2018-02-19 §
10:35 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1093 for alter table (duration: 00m 56s) [production]
10:27 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool db1098 s6 and s7 (duration: 00m 56s) [production]
10:10 <marostegui> Upgrade mariadb and kernel on db1098 [production]
09:59 <marostegui> Enable GTID on dbstore2002:3313 and dbstore2001:3316 [production]
09:57 <marostegui> Enable GTID on dbstore2002 and dbstore2001 for x1 [production]
09:55 <jynus> reenable gtid replication on db1053 and db2042 [production]
09:53 <jmm@puppetmaster1001> conftool action : set/pooled=inactive; selector: mw1260.eqiad.wmnet [production]
09:53 <jmm@puppetmaster1001> conftool action : set/pooled=inactive; selector: mw1259.eqiad.wmnet [production]
09:43 <marostegui> Upgrade mariadb and kernel on db2033 [production]
09:36 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Fully repool db1090 (duration: 00m 55s) [production]
09:18 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1105 - T162807 (duration: 00m 55s) [production]
08:55 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1098:3317 for mariadb and kernel upgrade (duration: 00m 55s) [production]
08:49 <marostegui> Deploy schema change on db1098:3316 - T187089 T185128 T153182 [production]
08:47 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1098:3316 for alter table (duration: 00m 55s) [production]
08:24 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Increase traffic for db1090 (duration: 00m 55s) [production]
08:11 <godog> repool mw1227 - T149287 [production]
08:02 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Promote db2034 to x1 codfw master - T184888 (duration: 00m 56s) [production]
07:58 <moritzm> installing werkzeug security updates on trusty [production]
07:42 <marostegui> Change topology on x1 codfw - T184888 [production]
07:21 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Slowly repool db1090 (duration: 00m 55s) [production]
07:01 <marostegui> Reboot db1090 for kernel ugprade, mariadb upgrade, socket path location upgrade [production]
07:00 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1090 (duration: 00m 55s) [production]
06:44 <marostegui> Stop MySQL on db1089 to update its socket path [production]
06:42 <marostegui> Deploy schema change on s6 codfw master (db2039), this will generate lag on codfw - T187089 T185128 T153182 [production]
06:40 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1089 and db1105 - T162807 (duration: 00m 56s) [production]
05:29 <andrew@tin> Finished deploy [horizon/deploy@6a40f84]: rolling out several horizon bugfixes (duration: 03m 14s) [production]
05:26 <andrew@tin> Started deploy [horizon/deploy@6a40f84]: rolling out several horizon bugfixes [production]
02:50 <l10nupdate@tin> scap sync-l10n completed (1.31.0-wmf.21) (duration: 10m 59s) [production]
2018-02-18 §
15:49 <_joe_> rolling restart (1 at a time, staggered by 2 minutes) of 18 api appservers in equiad [production]
2018-02-17 §
17:33 <twentyafterfour> restarting apache on phab1001 to clear deadlocked workers. refs T182832 [production]
03:15 <demon@tin> Pruned MediaWiki: 1.31.0-wmf.20 [keeping static files] (duration: 01m 17s) [production]
03:12 <demon@tin> Pruned MediaWiki: 1.31.0-wmf.17 (duration: 04m 32s) [production]
2018-02-16 §
21:12 <hashar> Upgraded Zuul to https://gerrit.wikimedia.org/r/#/c/411322/3 | T187567 [production]
20:40 <andrew@tin> Finished deploy [horizon/deploy@efcba2b]: sudo dashboard update (duration: 01m 16s) [production]
20:39 <andrew@tin> Started deploy [horizon/deploy@efcba2b]: sudo dashboard update [production]
20:11 <andrew@tin> Finished deploy [horizon/deploy@1fdd122]: two more small fixes (duration: 01m 21s) [production]
20:10 <andrew@tin> Started deploy [horizon/deploy@1fdd122]: two more small fixes [production]
19:54 <andrew@tin> Finished deploy [horizon/deploy@bdcc12b]: ocata branch with sidebar fix (duration: 03m 12s) [production]
19:51 <andrew@tin> Started deploy [horizon/deploy@bdcc12b]: ocata branch with sidebar fix [production]
18:34 <hashar> upgraded zuul [production]
16:21 <andrew@tin> Finished deploy [horizon/deploy@16f3d8e]: ocata branch with upper new requirements (duration: 08m 00s) [production]
16:13 <andrew@tin> Started deploy [horizon/deploy@16f3d8e]: ocata branch with upper new requirements [production]
16:06 <cmjohnson1> labstore1006 and labstore1007 down for rack relocation [production]
16:03 <andrew@tin> Finished deploy [horizon/deploy@16d0b17]: ocata branch with upper constraints (duration: 02m 18s) [production]
16:00 <andrew@tin> Started deploy [horizon/deploy@16d0b17]: ocata branch with upper constraints [production]
15:40 <andrew@tin> Finished deploy [horizon/deploy@29f9afb]: second attempt at ocata branch (duration: 03m 22s) [production]
15:37 <andrew@tin> Started deploy [horizon/deploy@29f9afb]: second attempt at ocata branch [production]
15:29 <andrew@tin> Finished deploy [horizon/deploy@58d2718]: first attempt at ocata branch (duration: 01m 28s) [production]
15:28 <andrew@tin> Started deploy [horizon/deploy@58d2718]: first attempt at ocata branch [production]
15:27 <godog> shut ms-be1018 for bbu swap - T186988 [production]