3801-3850 of 10000 results (70ms)
2019-06-04 §
11:19 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:494016|Add editcontentmodel right to the templateeditor group on testwiki]] (T217499) (duration: 00m 47s) [production]
11:15 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:491054|Add new namespaces for th.wiki]] (T216322) (duration: 00m 47s) [production]
11:09 <krinkle@deploy1001> Synchronized php-1.34.0-wmf.6/includes/: T221577 / 1286d131c01886 (duration: 01m 07s) [production]
11:02 <moritzm> draining ganeti1007 for eventual reboot to MDS-enabled Linux kernel [production]
11:02 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:02 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
10:44 <jbond42> mw1* restarts will be delayed untill 11:15 [production]
10:42 <jbond42> will start rolling reboots of mw1* servers 1t 10:50 [production]
09:27 <moritzm> draining ganeti1006 for eventual reboot to MDS-enabled Linux kernel [production]
09:25 <jijiki> disable puppet on mc* hosts to merge 511963 and 511973 [production]
09:01 <moritzm> draining ganeti1005 for eventual reboot to MDS-enabled Linux kernel [production]
08:57 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:57 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
08:32 <elukey> remove memcached nutcracker config from mw1* hosts (not used). Changes will be picked up when nutcracker will be restarted (after reboots, etc..) - T214275 [production]
08:23 <moritzm> draining ganeti1004 for eventual reboot to MDS-enabled Linux kernel [production]
08:04 <marostegui> Stop MySQL on db2046 to clone db2058 - T221533 [production]
08:04 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2046 (duration: 00m 47s) [production]
08:03 <elukey> restart hive-server2 on an-coord1001 to pick up new GC/Heap settings [production]
07:35 <mobrovac@deploy1001> Finished deploy [restbase/deploy@abcb534]: Use only Proton for PDF rendering - T210651 (duration: 19m 16s) [production]
07:22 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:21 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
07:21 <moritzm> draining ganeti1002 for eventual reboot to MDS-enabled Linux kernel [production]
07:16 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Move db2058 from s4 to s6 (duration: 00m 47s) [production]
07:16 <mobrovac@deploy1001> Started deploy [restbase/deploy@abcb534]: Use only Proton for PDF rendering - T210651 [production]
06:57 <elukey> restart hive metastore on an-coord1001 to apply new GC/heap settings [production]
06:42 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1091 after upgrade (duration: 00m 48s) [production]
06:21 <elukey> restart pdfrender on scb1002 (flapping) [production]
06:04 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1091 after upgrade (duration: 00m 47s) [production]
05:54 <marostegui> Stop MySQL on db2078:m3 - T221533 [production]
05:51 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1091 after upgrade (duration: 00m 47s) [production]
05:40 <marostegui> Stop MySQL on db1091 for MySQL upgrade T224852 [production]
05:40 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1091 for upgrade (duration: 00m 48s) [production]
05:27 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1097 after upgrade (duration: 00m 46s) [production]
05:19 <marostegui> Stop MySQL on db1097 for upgrade [production]
05:19 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1097 for upgrade (duration: 00m 47s) [production]
04:59 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db1081 from API (duration: 00m 49s) [production]
01:10 <bstorm_> T223406 depooled/repooled labsdb1009 for view updates [production]
00:09 <bstorm_> T223406 repooled labsdb1011 after completing view updates [production]
2019-06-03 §
22:20 <bstorm_> T223406 depooled labsdb1011 [production]
22:08 <bstorm_> T223406 repooled labsdb1010 after completing view updates [production]
21:29 <XioNoX> drop all ICMP frag on all routers - T224186 [production]
19:57 <XioNoX> stop sampling from cr2-eqiad [production]
18:47 <XioNoX> Add RPKI validators to all routers - T220669 [production]
18:35 <hashar> switch most Quibble jobs to node 10 T222406 - ttps://gerrit.wikimedia.org/r/#/c/integration/config/+/514034/ T222406 [production]
18:35 <XioNoX> drop all ICMP frag on cr1/2-eqiad - T224186 [production]
18:17 <XioNoX> add routinator 0.4.0 to APT repo - T220669 [production]
17:16 <onimisionipe@deploy1001> Finished deploy [wdqs/wdqs@9e3035c]: Blazegraph version wmf.4 (duration: 11m 29s) [production]
17:05 <onimisionipe@deploy1001> Started deploy [wdqs/wdqs@9e3035c]: Blazegraph version wmf.4 [production]
16:40 <onimisionipe> started osm-import on maps2004 - T224395 [production]
16:30 <bstorm_> T223406 depooled labsdb1010 for view updates [production]