201-250 of 10000 results (46ms)
2018-06-12 §
12:14 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1099:3311 for alter table (duration: 00m 52s) [production]
12:11 <marostegui> Deploy schema change on dbstore1002:s1 T191316 T192926 T89737 T195193 [production]
12:05 <akosiaris@puppetmaster1001> conftool action : set/weight=10; selector: dc=.*,service=mathoid,cluster=kubernetes,name=.* [production]
11:47 <moritzm> updated component/cassandra311 on apt.wikimedia.org to 3.11.2 [production]
10:26 <jynus> setting expire_log_days on db1066 as 30 [production]
10:21 <godog> bounce stuck rsyslog on lithium / wezen - T136312 [production]
09:41 <vgutierrez> cp3037 has been depooled due to unknown hardware issues T196974 [production]
08:48 <marostegui> Stop replication on db2094 to change triggers for archive table [production]
08:36 <volans> running puppet on failed hosts post small puppet outage and puppetdb reboot [production]
08:35 <akosiaris> rebalance ganeti codfw cluster [production]
08:35 <ema@neodymium> conftool action : set/pooled=no; selector: name=cp3037.esams.wmnet [production]
08:33 <akosiaris> reboot puppetdb1001 for spec-ctrl enable. Bundling it with a minor puppet outage to only have a torrent of harmless puppet failures once [production]
08:15 <akosiaris> ganeti2002 reboot for microcode update [production]
08:04 <akosiaris> ganeti2006 reboot for microcode update [production]
08:03 <marostegui> Deploy schema change on s1 codfw primary master (db2048) with replication, this will generate lag on codfw T191316 T192926 T89737 T195193 [production]
07:53 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1121 after alter table (duration: 00m 50s) [production]
07:43 <akosiaris> ganeti2007 reboot for microcode update [production]
07:41 <akosiaris> ganeti2003 reboot for microcode update [production]
07:31 <mutante> closing idle screen session on tin (about to be decomed, dont use anymore) [production]
06:37 <marostegui> Deploy schema change on db1121 with replication, this will generate lag on labsdb:s4 T191316 T192926 T89737 T195193 [production]
06:37 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1121 for alter table (duration: 00m 50s) [production]
06:31 <marostegui> Stop replication on db1095, db1102, db1125 to change triggers - T192926 [production]
06:30 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1091 after alter table (duration: 00m 51s) [production]
05:09 <marostegui> Deploy schema change on db1091 T191316 T192926 T89737 T195193 [production]
05:09 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1091 for alter table (duration: 00m 52s) [production]
02:45 <l10nupdate@deploy1001> ResourceLoader cache refresh completed at Tue Jun 12 02:45:53 UTC 2018 (duration 10m 18s) [production]
02:35 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.7) (duration: 14m 10s) [production]
00:35 <legoktm> remove non-deployers from wmf-deployment Gerrit group (T196959) [production]
00:16 <ejegg> updated CiviCRM from 69091d8b5f to f54f9810ab [production]
2018-06-11 §
23:45 <ebernhardson@deploy1001> Synchronized wmf-config/CirrusSearch-common.php: SWAT: Lower CirrusSearch delayed job drop timeout (duration: 00m 50s) [production]
23:26 <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: Tune CirrusSearch slow logging (duration: 00m 48s) [production]
23:18 <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Promote Cirrus MLR models from AB test to prod (duration: 00m 51s) [production]
23:02 <twentyafterfour> phabricator: restarting apache2 on phab1001 to free up apache workers [production]
22:17 <awight@deploy1001> Finished deploy [ores/deploy@6ee8775]: ORES: bswiki, euwiki, srwiki models (duration: 33m 58s) [production]
21:43 <awight@deploy1001> Started deploy [ores/deploy@6ee8775]: ORES: bswiki, euwiki, srwiki models [production]
20:57 <arlolra> Updated Parsoid to 06b74d2 (T191843) [production]
20:49 <arlolra@deploy1001> Finished deploy [parsoid/deploy@97cdab8]: Updating Parsoid to 06b74d2 (duration: 17m 09s) [production]
20:32 <arlolra@deploy1001> Started deploy [parsoid/deploy@97cdab8]: Updating Parsoid to 06b74d2 [production]
20:27 <otto@deploy1001> Finished deploy [eventstreams/deploy@6b013f9]: Enable composite stream and timestamp since param - T196009 , T187418 (duration: 09m 52s) [production]
20:17 <otto@deploy1001> Started deploy [eventstreams/deploy@6b013f9]: Enable composite stream and timestamp since param - T196009 , T187418 [production]
20:02 <ottomata> bouncing varnishkafka-webrequest on cp3039,cp3047,cp2007,cp3010 [production]
19:56 <ottomata> bouncing varnishkafka on cp3032 [production]
19:29 <aaron@deploy1001> Synchronized php-1.32.0-wmf.7/includes/libs/rdbms/ChronologyProtector.php: 11e596776f940 - add some logging details (duration: 00m 53s) [production]
18:50 <pnorman@deploy1001> Finished deploy [tilerator/deploy@074d01a] (cleartables): Restore full config (duration: 00m 16s) [production]
18:49 <pnorman@deploy1001> Started deploy [tilerator/deploy@074d01a] (cleartables): Restore full config [production]
18:15 <urandom> convert timeline indices to time-windowed compaction - T196024 [production]
17:59 <twentyafterfour> phabricator: taking phd offline while I clear out the queue backlog (downtime is logged in icinga) see T196840 [production]
17:51 <twentyafterfour> phabricator: rebuilding git parent caches [production]
17:33 <gehel@deploy1001> Finished deploy [wdqs/wdqs@37f6f32]: new version of wdqs GUI and updater (duration: 13m 38s) [production]
17:27 <twentyafterfour> phabricator: restarting phd for D1067 [production]