2751-2800 of 10000 results (47ms)
2019-02-27 ยง
21:44 <eileen> civicrm revision is c81fe7a4fd, config revision is 050abdf9e8 [production]
21:26 <XioNoX> delete local pref for peering sessions in eqord - T204281 [production]
20:53 <XioNoX> delete local pref for peering sessions in codfw/eqdfw - T204281 [production]
20:50 <hashar> 1.33.0-wmf.19 not rolled to group1. Pending T217285 (Wikibase raising exception on commonswiki). To be figured out during European day time. [production]
20:50 <eileen> civicrm revision changed from 224bf15206 to c81fe7a4fd, config revision is d1826e371b [production]
20:14 <hashar@deploy1001> rebuilt and synchronized wikiversions files: (no justification provided) [production]
20:04 <hashar@deploy1001> Synchronized php: group1 wikis to 1.33.0-wmf.19 (duration: 00m 53s) [production]
20:04 <hashar@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.19 [production]
19:49 <bstorm_> stopped slave on labsbd1004 for T193264 [production]
19:43 <bstorm_> downtimed labsdb1004 to stop mysql for transferring data for T193264 [production]
19:32 <SMalyshev> repooled wdqs1005, caught up [production]
19:26 <herron> replacing kafka on logstash1004 with logstash1010 T213898 [production]
18:56 <SMalyshev> depooled wdqs1005 to let it catch up [production]
18:36 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@465673b]: Redeploy GUI for T217161 (duration: 10m 51s) [production]
18:28 <cmjohnson1> powering off mw126[3-6] one at a time to move to different rack A5 T212348 [production]
18:25 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@465673b]: Redeploy GUI for T217161 [production]
18:21 <cmjohnson1> powering off mw1262 to move to different rack A5 T212348 [production]
18:15 <cmjohnson1> powering off mw1261 to move to different rack A5 T212348 [production]
17:57 <niharika29@deploy1001> Synchronized php-1.33.0-wmf.19/extensions/Flow/: Make VisualEditor unwrap <section> tags T217206 (duration: 01m 00s) [production]
17:56 <elukey> roll restart hadoop hdfs namenodes on an-master100[1,2] to pick up the new rack config of analytics1071 [production]
17:37 <niharika29@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Welcome survey: add a control group to viwiki T216669 (duration: 00m 54s) [production]
17:34 <niharika29@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Stop collecting data for CitaitonUsage and CitationUsagePageLoad T213969 (duration: 00m 55s) [production]
17:22 <elukey> drain + shutdown of analytics1071 to allow its move to A5 - T212348 [production]
17:19 <cmjohnson1> powering off wtp1030 to move to different rack A5 T212348 [production]
17:14 <cmjohnson1> powering off wtp1029 to move to different rack A5 T212348 [production]
17:06 <cmjohnson1> powering off wtp1029 to move to different rack A5 T212348 [production]
17:05 <RoanKattouw> Running foreachwikiindblist dblists/echo.dblist extensions/Echo/maintenance/removeOrphanedEvents.php on mwmaint1002 [production]
16:58 <hashar@deploy1001> Synchronized php-1.33.0-wmf.19/extensions/Score: Revert "beautify lilypond error message output" - T217241 (duration: 00m 56s) [production]
16:49 <jijiki> Deploy LVS for eventgate-analytics - T211247 [production]
16:26 <volans> temporarily disabled puppet on icinga[12]001 to deploy g/493171 [production]
16:20 <volans> force-rebooting icinga1001 (to test some puppet changes) - T214760 [production]
15:34 <jbond42> rolling openssl security updates to jessie canary servers [production]
14:25 <marostegui> Deploy schema change on abuse_filter_log on s7 codfw - lag will be generated on codfw - T187295 [production]
14:01 <marostegui> Change indexes on abuse_filter_log on db1089 - T187295 [production]
14:00 <moritzm> uploaded openssl 1.0.2r to jessie-wikimedia [production]
12:08 <jbond42> correction: rolling updates of apache on mw api servers *not* jobrunners [production]
12:04 <jbond42> rolling updates of apache on mw jobrunners [production]
11:38 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1077 after MySQL upgrade (duration: 00m 53s) [production]
11:28 <godog> cleanup log4j from lvs eqiad / ipvsadm -D -t logstash.svc.eqiad.wmnet:4560 [production]
11:25 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Give more traffic to db1077 after MySQL upgrade (duration: 00m 54s) [production]
11:17 <godog> roll-restart pybal after removing logstash log4j service [production]
10:54 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Give more traffic to db1077 after MySQL upgrade (duration: 00m 54s) [production]
10:35 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1077 with low weight after MySQL upgrade (duration: 00m 53s) [production]
09:55 <marostegui> Stop MySQL on db1077 for mysql upgrade - this will generate lag on labsdb:s3 [production]
09:55 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1077 for MySQL upgrade (duration: 00m 53s) [production]
09:50 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1082 (duration: 00m 54s) [production]
09:35 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool in API db1082 after mysql upgrade (duration: 00m 53s) [production]
09:21 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1082 after mysql upgrade (duration: 00m 54s) [production]
09:05 <marostegui> Stop MySQL on db1082 for mysql upgrade [production]
08:41 <godog> enable mmjsonparse by default on kafka outputs - T213189 [production]