251-300 of 10000 results (52ms)
2018-06-11 ยง
17:59 <twentyafterfour> phabricator: taking phd offline while I clear out the queue backlog (downtime is logged in icinga) see T196840 [production]
17:51 <twentyafterfour> phabricator: rebuilding git parent caches [production]
17:33 <gehel@deploy1001> Finished deploy [wdqs/wdqs@37f6f32]: new version of wdqs GUI and updater (duration: 13m 38s) [production]
17:27 <twentyafterfour> phabricator: restarting phd for D1067 [production]
17:25 <twentyafterfour> Phabricator: deploying hotfix (D1067) refs T196840 T196860 T196855 [production]
17:20 <gehel@deploy1001> Started deploy [wdqs/wdqs@37f6f32]: new version of wdqs GUI and updater [production]
17:19 <gehel@deploy1001> Finished deploy [wdqs/wdqs@37f6f32]: new version of wdqs GUI and updater (duration: 00m 03s) [production]
17:19 <gehel@deploy1001> Started deploy [wdqs/wdqs@37f6f32]: new version of wdqs GUI and updater [production]
17:17 <gehel@deploy1001> Finished deploy [wdqs/wdqs@37f6f32]: new version of wdqs GUI and updater (wdqs1009 only) (duration: 00m 26s) [production]
17:16 <gehel@deploy1001> Started deploy [wdqs/wdqs@37f6f32]: new version of wdqs GUI and updater (wdqs1009 only) [production]
17:00 <akosiaris> ganeti2008 reboot for microcode update [production]
16:42 <marostegui> Set disk 32:1 offline on db1065 to get a new one - T196806 [production]
16:02 <akosiaris@deploy1001> Finished deploy [proton/deploy@97ec4bf]: (no justification provided) (duration: 01m 32s) [production]
16:00 <akosiaris@deploy1001> Started deploy [proton/deploy@97ec4bf]: (no justification provided) [production]
15:44 <akosiaris@deploy1001> Finished deploy [proton/deploy@97ec4bf]: (no justification provided) (duration: 00m 33s) [production]
15:43 <akosiaris@deploy1001> Started deploy [proton/deploy@97ec4bf]: (no justification provided) [production]
15:43 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1084 after alter table (duration: 00m 50s) [production]
15:33 <akosiaris@deploy1001> Finished deploy [proton/deploy@97ec4bf]: (no justification provided) (duration: 00m 22s) [production]
15:33 <akosiaris@deploy1001> Started deploy [proton/deploy@97ec4bf]: (no justification provided) [production]
15:31 <marostegui> Set offline disk 32:3 on db1063 - T196806 [production]
15:31 <marostegui> Set offline disk 32:1 on db1065 - T196806 [production]
15:02 <akosiaris@deploy1001> Finished deploy [proton/deploy@97ec4bf]: (no justification provided) (duration: 35m 33s) [production]
14:55 <marostegui> Deploy schema change on db1084 T191316 T192926 T89737 T195193 [production]
14:55 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1084 for alter table (duration: 00m 50s) [production]
14:53 <akosiaris> reboot bohrium for kernel upgrades and spec-ctrl enabling. Manually stopped mysql behorehand [production]
14:50 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1081 after alter table (duration: 00m 50s) [production]
14:35 <akosiaris> reboot mx1001, poolcounter1001 for kernel upgrades and spec-ctrl enabling [production]
14:26 <akosiaris@deploy1001> Started deploy [proton/deploy@97ec4bf]: (no justification provided) [production]
14:25 <godog> upload scap 3.8.2-1 - T196710 [production]
14:21 <hoo> Updated operations/dumps/dcat (536bd5b..559dee3) on snapshot1008 [production]
13:58 <marostegui> Deploy schema change on db1081 T191316 T192926 T89737 T195193 [production]
13:56 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1081 for alter table (duration: 00m 48s) [production]
13:56 <otto@deploy1001> Finished deploy [eventlogging/analytics@08a1dff]: Producing events with kafka timestamp set to event time - T196407 (duration: 00m 04s) [production]
13:56 <otto@deploy1001> Started deploy [eventlogging/analytics@08a1dff]: Producing events with kafka timestamp set to event time - T196407 [production]
13:54 <otto@deploy1001> Finished deploy [eventlogging/eventbus@08a1dff]: Producing events with kafka timestamp set to event time - T196407 (duration: 01m 55s) [production]
13:52 <otto@deploy1001> Started deploy [eventlogging/eventbus@08a1dff]: Producing events with kafka timestamp set to event time - T196407 [production]
13:50 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1103:3314 after alter table (duration: 00m 50s) [production]
13:47 <hashar> European SWAT completed [production]
13:35 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Fix wgMetaNamespace for pswikivoyage - T196837 (duration: 00m 49s) [production]
13:29 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Fix wgMetaNamespace for pswikivoyage - T196837 (duration: 00m 50s) [production]
13:21 <volans@deploy1001> Finished deploy [debmonitor/deploy@81d7333]: Release v0.1.2 (duration: 00m 56s) [production]
13:20 <volans@deploy1001> Started deploy [debmonitor/deploy@81d7333]: Release v0.1.2 [production]
13:20 <hashar@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Use uploaded HD logo for bewikiquote - T196134 (duration: 00m 50s) [production]
13:17 <hashar@deploy1001> Synchronized static/images/project-logos: Change logo files for bewikiquote - T196134 (duration: 00m 50s) [production]
13:13 <hashar@deploy1001> Synchronized static/images/project-logos: Revert "Change bewikiquote logo" - T196134 (duration: 00m 51s) [production]
13:01 <volans@deploy1001> Finished deploy [debmonitor/deploy@81d7333]: Release v0.1.2 (duration: 07m 16s) [production]
12:54 <volans@deploy1001> Started deploy [debmonitor/deploy@81d7333]: Release v0.1.2 [production]
11:45 <arturo> T196633 downtime labcontrol100[3,4] due to unexpected puppet errors on installation of keystone [production]
11:38 <arturo> T196633 deploy keystone to labcontrol100[3,4].wikimedia.org. Dormant daemon, no DB yet [production]
11:30 <ppchelko@deploy1001> Finished deploy [cpjobqueue/deploy@b5396cd]: Tune cirrus jobs concurrencies (duration: 00m 42s) [production]