3551-3600 of 10000 results (26ms)
2020-05-18 ยง
19:07 <herron> performing rolling maintenance on kafka-main to pick up java security updates [production]
19:00 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Ic005093778d (duration: 01m 08s) [production]
18:58 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings-labs.php: Ic005093778d (duration: 01m 06s) [production]
18:49 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
18:46 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
18:38 <volans> upgraded spicerack to 0.0.37-1 on cumin[12]001 [production]
18:24 <hnowlan@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) [production]
18:13 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Fix English Wikipedia wordmark dimensions (T252143) (duration: 01m 06s) [production]
17:14 <XioNoX> update domain object for 56.15.185.in-addr.arpa - T247972 [production]
17:06 <bblack> dns1001 - removing downtimes, back in service - T241770 [production]
16:45 <bstorm_> updated views on labsdb1011 for the wb_terms changes T251598 [production]
16:32 <bblack@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:30 <bblack@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:17 <bblack> dns1001 - reimaging for new NIC - T241770 [production]
16:10 <volans> uploaded spicerack_0.0.37-1_amd64.deb to apt.wikimedia.org stretch-wikimedia [production]
15:52 <hnowlan> rolling codfw cassandra for java security updates [production]
15:51 <hnowlan@cumin1001> START - Cookbook sre.cassandra.roll-restart [production]
15:30 <hnowlan@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) [production]
15:22 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) [production]
15:11 <Krinkle> krinkle@mc1021 Pruning the old `echo:seen:` Redis keys that didn't have a ttl yet [production]
14:57 <jayme@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
14:56 <hnowlan> roll-restart of sessionstore cassandra hosts for java security update [production]
14:55 <hnowlan@cumin1001> START - Cookbook sre.cassandra.roll-restart [production]
14:53 <jayme@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'termbox' for release 'production' . [production]
14:50 <jayme@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'termbox' for release 'staging' . [production]
14:49 <jayme@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'termbox' for release 'test' . [production]
14:35 <hnowlan@deploy1001> Finished deploy [changeprop/deploy@16bf19f]: Stop consuming purges topic, purged is now doing this (duration: 01m 22s) [production]
14:34 <hnowlan@deploy1001> Started deploy [changeprop/deploy@16bf19f]: Stop consuming purges topic, purged is now doing this [production]
14:33 <_joe_> start consuming $dc.resource-purge kafka topic from purged in all of esams T133821 [production]
14:29 <_joe_> start consuming $dc.resource-purge kafka topic from purged in all of eqiad T133821 [production]
14:23 <_joe_> start consuming $dc.resource-purge kafka topic from purged in all of eqsin, ulsfo T133821 [production]
14:19 <_joe_> start consuming $dc.resource-purge kafka topic from purged in all of codfw T133821 [production]
14:15 <kormat@cumin1001> dbctl commit (dc=all): 'Depool db2073 while replacing it T252985', diff saved to https://phabricator.wikimedia.org/P11216 and previous config saved to /var/cache/conftool/dbconfig/20200518-141505-kormat.json [production]
14:12 <bblack> dns1001 - shutting down for T241770 [production]
14:09 <volans> uploaded spicerack_0.0.36-1_amd64.deb to apt.wikimedia.org stretch-wikimedia [production]
14:07 <bblack> authdns - ns[01] static routes on cr[12]-eqiad switching back to authdns1001 (oops, that's not the server we're taking offline today!) [production]
14:06 <vgutierrez> upload trafficserver 8.0.7-1wm9 to apt.wm.o (buster) [production]
14:02 <elukey@cumin1001> START - Cookbook sre.hadoop.roll-restart-workers [production]
14:00 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) [production]
13:57 <bblack> authdns - ns[01] static routes on cr[12]-eqiad switching from authdns1001 to dns1002 for T241770 [production]
13:29 <elukey@cumin1001> START - Cookbook sre.hadoop.roll-restart-workers [production]
13:00 <hashar@deploy1001> Synchronized php-1.35.0-wmf.32/skins/Vector/includes/VectorTemplate.php: VectorTemplate: SkinTemplateToolboxEnd hook isn't deprecated - T252906 (duration: 01m 07s) [production]
11:52 <marostegui> Install 10.1.43-2 on db1122 and db1109 - T251981 [production]
11:27 <Lucas_WMDE> EU SWAT done [production]
11:25 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.35.0-wmf.32/extensions/Wikibase/: SWAT: [[gerrit:596616|Fix core's TitleFactory not being used correctly (T252803)]] (duration: 01m 12s) [production]
11:20 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:597010|Update GrowthExperiments mentor list page for viwiki]] (duration: 01m 06s) [production]
11:10 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:596916|Make the threshold for Chinese WP to prevent publishing 5% more strict (T252786)]] (duration: 01m 06s) [production]
10:38 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:597033| Bumping portals to master (597033)]] (duration: 01m 06s) [production]
10:37 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:597033| Bumping portals to master (597033)]] (duration: 01m 32s) [production]
10:37 <elukey> copy prometheus-druid-exporter 0.8-1 from stretch to buster wikimedia [production]