901-950 of 10000 results (85ms)
2020-03-18 §
07:55 <moritzm> installing remaining libxslt security updates [production]
07:40 <oblivian@deploy1001> Synchronized wmf-config/ProductionServices.php: eventgate-analytics to use envoy everywhere (duration: 01m 10s) [production]
07:08 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:05 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
06:31 <marostegui> Reboot pc1008 to try to get its RAID redone - T247787 [production]
00:31 <Amir1> foreachwikiindblist medium deleteEqualMessages.php --delete (T247562) [production]
00:10 <crusnov@deploy1001> Finished deploy [netbox/deploy@14256f9]: netbox 2.7.10 upgrade (duration: 02m 29s) [production]
00:08 <crusnov@deploy1001> Started deploy [netbox/deploy@14256f9]: netbox 2.7.10 upgrade [production]
00:07 <crusnov@deploy1001> Finished deploy [netbox/deploy@14256f9]: netbox 2.7.10 upgrade (duration: 01m 17s) [production]
00:06 <crusnov@deploy1001> Started deploy [netbox/deploy@14256f9]: netbox 2.7.10 upgrade [production]
2020-03-17 §
22:49 <Amir1> warming up cache for Q80M to Q88M for new term store on db1111, db1126, db1104, db1092 (T219123) [production]
22:17 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@0adead4]: Update mobileapps to ec6fd6e (duration: 06m 08s) [production]
22:11 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@0adead4]: Update mobileapps to ec6fd6e [production]
21:54 <Krinkle> krinkle@mw2170$ disable-puppet (Testing for T99740) [production]
21:15 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: WikimediaEditorTasks: Enable Depicts counting (again) (T247874) (duration: 01m 07s) [production]
21:10 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings.php: WikimediaEditorTasks: Enable Depicts counting (T247874) (duration: 01m 07s) [production]
20:50 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.23/extensions/WikimediaEditorTasks: Fix revert counting for non-language-specific counters, take 2 (T244974) (duration: 01m 12s) [production]
20:33 <mutante> boron - systemctl start docker-reporter-k8s-images ; systemctl start docker-reporter-releng-images [production]
20:31 <mutante> boron - had degraded systemd state in Icinga - systemctl start docker-reporter-base-images [production]
19:54 <mutante> miscweb1001 - restarted ferm, reverted live hack [production]
19:53 <ppchelko@deploy1001> Finished deploy [restbase/deploy@8db09ed]: Various PCS endpoints additions and fixes T247295 T247096 T244175 (duration: 14m 31s) [production]
19:51 <mutante> miscweb1001 - testing if ferm 80 firewall hole is needed for envoy, temp. disabled puppet, restarted ferm [production]
19:38 <ppchelko@deploy1001> Started deploy [restbase/deploy@8db09ed]: Various PCS endpoints additions and fixes T247295 T247096 T244175 [production]
19:01 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q80M (T219123)]], take II (duration: 01m 06s) [production]
19:00 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:579925|Set up read new term store up to Q80M (T219123)]] (duration: 01m 07s) [production]
18:53 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.24/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseItemTermStoreWriter.php: [[gerrit:580390|Do not lock rows when there's no term returned (T247553 T246898)]], To catch the train (duration: 01m 08s) [production]
18:50 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
18:45 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
18:45 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
18:41 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
18:39 <mutante> removing mw1238 through mw1243 - decom with cookbook (T247780 T245099) [production]
18:38 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
18:38 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
18:37 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
18:35 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw123[8-9].eqiad.wmnet [production]
18:35 <dzahn@cumin1001> conftool action : set/pooled=inactive; selector: name=mw124[0-3].eqiad.wmnet [production]
18:29 <otto@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
18:01 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@b6bff94]: Update mobileapps to 3c73ca3 (duration: 06m 06s) [production]
18:00 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
17:58 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
17:56 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.23/languages/LanguageConverter.php: [[gerrit:580361|languages: Don't assume in LanguageConverter (T235360)]] (duration: 01m 07s) [production]
17:55 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@b6bff94]: Update mobileapps to 3c73ca3 [production]
17:55 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
17:53 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw124[0-3].eqiad.wmnet [production]
17:53 <dzahn@cumin1001> conftool action : set/pooled=no; selector: name=mw123[89].eqiad.wmnet [production]
17:52 <Amir1> warming up cache for Q70M to Q80M for new term store on db1111, db1126, db1104, db1092 (T219123) [production]
17:46 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.23/extensions/Wikibase/lib/includes/Store/Sql/Terms/DatabaseItemTermStoreWriter.php: [[gerrit:580352|Do not lock rows when there's no term returned (T247553 T246898)]] (duration: 01m 07s) [production]
17:42 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
17:40 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
17:37 <ejegg> updated payments-wiki from 86ce0361f9 to 72856949a1 [production]