551-600 of 10000 results (28ms)
2020-10-30 §
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:06 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:04 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:48 <cdanis> the above scap began (and mostly finished) several minutes ago but is hanging on a couple hosts down for maintenance [production]
18:48 <cdanis@deploy1001> Synchronized wmf-config/InitialiseSettings.php: lower frwiki featured feeds limit 1a41ef634 T266865 (duration: 05m 14s) [production]
18:47 <cdanis> ✔️ cdanis@deploy1001.eqiad.wmnet /srv/mediawiki-staging 🕝☕ scap sync-file wmf-config/InitialiseSettings.php 'lower frwiki featured feeds limit 1a41ef634 T266865' [production]
18:27 <hashar@deploy1001> Finished deploy [integration/docroot@c35e5e9]: Add ECS to doc.wikimedia.org index (duration: 00m 06s) [production]
18:27 <hashar@deploy1001> Started deploy [integration/docroot@c35e5e9]: Add ECS to doc.wikimedia.org index [production]
17:38 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:36 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:22 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:22 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
17:19 <effie> disable puppet on mc1036 and mc2036 - T252391 [production]
17:18 <effie> enable puppet on all mediawiki and mc* hosts [production]
16:19 <elukey> kafka-jumbo1006 still running with 1g nick [production]
15:36 <effie> stopping puppet on mediawiki and mc* hosts [production]
15:11 <rzl@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:11 <rzl@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:09 <rzl> downtiming mc2036 for buster reimage [production]
14:42 <elukey> stop kafka-jumbo1006 to swap NICs (1g -> 10g, d1 -> d4 rack) [production]
14:14 <cmjohnson1> moving mw1267 and mw168 to rack A8 eqiad T266164 [production]
12:29 <XioNoX> set normal VRRP balancing on cr2-eqiad [production]
10:08 <klausman@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:08 <klausman@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:02 <ladsgroup@deploy1001> Synchronized static/images/project-logos: Revert: Changing logo of Wikidata for the brithday (duration: 01m 12s) [production]
09:13 <elukey@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:07 <elukey@cumin1001> START - Cookbook sre.dns.netbox [production]
08:58 <elukey@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
08:54 <elukey> decom an-tool1006 (old analytics test vm) - T255139 [production]
08:53 <elukey@cumin1001> START - Cookbook sre.hosts.decommission [production]
2020-10-29 §
23:59 <eileen> process-control config revision is 6891d35bce [production]
23:39 <Urbanecm> Evening B&C window done [production]
23:38 <Urbanecm> urbanecm@mwmaint1002:~$ mwscript namespaceDupes.php --wiki=trwikiquote --add-prefix=BROKEN --fix # T266605 # P13112 [production]
23:37 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: ddb7e08e9c1d07f704c9f7585d8b6089f1895b5c: Add namespace aliases to Turkish Wikiquote (T266605) (duration: 00m 57s) [production]
23:36 <eileen> process-control config revision is 1114512f90 [production]
23:29 <Urbanecm> urbanecm@mwmaint1002:~$ mwscript namespaceDupes.php --wiki=trwikisource --add-prefix=BROKEN --fix # T266606 # P13111 [production]
23:27 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: c3a8555154673c4c5a65f6ec2a1219d0832f48e0: Add namespace aliases to Turkish Wikisource (T266606) (duration: 00m 56s) [production]
23:23 <Urbanecm> urbanecm@mwmaint1002:~$ mwscript namespaceDupes.php --wiki=trwikibooks --fix # T266608 [production]