3201-3250 of 10000 results (32ms)
2020-11-01 §
01:26 <rzl@cumin1001> START - Cookbook sre.hosts.downtime [production]
01:16 <rzl@cumin1001> dbctl commit (dc=all): 'Depool db1091', diff saved to https://phabricator.wikimedia.org/P13124 and previous config saved to /var/cache/conftool/dbconfig/20201101-011600-rzl.json [production]
2020-10-31 §
00:12 <mutante> removed Nuria from wmf group, she is already in nda group (T266086) [production]
2020-10-30 §
23:35 <foks> removing two files for legal compliance [production]
23:32 <mutante> adding query.wikidata.org to TLS cert for webserver-misc-apps.discovery.wmnet T266702 [production]
23:05 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
23:04 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:04 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:03 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:03 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:03 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
23:03 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
23:02 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:02 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:02 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:01 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:01 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:01 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:01 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
21:02 <jiji@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
21:00 <jiji@cumin2001> START - Cookbook sre.hosts.downtime [production]
20:59 <mutante> mw1267,mw1268 - scap pull and repool - back to prod - T266164 [production]
20:57 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1267.eqiad.wmnet [production]
20:57 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1268.eqiad.wmnet [production]
20:56 <mutante> mw1267,mw1268 - scap pull [production]
20:33 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
20:32 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:31 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
20:06 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
20:04 <robh@cumin1001> START - Cookbook sre.hosts.downtime [production]
18:48 <cdanis> the above scap began (and mostly finished) several minutes ago but is hanging on a couple hosts down for maintenance [production]
18:48 <cdanis@deploy1001> Synchronized wmf-config/InitialiseSettings.php: lower frwiki featured feeds limit 1a41ef634 T266865 (duration: 05m 14s) [production]
18:47 <cdanis> ✔️ cdanis@deploy1001.eqiad.wmnet /srv/mediawiki-staging 🕝☕ scap sync-file wmf-config/InitialiseSettings.php 'lower frwiki featured feeds limit 1a41ef634 T266865' [production]
18:27 <hashar@deploy1001> Finished deploy [integration/docroot@c35e5e9]: Add ECS to doc.wikimedia.org index (duration: 00m 06s) [production]
18:27 <hashar@deploy1001> Started deploy [integration/docroot@c35e5e9]: Add ECS to doc.wikimedia.org index [production]
17:38 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
17:36 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]