3501-3550 of 10000 results (73ms)
2019-12-21 §
22:45 <volans> powercycle cp3051 - T241306 [production]
13:39 <Amir1> ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=testwikidatawiki --sleep 2 --batch-size=50 (T241209) [production]
04:01 <mutante> LDAP - added ifried to wmf (T240988) for Turnilo / Superset access [production]
01:44 <reedy@deploy1001> update-interwiki-cache aborted: Update interwiki cache (duration: 00m 07s) [production]
01:39 <reedy@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T188369 (duration: 00m 53s) [production]
01:05 <volker-e@deploy1001> Finished deploy [design/style-guide@b61669a]: Deploy design/style-guide: (duration: 00m 07s) [production]
01:05 <volker-e@deploy1001> Started deploy [design/style-guide@b61669a]: Deploy design/style-guide: [production]
2019-12-20 §
23:10 <Amir1> ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=testwikidatawiki --sleep 2 --batch-size=10 (T241209) [production]
23:05 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@8373b0d]: Correct upload metadata usage (duration: 06m 41s) [production]
22:59 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@8373b0d]: Correct upload metadata usage [production]
22:52 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@d99bebf]: force utf-8 encoding when not detected (duration: 05m 22s) [production]
22:47 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@d99bebf]: force utf-8 encoding when not detected [production]
21:58 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@0ad64e7]: properly decode unicode on ltr model upload (duration: 06m 04s) [production]
21:52 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@0ad64e7]: properly decode unicode on ltr model upload [production]
18:44 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.11/extensions/WikimediaEditorTasks: Fix: Get RevisionRecord directly from the Revision in onPageContentSaveComplete (T241014) (duration: 00m 54s) [production]
17:48 <ejegg> updated fundraising internal dashboard from b75f9074de to 852f6871cf [production]
17:29 <bd808> Getting cloudweb2001-dev caught up with the MW train via a manual `scap pull` (T241251) [production]
17:19 <volans> re-generating sre-bot ro-token for Netbox, accidentally leaked to gerrit, already revoked within a minute from leak [production]
16:28 <cdanis> ✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕦☕ homer 'cr*' commit 'templatize BGP sessions with pmacct netflow collector cb096f509 0f56b2233 2e050ad33' [production]
16:27 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Ia9190a4e5, T240691: Disable wgExtractsExtendOpenSearchXml (duration: 00m 55s) [production]
14:38 <volans> temporarily disable puppet on netbox[12]001 to deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/555715 - T233183 [production]
14:14 <cdanis> homer 'cr*esams*' commit 'I022c62120 enable netflow collection in esams' [production]
13:16 <bblack> [correction] esams+eqsin edges switching back to digicert-2019a unified TLS cert [production]
13:15 <bblack> esams+eqiad edges switching back to digicert-2019a unified TLS cert [production]
12:43 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:43 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:43 <aborrero@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
12:42 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:00 <moritzm> installing glib2.0 security updates on stretch [production]
09:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Adjust main traffic weight for db1096:3316 and db1098:3316', diff saved to https://phabricator.wikimedia.org/P9998 and previous config saved to /var/cache/conftool/dbconfig/20191220-094016-marostegui.json [production]
09:32 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:32 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:31 <moritzm> applied Ganeti cluster setting to pass through CPU flags for MDS/SSBD to esams/ulsfo clusters T226444 T236216 [production]
09:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1096:3315 db1096:3316', diff saved to https://phabricator.wikimedia.org/P9997 and previous config saved to /var/cache/conftool/dbconfig/20191220-092805-marostegui.json [production]
09:26 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:26 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1096:3315 db1096:3316', diff saved to https://phabricator.wikimedia.org/P9996 and previous config saved to /var/cache/conftool/dbconfig/20191220-091934-marostegui.json [production]
09:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1096:3315 db1096:3316', diff saved to https://phabricator.wikimedia.org/P9995 and previous config saved to /var/cache/conftool/dbconfig/20191220-091050-marostegui.json [production]
09:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1096:3315 db1096:3316', diff saved to https://phabricator.wikimedia.org/P9994 and previous config saved to /var/cache/conftool/dbconfig/20191220-090204-marostegui.json [production]
08:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1096:3315, db1096:3316 for upgrade', diff saved to https://phabricator.wikimedia.org/P9993 and previous config saved to /var/cache/conftool/dbconfig/20191220-085300-marostegui.json [production]
08:51 <addshore> addshore@graphite1004&2003:~$ sudo -u _graphite find /var/lib/carbon/whisper/daily/wikidata/api/actions -delete # T227594 [production]
08:45 <moritzm> failover ganeti master on the new esams cluster [production]
08:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1103:3312, db1103:3314', diff saved to https://phabricator.wikimedia.org/P9992 and previous config saved to /var/cache/conftool/dbconfig/20191220-080400-marostegui.json [production]
07:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1103:3312, db1103:3314', diff saved to https://phabricator.wikimedia.org/P9991 and previous config saved to /var/cache/conftool/dbconfig/20191220-075434-marostegui.json [production]
07:45 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1103:3312, db1103:3314', diff saved to https://phabricator.wikimedia.org/P9990 and previous config saved to /var/cache/conftool/dbconfig/20191220-074504-marostegui.json [production]
07:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1103:3312, db1103:3314', diff saved to https://phabricator.wikimedia.org/P9989 and previous config saved to /var/cache/conftool/dbconfig/20191220-073017-marostegui.json [production]
06:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1103:3312, db1103:3314 for upgrade', diff saved to https://phabricator.wikimedia.org/P9988 and previous config saved to /var/cache/conftool/dbconfig/20191220-065753-marostegui.json [production]
06:56 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1089', diff saved to https://phabricator.wikimedia.org/P9987 and previous config saved to /var/cache/conftool/dbconfig/20191220-065628-marostegui.json [production]
06:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1089', diff saved to https://phabricator.wikimedia.org/P9986 and previous config saved to /var/cache/conftool/dbconfig/20191220-065022-marostegui.json [production]
06:41 <mutante> netflow3001 - rebooting [production]