1001-1050 of 10000 results (30ms)
2021-06-21 ยง
13:12 <elukey> upload istioctl 1.9.5 to {buster,stretch}-wikimedia [production]
13:12 <dcaro@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 40 hosts with reason: Merged broken patch [production]
13:12 <dcaro@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on 40 hosts with reason: Merged broken patch [production]
13:09 <klausman> starting update of ML team's etcd machines in codfw [production]
12:55 <godog> move librenms alerts with "max alerts" == -1 to "interval" being 15m - T285205 [production]
12:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 100%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16672 and previous config saved to /var/cache/conftool/dbconfig/20210621-124030-root.json [production]
12:39 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 100%: Repool db1135 after schema change', diff saved to https://phabricator.wikimedia.org/P16671 and previous config saved to /var/cache/conftool/dbconfig/20210621-123906-root.json [production]
12:35 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/Wikibase: Backport: [[gerrit:700528|Rewrite SerializationModifier to be more efficient]] (duration: 01m 02s) [production]
12:25 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps1010.eqiad.wmnet [production]
12:25 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 75%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16670 and previous config saved to /var/cache/conftool/dbconfig/20210621-122526-root.json [production]
12:24 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 75%: Repool db1135 after schema change', diff saved to https://phabricator.wikimedia.org/P16669 and previous config saved to /var/cache/conftool/dbconfig/20210621-122403-root.json [production]
12:19 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host maps1010.eqiad.wmnet [production]
12:12 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps2008.codfw.wmnet [production]
12:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 50%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16668 and previous config saved to /var/cache/conftool/dbconfig/20210621-121023-root.json [production]
12:10 <godog> bump space for k8s and ops prometheus on prometheus1004 (prometheus1003 has been expanded previously but not logged) [production]
12:08 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 50%: Repool db1135 after schema change', diff saved to https://phabricator.wikimedia.org/P16667 and previous config saved to /var/cache/conftool/dbconfig/20210621-120859-root.json [production]
11:57 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host maps2008.codfw.wmnet [production]
11:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 25%: Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P16665 and previous config saved to /var/cache/conftool/dbconfig/20210621-115519-root.json [production]
11:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1099:3311 T283499', diff saved to https://phabricator.wikimedia.org/P16664 and previous config saved to /var/cache/conftool/dbconfig/20210621-115441-marostegui.json [production]
11:53 <marostegui@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 25%: Repool db1135 after schema change', diff saved to https://phabricator.wikimedia.org/P16663 and previous config saved to /var/cache/conftool/dbconfig/20210621-115355-root.json [production]
11:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1135 T283499', diff saved to https://phabricator.wikimedia.org/P16662 and previous config saved to /var/cache/conftool/dbconfig/20210621-115143-marostegui.json [production]
11:23 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 0bf35e0d3934a2c44e95fc4d7bc30a2a6c74c59a: Disable indexing user (sub)pages and draft-related pages on hrwiki (T284384) (duration: 00m 56s) [production]
11:21 <urbanecm@deploy1002> Synchronized logos/config.yaml: 1b97376dea470122f9da9ea8dddde7e8fe94231f: Change vi.wikisource logo to the same logo being used at en.wikisource (T284612) (duration: 00m 56s) [production]
11:20 <urbanecm@deploy1002> Synchronized wmf-config/logos.php: 1b97376dea470122f9da9ea8dddde7e8fe94231f: Change vi.wikisource logo to the same logo being used at en.wikisource (T284612) (duration: 00m 57s) [production]
11:17 <oblivian@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
11:16 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 464cc0bb0d9ff6c47e2a90cba5232e9506ff5dec: ptwikinews: Remove NS ID 102,103 (T285163) (duration: 00m 56s) [production]
11:11 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:700160|Add WMCS public addresses to $wgSoftBlockRanges]] (duration: 00m 56s) [production]
11:04 <jbond@deploy1002> Finished deploy [netbox/deploy@977f7b6]: deploy v2.10.4-wmf4 (duration: 02m 53s) [production]
11:01 <jbond@deploy1002> Started deploy [netbox/deploy@977f7b6]: deploy v2.10.4-wmf4 [production]
10:55 <moritzm> restarting FPM on mw canaries to pick up nettle security updates [production]
10:45 <volans@deploy1002> Finished deploy [netbox/deploy@977f7b6]: Force re-creation of the virtualenv (duration: 00m 54s) [production]
10:45 <moritzm> installing nettle security updates on buster [production]
10:44 <volans@deploy1002> Started deploy [netbox/deploy@977f7b6]: Force re-creation of the virtualenv [production]
10:44 <volans@deploy1002> Finished deploy [netbox/deploy@977f7b6]: Force re-creation of the virtualenv (duration: 00m 54s) [production]
10:43 <volans@deploy1002> Started deploy [netbox/deploy@977f7b6]: Force re-creation of the virtualenv [production]
10:41 <volans@deploy1002> Finished deploy [netbox/deploy@977f7b6]: Force re-creation of the virtualenv (duration: 00m 50s) [production]
10:41 <jdrewniak@deploy1002> Synchronized portals: Wikimedia Portals Update: [[gerrit:692281| Bumping portals to master (T128546)]] (duration: 00m 56s) [production]
10:40 <volans@deploy1002> Started deploy [netbox/deploy@977f7b6]: Force re-creation of the virtualenv [production]
10:40 <jdrewniak@deploy1002> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:692281| Bumping portals to master (T128546)]] (duration: 00m 56s) [production]
10:37 <jbond@deploy1002> Finished deploy [netbox/deploy@977f7b6]: deploy v2.10.4-wmf4 (duration: 02m 22s) [production]
10:36 <jbond@deploy1002> Started deploy [netbox/deploy@977f7b6]: deploy v2.10.4-wmf4 [production]
10:36 <jbond@deploy1002> Finished deploy [netbox/deploy@977f7b6]: deploy v2.10.4-wmf4 to netbox-next (duration: 00m 56s) [production]
10:29 <jbond@deploy1002> Started deploy [netbox/deploy@977f7b6]: deploy v2.10.4-wmf4 to netbox-next [production]
10:27 <jbond@deploy1002> Finished deploy [netbox/deploy@6b69f2c]: deploy v2.10.4-wmf4 to netbox-next (duration: 03m 12s) [production]
10:24 <jbond@deploy1002> Started deploy [netbox/deploy@6b69f2c]: deploy v2.10.4-wmf4 to netbox-next [production]
10:22 <jbond@deploy1002> Finished deploy [netbox/deploy@c7762f8]: deploy v2.10.4-wmf4 to netbox-next (duration: 02m 22s) [production]
10:20 <jbond@deploy1002> Started deploy [netbox/deploy@c7762f8]: deploy v2.10.4-wmf4 to netbox-next [production]
10:19 <jbond@deploy1002> Finished deploy [netbox/deploy@c7762f8]: deploy v2.10.4-wmf4 to netbox-next (duration: 02m 13s) [production]
10:17 <jbond@deploy1002> Started deploy [netbox/deploy@c7762f8]: deploy v2.10.4-wmf4 to netbox-next [production]
10:16 <jbond@deploy1002> Finished deploy [netbox/deploy@c7762f8]: deploy v2.10.4-wmf4 to netbox-next (duration: 01m 03s) [production]