3251-3300 of 10000 results (37ms)
2021-02-22 ยง
16:19 <urbanecm@deploy1001> Synchronized langlist: Creating mniwiki (T273456) (duration: 00m 54s) [production]
16:18 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Creating mniwiki (T273456) (duration: 00m 56s) [production]
16:17 <urbanecm@deploy1001> Synchronized wmf-config/logos.php: Creating mniwiki (T273456) (duration: 00m 56s) [production]
16:15 <urbanecm@deploy1001> Synchronized static/images/project-logos/: Creating mniwiki (T273456) (duration: 00m 55s) [production]
16:14 <urbanecm@deploy1001> rebuilt and synchronized wikiversions files: Creating mniwiki (T273456) [production]
16:13 <urbanecm@deploy1001> Synchronized dblists: Creating mniwiki (T273456) (duration: 00m 57s) [production]
16:12 <urbanecm@deploy1001> Synchronized wmf-config/db-codfw.php: Creating mniwiki (T273456) (duration: 00m 55s) [production]
16:11 <urbanecm@deploy1001> Synchronized wmf-config/db-eqiad.php: Creating mniwiki (T273456) (duration: 00m 56s) [production]
16:08 <urbanecm@deploy1001> Synchronized langlist: Creating altwiki (T271980) (duration: 00m 55s) [production]
16:03 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Creating altwiki (T271980) (duration: 00m 55s) [production]
16:01 <urbanecm@deploy1001> rebuilt and synchronized wikiversions files: Creating altwiki (T271980) [production]
16:00 <urbanecm@deploy1001> Synchronized dblists: Creating altwiki (T271980) (duration: 00m 54s) [production]
15:59 <urbanecm@deploy1001> Synchronized wmf-config/db-codfw.php: Creating altwiki (T271980) (duration: 00m 59s) [production]
15:57 <Urbanecm> Temporarily replace /srv/mediawiki/php-1.36.0-wmf.31/extensions/WikimediaMaintenance/addWiki.php with /home/urbanecm/addWiki.php at mwmaint1002 to unbreak addWiki.php [production]
15:53 <jayme@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
15:43 <urbanecm@deploy1001> Synchronized wmf-config/db-eqiad.php: Creating altwiki (T271980) (duration: 00m 56s) [production]
15:32 <jayme@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
14:16 <herron> roll restarting kafkamon hosts for updates [production]
13:57 <filippo@cumin1001> conftool action : set/pooled=false; selector: dnsdisc=swift,name=codfw [production]
13:47 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus4001.ulsfo.wmnet [production]
13:40 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.31/extensions/ContentTranslation/app/: f9e823e: CX3 Build 0.1.0+20210216 (fixes missing bits in T271397) (duration: 00m 55s) [production]
13:40 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus3001.esams.wmnet [production]
13:37 <moritzm> installing openldap security updates on corp replicas [production]
13:36 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.31/extensions/FlaggedRevs/extension.json: a4cd98e7a581fe18634da05ba04eaf8035023c26: Grant sysops review and unreviewed pages right by default (apparently i forgot to rebase the first time, resync; T275293) (duration: 00m 57s) [production]
13:32 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus4001.ulsfo.wmnet [production]
13:31 <godog> reset-failed ifup@ens14 on prometheus3001 - T273026 [production]
13:30 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cuminunpriv1001.eqiad.wmnet [production]
13:29 <akosiaris> repool sessionstore in eqiad after sessionstore certificate refresh. T274564 [production]
13:29 <akosiaris@cumin1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=sessionstore [production]
13:27 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus3001.esams.wmnet [production]
13:26 <jmm@cumin2001> START - Cookbook sre.hosts.reboot-single for host cuminunpriv1001.eqiad.wmnet [production]
13:17 <dcaro@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on cloudvirt-wdqs[1001-1003].eqiad.wmnet with reason: Restarting cloudcanary instances [production]
13:16 <dcaro@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on cloudvirt-wdqs[1001-1003].eqiad.wmnet with reason: Restarting cloudcanary instances [production]
13:16 <dcaro@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on 27 hosts with reason: Restarting cloudcanary instances [production]
13:16 <dcaro@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on 27 hosts with reason: Restarting cloudcanary instances [production]
13:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1175 (re)pooling @ 100%: Slowly repool db1175', diff saved to https://phabricator.wikimedia.org/P14439 and previous config saved to /var/cache/conftool/dbconfig/20210222-131153-root.json [production]
12:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1175 (re)pooling @ 75%: Slowly repool db1175', diff saved to https://phabricator.wikimedia.org/P14438 and previous config saved to /var/cache/conftool/dbconfig/20210222-125650-root.json [production]
12:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1175 (re)pooling @ 50%: Slowly repool db1175', diff saved to https://phabricator.wikimedia.org/P14437 and previous config saved to /var/cache/conftool/dbconfig/20210222-124146-root.json [production]
12:32 <akosiaris@cumin1001> conftool action : set/pooled=false; selector: name=eqiad,dnsdisc=sessionstore [production]
12:28 <akosiaris@cumin1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=sessionstore [production]
12:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1175 (re)pooling @ 25%: Slowly repool db1175', diff saved to https://phabricator.wikimedia.org/P14436 and previous config saved to /var/cache/conftool/dbconfig/20210222-122643-root.json [production]
12:24 <urbanecm@deploy1001> Synchronized wmf-config//throttle.php: d806f3a986244f8027aba730e72d99babe3b37e9: Add a throttle rule for for edit-a-thon (T275237) (duration: 00m 54s) [production]
12:22 <akosiaris> depool sessionstore in eqiad for sessionstore certificate refresh. T274564 [production]
12:21 <akosiaris> repool sessionstore in codfw after sessionstore certificate refresh. T274564 [production]
12:21 <akosiaris@cumin1001> conftool action : set/pooled=true; selector: name=codfw,dnsdisc=sessionstore [production]
12:20 <urbanecm@deploy1001> Synchronized php-1.36.0-wmf.31/extensions/FlaggedRevs/extension.json: a4cd98e7a581fe18634da05ba04eaf8035023c26: Grant sysops review and unreviewed pages right by default (T275293) (duration: 00m 55s) [production]
12:18 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 7bd26dc6160a5bc3ba9235ce93c01e7ab9744487: Add inaturalist-open-data.s3.amazonaws.com to copyupload list (T275318) (duration: 00m 56s) [production]
12:15 <urbanecm@deploy1001> Synchronized wmf-config/abusefilter.php: 391900b8db9ffdee8565d82c38c089843876a27b: ukwikivoyage: Enable block AbuseFilter action (T275271) (duration: 00m 55s) [production]
12:11 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: a1f8ce48249ad457d79c57e27836ee492eb00427: Enable Section Translation on Bengali Wikipedia (T271397) (duration: 00m 56s) [production]
12:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1175 (re)pooling @ 10%: Slowly repool db1175', diff saved to https://phabricator.wikimedia.org/P14435 and previous config saved to /var/cache/conftool/dbconfig/20210222-121139-root.json [production]