1801-1850 of 10000 results (43ms)
2022-03-07 ยง
14:28 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1085.eqiad.wmnet with reason: host reimage [production]
14:28 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica1004.wikimedia.org [production]
14:27 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host failoid2002.codfw.wmnet [production]
14:26 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica1004.wikimedia.org [production]
14:25 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp1085.eqiad.wmnet with reason: host reimage [production]
14:23 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetboard1002.eqiad.wmnet [production]
14:22 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica1003.wikimedia.org [production]
14:21 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host puppetboard1002.eqiad.wmnet [production]
14:20 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica1003.wikimedia.org [production]
14:19 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 10%: repooling after schema change', diff saved to https://phabricator.wikimedia.org/P21992 and previous config saved to /var/cache/conftool/dbconfig/20220307-141915-root.json [production]
14:19 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 100%: repooling after schema change', diff saved to https://phabricator.wikimedia.org/P21991 and previous config saved to /var/cache/conftool/dbconfig/20220307-141911-root.json [production]
14:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:18 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 64b128459f04514cd0093745d7a83166555449b2: Enable reply tool by default on enwiki (T296645) (duration: 00m 49s) [production]
14:18 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142', diff saved to https://phabricator.wikimedia.org/P21990 and previous config saved to /var/cache/conftool/dbconfig/20220307-141724-ladsgroup.json [production]
14:16 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:16 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 8f20ec9f5bd7f507580d8e8860116e3b1842ac9a: fawiki: Disable creating community books and remove "Create a book" link from sidebar (T303173) (duration: 00m 49s) [production]
14:15 <vgutierrez@cumin1001> START - Cookbook sre.hosts.reimage for host cp4030.ulsfo.wmnet with OS buster [production]
14:15 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica2006.wikimedia.org [production]
14:13 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica2006.wikimedia.org [production]
14:11 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:10 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:09 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:09 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica2005.wikimedia.org [production]
14:09 <vgutierrez@cumin1001> START - Cookbook sre.hosts.reimage for host cp1085.eqiad.wmnet with OS buster [production]
14:08 <urbanecm@deploy1002> Synchronized logos/config.yaml: 8619f5933966071cdb39097a3e0d38fdead40b66: etwikiquote: Update logo (T302683; 3/3) (duration: 00m 49s) [production]
14:07 <urbanecm@deploy1002> Synchronized wmf-config/logos.php: 8619f5933966071cdb39097a3e0d38fdead40b66: etwikiquote: Update logo (T302683; 2/3) (duration: 00m 49s) [production]
14:07 <urbanecm> Purge https://en.wikipedia.org/static/images/project-logos/etwikiquote.png (T302683) [production]
14:07 <urbanecm@deploy1002> Synchronized static/images/project-logos/: 8619f5933966071cdb39097a3e0d38fdead40b66: etwikiquote: Update logo (T302683; 1/3) (duration: 00m 50s) [production]
14:05 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica2005.wikimedia.org [production]
14:05 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetboard2002.codfw.wmnet [production]
14:04 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 75%: repooling after schema change', diff saved to https://phabricator.wikimedia.org/P21989 and previous config saved to /var/cache/conftool/dbconfig/20220307-140408-root.json [production]
14:02 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp2037.codfw.wmnet with OS buster [production]
14:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142', diff saved to https://phabricator.wikimedia.org/P21988 and previous config saved to /var/cache/conftool/dbconfig/20220307-140219-ladsgroup.json [production]
14:02 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host people2002.codfw.wmnet [production]
14:00 <kormat> removing cumin2001 grants from all db sections T276589 [production]
14:00 <vgutierrez> pool cp2037 with HAProxy as TLS termination layer - T290005 [production]
14:00 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host puppetboard2002.codfw.wmnet [production]
13:59 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host people2002.codfw.wmnet [production]
13:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T300992)', diff saved to https://phabricator.wikimedia.org/P21987 and previous config saved to /var/cache/conftool/dbconfig/20220307-135614-ladsgroup.json [production]
13:49 <marostegui@cumin1001> dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 50%: repooling after schema change', diff saved to https://phabricator.wikimedia.org/P21986 and previous config saved to /var/cache/conftool/dbconfig/20220307-134904-root.json [production]
13:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1127 (T300381)', diff saved to https://phabricator.wikimedia.org/P21985 and previous config saved to /var/cache/conftool/dbconfig/20220307-134848-marostegui.json [production]
13:48 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1127.eqiad.wmnet with reason: Maintenance [production]
13:48 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1127.eqiad.wmnet with reason: Maintenance [production]
13:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1174 (T300381)', diff saved to https://phabricator.wikimedia.org/P21984 and previous config saved to /var/cache/conftool/dbconfig/20220307-134840-marostegui.json [production]
13:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142 (T302950)', diff saved to https://phabricator.wikimedia.org/P21983 and previous config saved to /var/cache/conftool/dbconfig/20220307-134715-ladsgroup.json [production]
13:47 <aqu@deploy1002> Finished deploy [analytics/refinery@51d074b] (hadoop-test): Migrate wikidata/item_page_link/weekly from Oozie to Airflow [analytics/refinery@51d074b] (duration: 07m 17s) [production]
13:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P21982 and previous config saved to /var/cache/conftool/dbconfig/20220307-134109-ladsgroup.json [production]