1-50 of 10000 results (74ms)
2024-02-13 ยง
23:55 <dzahn@cumin1002> START - Cookbook sre.hosts.reimage for host ncmonitor1001.eqiad.wmnet with OS bookworm [production]
21:23 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1190 (T352010)', diff saved to https://phabricator.wikimedia.org/P56734 and previous config saved to /var/cache/conftool/dbconfig/20240213-212343-ladsgroup.json [production]
21:23 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1190.eqiad.wmnet with reason: Maintenance [production]
21:23 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1190.eqiad.wmnet with reason: Maintenance [production]
21:23 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1160 (T352010)', diff saved to https://phabricator.wikimedia.org/P56733 and previous config saved to /var/cache/conftool/dbconfig/20240213-212321-ladsgroup.json [production]
21:08 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1160', diff saved to https://phabricator.wikimedia.org/P56732 and previous config saved to /var/cache/conftool/dbconfig/20240213-210814-ladsgroup.json [production]
21:08 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2190 (T352010)', diff saved to https://phabricator.wikimedia.org/P56731 and previous config saved to /var/cache/conftool/dbconfig/20240213-210813-ladsgroup.json [production]
20:59 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in search_codfw [production]
20:59 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in search_codfw [production]
20:53 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1160', diff saved to https://phabricator.wikimedia.org/P56730 and previous config saved to /var/cache/conftool/dbconfig/20240213-205308-ladsgroup.json [production]
20:53 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2190', diff saved to https://phabricator.wikimedia.org/P56729 and previous config saved to /var/cache/conftool/dbconfig/20240213-205307-ladsgroup.json [production]
20:38 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2190', diff saved to https://phabricator.wikimedia.org/P56728 and previous config saved to /var/cache/conftool/dbconfig/20240213-203800-ladsgroup.json [production]
20:23 <mutante> phab1004 - running public_task_dump.py T355502 [production]
20:22 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2190 (T352010)', diff saved to https://phabricator.wikimedia.org/P56727 and previous config saved to /var/cache/conftool/dbconfig/20240213-202254-ladsgroup.json [production]
20:22 <brennen@deploy2002> Finished deploy [phabricator/deployment@f4a7f50]: deploy to phab1004 for T357464 (duration: 00m 48s) [production]
20:21 <brennen@deploy2002> Started deploy [phabricator/deployment@f4a7f50]: deploy to phab1004 for T357464 [production]
20:20 <brennen@deploy2002> Finished deploy [phabricator/deployment@f4a7f50]: test deploy to phab2002 for T357464 (duration: 00m 29s) [production]
20:20 <brennen@deploy2002> Started deploy [phabricator/deployment@f4a7f50]: test deploy to phab2002 for T357464 [production]
20:08 <eileen> civicrm upgraded from ac69725f to 497e0899 [production]
19:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2190 (T352010)', diff saved to Unable to send diff to phaste and previous config saved to /var/cache/conftool/dbconfig/20240213-195724-ladsgroup.json [production]
19:57 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2190.codfw.wmnet with reason: Maintenance [production]
19:57 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2190.codfw.wmnet with reason: Maintenance [production]
19:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T352010)', diff saved to https://phabricator.wikimedia.org/P56725 and previous config saved to /var/cache/conftool/dbconfig/20240213-195701-ladsgroup.json [production]
19:41 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P56724 and previous config saved to /var/cache/conftool/dbconfig/20240213-194155-ladsgroup.json [production]
19:26 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P56723 and previous config saved to /var/cache/conftool/dbconfig/20240213-192648-ladsgroup.json [production]
19:11 <brennen@deploy2002> rebuilt and synchronized wikiversions files: group0 wikis to 1.42.0-wmf.18 refs T354436 [production]
19:11 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T352010)', diff saved to https://phabricator.wikimedia.org/P56722 and previous config saved to /var/cache/conftool/dbconfig/20240213-191142-ladsgroup.json [production]
19:01 <brennen> train 1.42.0-wmf.18 (T354436): no current blockers, rolling to group0. [production]
18:43 <brouberol@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-test-ui1001.eqiad.wmnet with OS bullseye [production]
18:41 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2177 (T352010)', diff saved to https://phabricator.wikimedia.org/P56721 and previous config saved to /var/cache/conftool/dbconfig/20240213-184159-ladsgroup.json [production]
18:41 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
18:41 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
18:41 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T352010)', diff saved to https://phabricator.wikimedia.org/P56720 and previous config saved to /var/cache/conftool/dbconfig/20240213-184137-ladsgroup.json [production]
18:26 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P56718 and previous config saved to /var/cache/conftool/dbconfig/20240213-182630-ladsgroup.json [production]
18:11 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P56717 and previous config saved to /var/cache/conftool/dbconfig/20240213-181124-ladsgroup.json [production]
18:01 <jclark@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
18:00 <ladsgroup@deploy2002> Finished scap: Backport for [[gerrit:992206|ruwiki: Add 'edituserjson' right to 'engineers' group (T355499)]] (duration: 08m 28s) [production]
17:56 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T352010)', diff saved to https://phabricator.wikimedia.org/P56716 and previous config saved to /var/cache/conftool/dbconfig/20240213-175617-ladsgroup.json [production]
17:53 <ladsgroup@deploy2002> ammarpad and ladsgroup: Continuing with sync [production]
17:53 <ladsgroup@deploy2002> ammarpad and ladsgroup: Backport for [[gerrit:992206|ruwiki: Add 'edituserjson' right to 'engineers' group (T355499)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
17:52 <ladsgroup@deploy2002> Started scap: Backport for [[gerrit:992206|ruwiki: Add 'edituserjson' right to 'engineers' group (T355499)]] [production]
17:49 <logmsgbot> lucaswerkmeister-wmde@deploy2002 Finished scap: Backport for [[gerrit:1002950|Use EditEntity for ItemMergeInteractor (T356149 T356764)]], [[gerrit:1002951|Use EditEntity for MergeLexemesInteractor (T356149 T356764)]], [[gerrit:1002952|Use EditEntity for ItemMergeInteractor (T356149 T356764)]], [[gerrit:1002953|Use EditEntity for MergeLexemesInteractor (T356149 T356764)]] (duration: 10m 11s) [production]
17:46 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on restbase1035.eqiad.wmnet with reason: host reimage [production]
17:43 <brouberol@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-test-ui1001.eqiad.wmnet with reason: host reimage [production]
17:42 <logmsgbot> lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde: Continuing with sync [production]
17:41 <jclark@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on restbase1035.eqiad.wmnet with reason: host reimage [production]
17:41 <brouberol@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on an-test-ui1001.eqiad.wmnet with reason: host reimage [production]
17:40 <logmsgbot> lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde: Backport for [[gerrit:1002950|Use EditEntity for ItemMergeInteractor (T356149 T356764)]], [[gerrit:1002951|Use EditEntity for MergeLexemesInteractor (T356149 T356764)]], [[gerrit:1002952|Use EditEntity for ItemMergeInteractor (T356149 T356764)]], [[gerrit:1002953|Use EditEntity for MergeLexemesInteractor (T356149 T356764)]] synced to the testservers (https://wik [production]
17:39 <logmsgbot> lucaswerkmeister-wmde@deploy2002 Started scap: Backport for [[gerrit:1002950|Use EditEntity for ItemMergeInteractor (T356149 T356764)]], [[gerrit:1002951|Use EditEntity for MergeLexemesInteractor (T356149 T356764)]], [[gerrit:1002952|Use EditEntity for ItemMergeInteractor (T356149 T356764)]], [[gerrit:1002953|Use EditEntity for MergeLexemesInteractor (T356149 T356764)]] [production]
17:37 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp[2027-2028].codfw.wmnet [production]