2401-2450 of 10000 results (99ms)
2024-06-17 ยง
20:59 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1174 (T364069)', diff saved to https://phabricator.wikimedia.org/P65117 and previous config saved to /var/cache/conftool/dbconfig/20240617-205955-marostegui.json [production]
20:59 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1174.eqiad.wmnet with reason: Maintenance [production]
20:59 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1174.eqiad.wmnet with reason: Maintenance [production]
20:55 <jforrester@deploy1002> jforrester: Continuing with sync [production]
20:52 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
20:50 <jforrester@deploy1002> jforrester: Backport for [[gerrit:1046767|Fix styles for new heading HTML (T367468)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:50 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
20:46 <jforrester@deploy1002> Started scap: Backport for [[gerrit:1046767|Fix styles for new heading HTML (T367468)]] [production]
20:34 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt1040.eqiad.wmnet with OS bookworm [production]
20:33 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1039.eqiad.wmnet with OS bookworm [production]
20:08 <brett@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp4042.ulsfo.wmnet [production]
20:07 <jforrester@deploy1002> jforrester: Continuing with sync [production]
20:06 <jforrester@deploy1002> jforrester: Backport for [[gerrit:1041659|[wikifunctionswiki] Remove right to promote/demote sysops and bureaucrats from staff (T365627)]], [[gerrit:1039767|Add a note that you cannot change wgCategoryCollation easily (T362494 T366809)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:06 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1039.eqiad.wmnet with reason: host reimage [production]
20:06 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4042.ulsfo.wmnet with OS bullseye [production]
20:02 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1039.eqiad.wmnet with reason: host reimage [production]
20:01 <jforrester@deploy1002> Started scap: Backport for [[gerrit:1041659|[wikifunctionswiki] Remove right to promote/demote sysops and bureaucrats from staff (T365627)]], [[gerrit:1039767|Add a note that you cannot change wgCategoryCollation easily (T362494 T366809)]] [production]
19:55 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2204 (T352010)', diff saved to https://phabricator.wikimedia.org/P65116 and previous config saved to /var/cache/conftool/dbconfig/20240617-195520-ladsgroup.json [production]
19:55 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2204.codfw.wmnet with reason: Maintenance [production]
19:55 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2204.codfw.wmnet with reason: Maintenance [production]
19:43 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt1039.eqiad.wmnet with OS bookworm [production]
19:40 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4042.ulsfo.wmnet with reason: host reimage [production]
19:38 <brett@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp4042.ulsfo.wmnet with reason: host reimage [production]
19:22 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1038.eqiad.wmnet with OS bookworm [production]
19:15 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host cp4042.ulsfo.wmnet with OS bullseye [production]
19:15 <brett@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4042.ulsfo.wmnet with OS bullseye [production]
18:57 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1038.eqiad.wmnet with reason: host reimage [production]
18:56 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host cp4042.ulsfo.wmnet with OS bullseye [production]
18:54 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1038.eqiad.wmnet with reason: host reimage [production]
18:42 <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:1027150|Change static footer icons to the new one (T256190)]], [[gerrit:1046750|Remove footer override]] (duration: 17m 12s) [production]
18:36 <ejegg> fundraising civicrm upgraded from 66acce1f to a25a359b [production]
18:36 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt1038.eqiad.wmnet with OS bookworm [production]
18:33 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1037.eqiad.wmnet with OS bookworm [production]
18:30 <ladsgroup@deploy1002> ladsgroup, jforrester: Continuing with sync [production]
18:29 <ladsgroup@deploy1002> ladsgroup, jforrester: Backport for [[gerrit:1027150|Change static footer icons to the new one (T256190)]], [[gerrit:1046750|Remove footer override]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
18:24 <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:1027150|Change static footer icons to the new one (T256190)]], [[gerrit:1046750|Remove footer override]] [production]
18:19 <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:1027150|Change static footer icons to the new one (T256190)]] [production]
18:17 <ejegg> standalone SmashPig upgraded from 1d1b770c to c8993ec6 [production]
18:12 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/data-gateway: sync [production]
18:12 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/data-gateway: sync [production]
18:11 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/data-gateway: apply [production]
18:10 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/data-gateway: apply [production]
18:09 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/page-analytics: apply [production]
18:09 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/page-analytics: apply [production]
18:08 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/media-analytics: apply [production]
18:07 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/media-analytics: apply [production]
18:07 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/image-suggestion: apply [production]
18:06 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/image-suggestion: apply [production]
18:05 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1037.eqiad.wmnet with reason: host reimage [production]
18:05 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/geo-analytics: apply [production]