101-150 of 10000 results (75ms)
2024-04-04 §
18:19 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudbackup1002-dev.eqiad.wmnet with OS bookworm [production]
18:16 <sukhe@cumin1002> START - Cookbook sre.hosts.reimage for host cp4052.ulsfo.wmnet with OS bullseye [production]
18:16 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1227 (T360332)', diff saved to https://phabricator.wikimedia.org/P59554 and previous config saved to /var/cache/conftool/dbconfig/20240404-181601-arnaudb.json [production]
18:13 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1227 (T360332)', diff saved to https://phabricator.wikimedia.org/P59553 and previous config saved to /var/cache/conftool/dbconfig/20240404-181326-arnaudb.json [production]
18:13 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1227.eqiad.wmnet with reason: Maintenance [production]
18:13 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1227.eqiad.wmnet with reason: Maintenance [production]
18:13 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1202 (T360332)', diff saved to https://phabricator.wikimedia.org/P59552 and previous config saved to /var/cache/conftool/dbconfig/20240404-181303-arnaudb.json [production]
18:09 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1211 (T355609)', diff saved to https://phabricator.wikimedia.org/P59551 and previous config saved to /var/cache/conftool/dbconfig/20240404-180913-marostegui.json [production]
18:07 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1207 (T356166)', diff saved to https://phabricator.wikimedia.org/P59550 and previous config saved to /var/cache/conftool/dbconfig/20240404-180733-marostegui.json [production]
18:04 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudbackup1002-dev.eqiad.wmnet with reason: host reimage [production]
18:01 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudbackup1002-dev.eqiad.wmnet with reason: host reimage [production]
17:57 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P59549 and previous config saved to /var/cache/conftool/dbconfig/20240404-175756-arnaudb.json [production]
17:49 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudbackup1002-dev.eqiad.wmnet with OS bookworm [production]
17:48 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudbackup1001-dev.eqiad.wmnet with OS bookworm [production]
17:48 <sukhe> depool cp4052 to prepare for reimaging [production]
17:46 <sukhe@cumin1002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts cp4052.ulsfo.wmnet [production]
17:42 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P59548 and previous config saved to /var/cache/conftool/dbconfig/20240404-174246-arnaudb.json [production]
17:37 <sukhe@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp4052.ulsfo.wmnet [production]
17:36 <ebernhardson@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
17:35 <ebernhardson@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
17:29 <moritzm> installing isl bugfix updates from Bookworm point release [production]
17:27 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1202 (T360332)', diff saved to https://phabricator.wikimedia.org/P59547 and previous config saved to /var/cache/conftool/dbconfig/20240404-172739-arnaudb.json [production]
17:24 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1202 (T360332)', diff saved to https://phabricator.wikimedia.org/P59546 and previous config saved to /var/cache/conftool/dbconfig/20240404-172408-arnaudb.json [production]
17:24 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1202.eqiad.wmnet with reason: Maintenance [production]
17:23 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1202.eqiad.wmnet with reason: Maintenance [production]
17:23 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1194 (T360332)', diff saved to https://phabricator.wikimedia.org/P59545 and previous config saved to /var/cache/conftool/dbconfig/20240404-172343-arnaudb.json [production]
17:22 <moritzm> installing qemu security updates on bookworm [production]
17:19 <jforrester@deploy1002> helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply [production]
17:17 <jforrester@deploy1002> helmfile [eqiad] START helmfile.d/services/wikifunctions: apply [production]
17:17 <jforrester@deploy1002> helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply [production]
17:15 <jforrester@deploy1002> helmfile [codfw] START helmfile.d/services/wikifunctions: apply [production]
17:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1211 (T355609)', diff saved to https://phabricator.wikimedia.org/P59544 and previous config saved to /var/cache/conftool/dbconfig/20240404-171347-marostegui.json [production]
17:13 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1211.eqiad.wmnet with reason: Maintenance [production]
17:13 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1211.eqiad.wmnet with reason: Maintenance [production]
17:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1203 (T355609)', diff saved to https://phabricator.wikimedia.org/P59543 and previous config saved to /var/cache/conftool/dbconfig/20240404-171324-marostegui.json [production]
17:13 <jforrester@deploy1002> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
17:12 <jforrester@deploy1002> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
17:09 <fabfur@cumin1002> conftool action : set/pooled=yes; selector: name=cp3068.esams.wmnet [production]
17:08 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1194', diff saved to https://phabricator.wikimedia.org/P59542 and previous config saved to /var/cache/conftool/dbconfig/20240404-170836-arnaudb.json [production]
16:58 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P59541 and previous config saved to /var/cache/conftool/dbconfig/20240404-165816-marostegui.json [production]
16:58 <pmiazga> T355281 executed “mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki --skipclusters=main,echo,growth,mediamoderation,extstore en wikipedia test2wiki test2.wikipedia.beta.wmcloud.org” on deployment-deploy03.deployment-prep [production]
16:55 <fabfur@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp3068.esams.wmnet with OS bullseye [production]
16:54 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudbackup1001-dev.eqiad.wmnet with reason: host reimage [production]
16:53 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1194', diff saved to https://phabricator.wikimedia.org/P59540 and previous config saved to /var/cache/conftool/dbconfig/20240404-165328-arnaudb.json [production]
16:51 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudbackup1001-dev.eqiad.wmnet with reason: host reimage [production]
16:43 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P59539 and previous config saved to /var/cache/conftool/dbconfig/20240404-164309-marostegui.json [production]
16:42 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudbackup1001-dev.eqiad.wmnet with OS bookworm [production]
16:38 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1194 (T360332)', diff saved to https://phabricator.wikimedia.org/P59538 and previous config saved to /var/cache/conftool/dbconfig/20240404-163819-arnaudb.json [production]
16:35 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1194 (T360332)', diff saved to https://phabricator.wikimedia.org/P59537 and previous config saved to /var/cache/conftool/dbconfig/20240404-163549-arnaudb.json [production]
16:35 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1194.eqiad.wmnet with reason: Maintenance [production]