3501-3550 of 10000 results (94ms)
2024-04-08 ยง
12:46 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host db1180.eqiad.wmnet [production]
12:46 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1194.eqiad.wmnet [production]
12:46 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
12:46 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
12:44 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:27 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1237.eqiad.wmnet [production]
12:25 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1160 (T360332)', diff saved to https://phabricator.wikimedia.org/P59827 and previous config saved to /var/cache/conftool/dbconfig/20240408-122527-arnaudb.json [production]
12:22 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
12:22 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1160 (T360332)', diff saved to https://phabricator.wikimedia.org/P59826 and previous config saved to /var/cache/conftool/dbconfig/20240408-122209-arnaudb.json [production]
12:22 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1160.eqiad.wmnet with reason: Maintenance [production]
12:21 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1160.eqiad.wmnet with reason: Maintenance [production]
12:19 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:19 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1213 (T360332)', diff saved to https://phabricator.wikimedia.org/P59825 and previous config saved to /var/cache/conftool/dbconfig/20240408-121920-arnaudb.json [production]
12:17 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host db1237.eqiad.wmnet [production]
12:17 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host es1022.eqiad.wmnet [production]
12:17 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
12:16 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1213 (T360332)', diff saved to https://phabricator.wikimedia.org/P59824 and previous config saved to /var/cache/conftool/dbconfig/20240408-121642-arnaudb.json [production]
12:16 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1213.eqiad.wmnet with reason: Maintenance [production]
12:16 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1213.eqiad.wmnet with reason: Maintenance [production]
12:16 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1210 (T360332)', diff saved to https://phabricator.wikimedia.org/P59823 and previous config saved to /var/cache/conftool/dbconfig/20240408-121609-arnaudb.json [production]
12:15 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:14 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
12:11 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:09 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host es1022.eqiad.wmnet [production]
12:07 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host es1031.eqiad.wmnet [production]
12:04 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
12:02 <btullis@cumin1002> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host stat1011.eqiad.wmnet [production]
12:02 <btullis@cumin1002> START - Cookbook sre.hosts.reboot-single for host stat1011.eqiad.wmnet [production]
12:01 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:01 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1210', diff saved to https://phabricator.wikimedia.org/P59822 and previous config saved to /var/cache/conftool/dbconfig/20240408-120101-arnaudb.json [production]
11:57 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host es1031.eqiad.wmnet [production]
11:57 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host es1030.eqiad.wmnet [production]
11:49 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host es1030.eqiad.wmnet [production]
11:49 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host es1029.eqiad.wmnet [production]
11:45 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1210', diff saved to https://phabricator.wikimedia.org/P59821 and previous config saved to /var/cache/conftool/dbconfig/20240408-114552-arnaudb.json [production]
11:42 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host es1029.eqiad.wmnet [production]
11:35 <moritzm> installing glibc security updates on bullseye [production]
11:30 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1210 (T360332)', diff saved to https://phabricator.wikimedia.org/P59820 and previous config saved to /var/cache/conftool/dbconfig/20240408-113045-arnaudb.json [production]
11:28 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1210 (T360332)', diff saved to https://phabricator.wikimedia.org/P59819 and previous config saved to /var/cache/conftool/dbconfig/20240408-112807-arnaudb.json [production]
11:28 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1210.eqiad.wmnet with reason: Maintenance [production]
11:27 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1210.eqiad.wmnet with reason: Maintenance [production]
11:27 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1200 (T360332)', diff saved to https://phabricator.wikimedia.org/P59818 and previous config saved to /var/cache/conftool/dbconfig/20240408-112744-arnaudb.json [production]
11:20 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1178 (T360332)', diff saved to https://phabricator.wikimedia.org/P59817 and previous config saved to /var/cache/conftool/dbconfig/20240408-112052-arnaudb.json [production]
11:13 <hnowlan@deploy1002> helmfile [codfw] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
11:12 <hnowlan@deploy1002> helmfile [codfw] [canary] DONE helmfile.d/services/mw-jobrunner : sync [production]
11:12 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P59816 and previous config saved to /var/cache/conftool/dbconfig/20240408-111236-arnaudb.json [production]
11:12 <hnowlan@deploy1002> helmfile [codfw] [canary] START helmfile.d/services/mw-jobrunner : sync [production]
11:12 <hnowlan@deploy1002> helmfile [codfw] [main] START helmfile.d/services/mw-jobrunner : sync [production]
11:11 <hnowlan@deploy1002> helmfile [eqiad] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
11:10 <hnowlan@deploy1002> helmfile [eqiad] [canary] DONE helmfile.d/services/mw-jobrunner : sync [production]