3001-3050 of 10000 results (80ms)
2022-11-10 §
04:40 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1143.eqiad.wmnet with reason: Maintenance [production]
04:40 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1143.eqiad.wmnet with reason: Maintenance [production]
04:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142 (T318605)', diff saved to https://phabricator.wikimedia.org/P38861 and previous config saved to /var/cache/conftool/dbconfig/20221110-044028-ladsgroup.json [production]
04:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142', diff saved to https://phabricator.wikimedia.org/P38860 and previous config saved to /var/cache/conftool/dbconfig/20221110-042522-ladsgroup.json [production]
04:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142', diff saved to https://phabricator.wikimedia.org/P38859 and previous config saved to /var/cache/conftool/dbconfig/20221110-041016-ladsgroup.json [production]
03:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142 (T318605)', diff saved to https://phabricator.wikimedia.org/P38858 and previous config saved to /var/cache/conftool/dbconfig/20221110-035509-ladsgroup.json [production]
00:03 <tzatziki> removing 1 file for legal compliance [production]
2022-11-09 §
23:57 <tzatziki> removing 1 file for legal compliance [production]
23:44 <tzatziki> removing 2 files for legal compliance [production]
23:22 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1034.eqiad.wmnet with OS bullseye [production]
23:17 <tzatziki> removing 1 file for legal compliance [production]
23:07 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti1034.eqiad.wmnet with reason: host reimage [production]
23:04 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti1034.eqiad.wmnet with reason: host reimage [production]
23:03 <aikochou@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
23:00 <aikochou@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
22:51 <robh@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti1034.eqiad.wmnet with OS bullseye [production]
22:34 <damilare> civicrm upgraded from f2017495 to 07fdeed5 [production]
22:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1142 (T318605)', diff saved to https://phabricator.wikimedia.org/P38857 and previous config saved to /var/cache/conftool/dbconfig/20221109-221551-ladsgroup.json [production]
22:15 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1142.eqiad.wmnet with reason: Maintenance [production]
22:15 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1142.eqiad.wmnet with reason: Maintenance [production]
22:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141 (T318605)', diff saved to https://phabricator.wikimedia.org/P38856 and previous config saved to /var/cache/conftool/dbconfig/20221109-221529-ladsgroup.json [production]
22:06 <robh@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host ganeti1034.eqiad.wmnet with OS bullseye [production]
22:01 <robh@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti1034.eqiad.wmnet with OS bullseye [production]
22:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141', diff saved to https://phabricator.wikimedia.org/P38855 and previous config saved to /var/cache/conftool/dbconfig/20221109-220023-ladsgroup.json [production]
21:55 <robh@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host ganeti1034.eqiad.wmnet with OS bullseye [production]
21:48 <robh@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti1034.eqiad.wmnet with OS bullseye [production]
21:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141', diff saved to https://phabricator.wikimedia.org/P38854 and previous config saved to /var/cache/conftool/dbconfig/20221109-214516-ladsgroup.json [production]
21:37 <TheresNoTime> closing UTC late backport window [production]
21:35 <samtar@deploy1002> Finished scap: Backport for [[gerrit:855068|Fix TOC misaligned when max width option is disable (T322162)]] (duration: 08m 48s) [production]
21:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141 (T318605)', diff saved to https://phabricator.wikimedia.org/P38853 and previous config saved to /var/cache/conftool/dbconfig/20221109-213010-ladsgroup.json [production]
21:30 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
21:29 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
21:29 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
21:28 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
21:26 <samtar@deploy1002> samtar and nray: Backport for [[gerrit:855068|Fix TOC misaligned when max width option is disable (T322162)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
21:26 <samtar@deploy1002> Started scap: Backport for [[gerrit:855068|Fix TOC misaligned when max width option is disable (T322162)]] [production]
21:18 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
21:17 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
21:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
21:16 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
21:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2110 (T318605)', diff saved to https://phabricator.wikimedia.org/P38852 and previous config saved to /var/cache/conftool/dbconfig/20221109-211613-ladsgroup.json [production]
21:16 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2110.codfw.wmnet with reason: Maintenance [production]
21:15 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2110.codfw.wmnet with reason: Maintenance [production]
21:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2106 (T318605)', diff saved to https://phabricator.wikimedia.org/P38851 and previous config saved to /var/cache/conftool/dbconfig/20221109-211551-ladsgroup.json [production]
21:15 <samtar@deploy1002> Finished scap: Backport for [[gerrit:854090|Only Enable LBFactory config callback in CLI in production (T298485)]] (duration: 05m 41s) [production]
21:11 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
21:10 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
21:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
21:10 <samtar@deploy1002> samtar and dancy: Backport for [[gerrit:854090|Only Enable LBFactory config callback in CLI in production (T298485)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet [production]
21:09 <samtar@deploy1002> Started scap: Backport for [[gerrit:854090|Only Enable LBFactory config callback in CLI in production (T298485)]] [production]