2951-3000 of 10000 results (71ms)
2024-02-17 §
02:21 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2110.codfw.wmnet with reason: Maintenance [production]
02:21 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2106 (T352010)', diff saved to https://phabricator.wikimedia.org/P56904 and previous config saved to /var/cache/conftool/dbconfig/20240217-022137-ladsgroup.json [production]
02:06 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2106', diff saved to https://phabricator.wikimedia.org/P56903 and previous config saved to /var/cache/conftool/dbconfig/20240217-020630-ladsgroup.json [production]
01:51 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2106', diff saved to https://phabricator.wikimedia.org/P56902 and previous config saved to /var/cache/conftool/dbconfig/20240217-015123-ladsgroup.json [production]
01:36 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2106 (T352010)', diff saved to https://phabricator.wikimedia.org/P56901 and previous config saved to /var/cache/conftool/dbconfig/20240217-013617-ladsgroup.json [production]
2024-02-16 §
21:46 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2205.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:45 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2205.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:43 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2204.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:43 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2204.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:42 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:41 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
21:41 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2204.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:41 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2205.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:40 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2203.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:40 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2205.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:40 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2204.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:40 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2203.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:39 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:38 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding db2203 to codfw - jhancock@cumin2002" [production]
21:38 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding db2203 to codfw - jhancock@cumin2002" [production]
21:35 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
21:32 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host db2201.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:22 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host db2200.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:21 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host db2202.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:03 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2202.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:03 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2201.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:03 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2200.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:58 <tzatziki> removing 2 files for legal compliance [production]
20:47 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2156 (T352010)', diff saved to https://phabricator.wikimedia.org/P56900 and previous config saved to /var/cache/conftool/dbconfig/20240216-204746-ladsgroup.json [production]
20:47 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
20:47 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
20:47 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
20:47 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
20:47 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T352010)', diff saved to https://phabricator.wikimedia.org/P56899 and previous config saved to /var/cache/conftool/dbconfig/20240216-204709-ladsgroup.json [production]
20:38 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:38 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:35 <hashar@deploy2002> Finished deploy [integration/docroot@7a9d46f]: build: Upgrade mediawiki/mediawiki-codesniffer to v43.0.0 (duration: 00m 07s) [production]
20:35 <hashar@deploy2002> Started deploy [integration/docroot@7a9d46f]: build: Upgrade mediawiki/mediawiki-codesniffer to v43.0.0 [production]
20:33 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:33 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:32 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P56898 and previous config saved to /var/cache/conftool/dbconfig/20240216-203202-ladsgroup.json [production]
20:28 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2200.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:28 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2200.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:23 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:22 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
20:17 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2200.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:17 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host db2200.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:16 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P56897 and previous config saved to /var/cache/conftool/dbconfig/20240216-201656-ladsgroup.json [production]
20:16 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host db2202.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:16 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]