|
2023-02-08
ยง
|
| 19:58 |
<pt1979@cumin2002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" |
[production] |
| 19:55 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1139.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:55 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1139.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1129 (T328817)', diff saved to https://phabricator.wikimedia.org/P43883 and previous config saved to /var/cache/conftool/dbconfig/20230208-195542-marostegui.json |
[production] |
| 19:52 |
<dancy@deploy1002> |
say aborted: (duration: 00m 01s) |
[production] |
| 19:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175 (T328817)', diff saved to https://phabricator.wikimedia.org/P43882 and previous config saved to /var/cache/conftool/dbconfig/20230208-194745-marostegui.json |
[production] |
| 19:42 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2420.codfw.wmnet with reason: host reimage |
[production] |
| 19:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P43881 and previous config saved to /var/cache/conftool/dbconfig/20230208-194036-marostegui.json |
[production] |
| 19:39 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on mw2420.codfw.wmnet with reason: host reimage |
[production] |
| 19:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P43880 and previous config saved to /var/cache/conftool/dbconfig/20230208-193239-marostegui.json |
[production] |
| 19:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P43879 and previous config saved to /var/cache/conftool/dbconfig/20230208-192530-marostegui.json |
[production] |
| 19:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2109 (T328255)', diff saved to https://phabricator.wikimedia.org/P43878 and previous config saved to /var/cache/conftool/dbconfig/20230208-192136-ladsgroup.json |
[production] |
| 19:21 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2109.codfw.wmnet with reason: Maintenance |
[production] |
| 19:21 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2109.codfw.wmnet with reason: Maintenance |
[production] |
| 19:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105 (T328255)', diff saved to https://phabricator.wikimedia.org/P43877 and previous config saved to /var/cache/conftool/dbconfig/20230208-192115-ladsgroup.json |
[production] |
| 19:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P43876 and previous config saved to /var/cache/conftool/dbconfig/20230208-191732-marostegui.json |
[production] |
| 19:12 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host mw2420.codfw.wmnet with OS buster |
[production] |
| 19:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1129 (T328817)', diff saved to https://phabricator.wikimedia.org/P43875 and previous config saved to /var/cache/conftool/dbconfig/20230208-191023-marostegui.json |
[production] |
| 19:08 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@9101b03] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@9101b03] (duration: 01m 20s) |
[production] |
| 19:06 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@9101b03] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@9101b03] |
[production] |
| 19:06 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105', diff saved to https://phabricator.wikimedia.org/P43874 and previous config saved to /var/cache/conftool/dbconfig/20230208-190608-ladsgroup.json |
[production] |
| 19:06 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@9101b03] (thin): Regular analytics weekly train THIN [analytics/refinery@9101b03] (duration: 00m 07s) |
[production] |
| 19:05 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@9101b03] (thin): Regular analytics weekly train THIN [analytics/refinery@9101b03] |
[production] |
| 19:05 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@9101b03]: Regular analytics weekly train [analytics/refinery@9101b03] (duration: 06m 28s) |
[production] |
| 19:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1129 (T328817)', diff saved to https://phabricator.wikimedia.org/P43873 and previous config saved to /var/cache/conftool/dbconfig/20230208-190410-marostegui.json |
[production] |
| 19:04 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1129.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:03 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1129.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1122 (T328817)', diff saved to https://phabricator.wikimedia.org/P43872 and previous config saved to /var/cache/conftool/dbconfig/20230208-190349-marostegui.json |
[production] |
| 19:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2175 (T328817)', diff saved to https://phabricator.wikimedia.org/P43871 and previous config saved to /var/cache/conftool/dbconfig/20230208-190226-marostegui.json |
[production] |
| 18:59 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@9101b03]: Regular analytics weekly train [analytics/refinery@9101b03] |
[production] |
| 18:57 |
<pt1979@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host mw2420.codfw.wmnet with OS buster |
[production] |
| 18:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2175 (T328817)', diff saved to https://phabricator.wikimedia.org/P43870 and previous config saved to /var/cache/conftool/dbconfig/20230208-185513-marostegui.json |
[production] |
| 18:55 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2175.codfw.wmnet with reason: Maintenance |
[production] |
| 18:54 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2175.codfw.wmnet with reason: Maintenance |
[production] |
| 18:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T328817)', diff saved to https://phabricator.wikimedia.org/P43869 and previous config saved to /var/cache/conftool/dbconfig/20230208-185451-marostegui.json |
[production] |
| 18:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105', diff saved to https://phabricator.wikimedia.org/P43868 and previous config saved to /var/cache/conftool/dbconfig/20230208-185102-ladsgroup.json |
[production] |
| 18:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P43867 and previous config saved to /var/cache/conftool/dbconfig/20230208-184842-marostegui.json |
[production] |
| 18:42 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host mw2420.codfw.wmnet with OS buster |
[production] |
| 18:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312', diff saved to https://phabricator.wikimedia.org/P43866 and previous config saved to /var/cache/conftool/dbconfig/20230208-183945-marostegui.json |
[production] |
| 18:39 |
<mutante> |
adding az.wikimedia.org to DNS - approved by affcom T306015 |
[production] |
| 18:35 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2105 (T328255)', diff saved to https://phabricator.wikimedia.org/P43865 and previous config saved to /var/cache/conftool/dbconfig/20230208-183556-ladsgroup.json |
[production] |
| 18:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P43864 and previous config saved to /var/cache/conftool/dbconfig/20230208-183336-marostegui.json |
[production] |
| 18:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312', diff saved to https://phabricator.wikimedia.org/P43863 and previous config saved to /var/cache/conftool/dbconfig/20230208-182439-marostegui.json |
[production] |
| 18:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1122 (T328817)', diff saved to https://phabricator.wikimedia.org/P43862 and previous config saved to /var/cache/conftool/dbconfig/20230208-181829-marostegui.json |
[production] |
| 18:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1122 (T328817)', diff saved to https://phabricator.wikimedia.org/P43861 and previous config saved to /var/cache/conftool/dbconfig/20230208-181222-marostegui.json |
[production] |
| 18:12 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1122.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:12 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1122.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1105:3312 (T328817)', diff saved to https://phabricator.wikimedia.org/P43860 and previous config saved to /var/cache/conftool/dbconfig/20230208-181200-marostegui.json |
[production] |
| 18:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T328817)', diff saved to https://phabricator.wikimedia.org/P43859 and previous config saved to /var/cache/conftool/dbconfig/20230208-180933-marostegui.json |
[production] |
| 18:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2170:3312 (T328817)', diff saved to https://phabricator.wikimedia.org/P43858 and previous config saved to /var/cache/conftool/dbconfig/20230208-180216-marostegui.json |
[production] |