|
2022-11-10
§
|
| 23:13 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance |
[production] |
| 23:13 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122 (T322618)', diff saved to https://phabricator.wikimedia.org/P39134 and previous config saved to /var/cache/conftool/dbconfig/20221110-231306-ladsgroup.json |
[production] |
| 23:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P39133 and previous config saved to /var/cache/conftool/dbconfig/20221110-230759-marostegui.json |
[production] |
| 22:58 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P39132 and previous config saved to /var/cache/conftool/dbconfig/20221110-225759-ladsgroup.json |
[production] |
| 22:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P39131 and previous config saved to /var/cache/conftool/dbconfig/20221110-225253-marostegui.json |
[production] |
| 22:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P39130 and previous config saved to /var/cache/conftool/dbconfig/20221110-224253-ladsgroup.json |
[production] |
| 22:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2116 (T321123)', diff saved to https://phabricator.wikimedia.org/P39129 and previous config saved to /var/cache/conftool/dbconfig/20221110-223746-marostegui.json |
[production] |
| 22:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2116 (T321123)', diff saved to https://phabricator.wikimedia.org/P39128 and previous config saved to /var/cache/conftool/dbconfig/20221110-223537-marostegui.json |
[production] |
| 22:35 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2116.codfw.wmnet with reason: Maintenance |
[production] |
| 22:35 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2116.codfw.wmnet with reason: Maintenance |
[production] |
| 22:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2103 (T321123)', diff saved to https://phabricator.wikimedia.org/P39127 and previous config saved to /var/cache/conftool/dbconfig/20221110-223515-marostegui.json |
[production] |
| 22:27 |
<eileen> |
thank you back on config revision changed from bbdd4315 to 2bb73bb1 |
[production] |
| 22:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122 (T322618)', diff saved to https://phabricator.wikimedia.org/P39126 and previous config saved to /var/cache/conftool/dbconfig/20221110-222746-ladsgroup.json |
[production] |
| 22:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2122 (T322618)', diff saved to https://phabricator.wikimedia.org/P39125 and previous config saved to /var/cache/conftool/dbconfig/20221110-222526-ladsgroup.json |
[production] |
| 22:25 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2122.codfw.wmnet with reason: Maintenance |
[production] |
| 22:25 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2122.codfw.wmnet with reason: Maintenance |
[production] |
| 22:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121 (T322618)', diff saved to https://phabricator.wikimedia.org/P39124 and previous config saved to /var/cache/conftool/dbconfig/20221110-222505-ladsgroup.json |
[production] |
| 22:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2103', diff saved to https://phabricator.wikimedia.org/P39123 and previous config saved to /var/cache/conftool/dbconfig/20221110-222009-marostegui.json |
[production] |
| 22:09 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121', diff saved to https://phabricator.wikimedia.org/P39122 and previous config saved to /var/cache/conftool/dbconfig/20221110-220958-ladsgroup.json |
[production] |
| 22:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2103', diff saved to https://phabricator.wikimedia.org/P39121 and previous config saved to /var/cache/conftool/dbconfig/20221110-220502-marostegui.json |
[production] |
| 21:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121', diff saved to https://phabricator.wikimedia.org/P39120 and previous config saved to /var/cache/conftool/dbconfig/20221110-215452-ladsgroup.json |
[production] |
| 21:53 |
<jgleeson> |
payments updated from 17cd1956 to a058fdbc |
[production] |
| 21:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2103 (T321123)', diff saved to https://phabricator.wikimedia.org/P39119 and previous config saved to /var/cache/conftool/dbconfig/20221110-214956-marostegui.json |
[production] |
| 21:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2103 (T321123)', diff saved to https://phabricator.wikimedia.org/P39118 and previous config saved to /var/cache/conftool/dbconfig/20221110-214746-marostegui.json |
[production] |
| 21:47 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 21:47 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2103.codfw.wmnet with reason: Maintenance |
[production] |
| 21:47 |
<greg-g> |
3:43:33 <eileen> !civicrm upgraded from 6c2e07e0 to fd60273a |
[production] |
| 21:43 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2102.codfw.wmnet with reason: Maintenance |
[production] |
| 21:43 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2102.codfw.wmnet with reason: Maintenance |
[production] |
| 21:43 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2097.codfw.wmnet with reason: Maintenance |
[production] |
| 21:42 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2097.codfw.wmnet with reason: Maintenance |
[production] |
| 21:42 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:42 |
<eileen> |
process-control config revision changed from 4e438cf5 to bbdd4315 (disable thank you) |
[production] |
| 21:42 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1196 (T321123)', diff saved to https://phabricator.wikimedia.org/P39117 and previous config saved to /var/cache/conftool/dbconfig/20221110-214240-marostegui.json |
[production] |
| 21:39 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121 (T322618)', diff saved to https://phabricator.wikimedia.org/P39116 and previous config saved to /var/cache/conftool/dbconfig/20221110-213945-ladsgroup.json |
[production] |
| 21:37 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2121 (T322618)', diff saved to https://phabricator.wikimedia.org/P39115 and previous config saved to /var/cache/conftool/dbconfig/20221110-213726-ladsgroup.json |
[production] |
| 21:37 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance |
[production] |
| 21:37 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance |
[production] |
| 21:37 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2120 (T322618)', diff saved to https://phabricator.wikimedia.org/P39114 and previous config saved to /var/cache/conftool/dbconfig/20221110-213704-ladsgroup.json |
[production] |
| 21:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P39113 and previous config saved to /var/cache/conftool/dbconfig/20221110-212734-marostegui.json |
[production] |
| 21:27 |
<dancy@deploy1002> |
Installation of scap version "4.28.0" completed for 559 hosts |
[production] |
| 21:26 |
<dancy@deploy1002> |
Installing scap version "4.28.0" for 559 hosts |
[production] |
| 21:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2120', diff saved to https://phabricator.wikimedia.org/P39112 and previous config saved to /var/cache/conftool/dbconfig/20221110-212158-ladsgroup.json |
[production] |
| 21:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P39111 and previous config saved to /var/cache/conftool/dbconfig/20221110-211227-marostegui.json |
[production] |
| 21:10 |
<mutante> |
deploy1002 - armed the keyholder with deployment keys - 2 hours ago alerts started that it was not armed (does it notify people?) - got pinged that deployers got scap problems - unknown why it was disarmed - now it is armed again |
[production] |
| 21:06 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2120', diff saved to https://phabricator.wikimedia.org/P39110 and previous config saved to /var/cache/conftool/dbconfig/20221110-210651-ladsgroup.json |
[production] |
| 21:04 |
<jclark@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 21:03 |
<jclark@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 21:03 |
<cdanis> |
✔️ cdanis@cumin1001.eqiad.wmnet ~ 🕞🍵 sudo cumin -b 8 A:cp 'run-puppet-agent --enable T306580' |
[production] |