|
2024-02-24
§
|
| 08:31 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2106', diff saved to https://phabricator.wikimedia.org/P57873 and previous config saved to /var/cache/conftool/dbconfig/20240224-083138-arnaudb.json |
[production] |
| 07:12 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db2106 (T357189)', diff saved to https://phabricator.wikimedia.org/P57871 and previous config saved to /var/cache/conftool/dbconfig/20240224-071221-arnaudb.json |
[production] |
| 07:12 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2106.codfw.wmnet with reason: Maintenance |
[production] |
| 07:12 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2106.codfw.wmnet with reason: Maintenance |
[production] |
| 06:17 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2099.codfw.wmnet with reason: Maintenance |
[production] |
| 06:17 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2099.codfw.wmnet with reason: Maintenance |
[production] |
| 05:23 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:23 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 05:23 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1249 (T357189)', diff saved to https://phabricator.wikimedia.org/P57870 and previous config saved to /var/cache/conftool/dbconfig/20240224-052320-arnaudb.json |
[production] |
| 05:08 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1249', diff saved to https://phabricator.wikimedia.org/P57869 and previous config saved to /var/cache/conftool/dbconfig/20240224-050814-arnaudb.json |
[production] |
| 04:53 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1249', diff saved to https://phabricator.wikimedia.org/P57868 and previous config saved to /var/cache/conftool/dbconfig/20240224-045307-arnaudb.json |
[production] |
| 04:38 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1249 (T357189)', diff saved to https://phabricator.wikimedia.org/P57867 and previous config saved to /var/cache/conftool/dbconfig/20240224-043801-arnaudb.json |
[production] |
| 03:33 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1249 (T357189)', diff saved to https://phabricator.wikimedia.org/P57866 and previous config saved to /var/cache/conftool/dbconfig/20240224-033304-arnaudb.json |
[production] |
| 03:32 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1249.eqiad.wmnet with reason: Maintenance |
[production] |
| 03:32 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1249.eqiad.wmnet with reason: Maintenance |
[production] |
| 03:32 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1248 (T357189)', diff saved to https://phabricator.wikimedia.org/P57865 and previous config saved to /var/cache/conftool/dbconfig/20240224-033241-arnaudb.json |
[production] |
| 03:17 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1248', diff saved to https://phabricator.wikimedia.org/P57864 and previous config saved to /var/cache/conftool/dbconfig/20240224-031735-arnaudb.json |
[production] |
| 03:02 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1248', diff saved to https://phabricator.wikimedia.org/P57863 and previous config saved to /var/cache/conftool/dbconfig/20240224-030228-arnaudb.json |
[production] |
| 02:47 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1248 (T357189)', diff saved to https://phabricator.wikimedia.org/P57862 and previous config saved to /var/cache/conftool/dbconfig/20240224-024722-arnaudb.json |
[production] |
| 01:47 |
<brett> |
Upload ncmonitor 0.0.3 to bookworm-wikimedia |
[production] |
| 01:47 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1248 (T357189)', diff saved to https://phabricator.wikimedia.org/P57861 and previous config saved to /var/cache/conftool/dbconfig/20240224-014734-arnaudb.json |
[production] |
| 01:47 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1248.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:47 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1248.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:47 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1247 (T357189)', diff saved to https://phabricator.wikimedia.org/P57860 and previous config saved to /var/cache/conftool/dbconfig/20240224-014711-arnaudb.json |
[production] |
| 01:32 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1247', diff saved to https://phabricator.wikimedia.org/P57859 and previous config saved to /var/cache/conftool/dbconfig/20240224-013205-arnaudb.json |
[production] |
| 01:16 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1247', diff saved to https://phabricator.wikimedia.org/P57858 and previous config saved to /var/cache/conftool/dbconfig/20240224-011658-arnaudb.json |
[production] |
| 01:01 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1247 (T357189)', diff saved to https://phabricator.wikimedia.org/P57857 and previous config saved to /var/cache/conftool/dbconfig/20240224-010152-arnaudb.json |
[production] |
|
2024-02-23
§
|
| 23:59 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1247 (T357189)', diff saved to https://phabricator.wikimedia.org/P57856 and previous config saved to /var/cache/conftool/dbconfig/20240223-235919-arnaudb.json |
[production] |
| 23:59 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1247.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:59 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1247.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:14 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1245.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:14 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1245.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:14 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1244 (T357189)', diff saved to https://phabricator.wikimedia.org/P57855 and previous config saved to /var/cache/conftool/dbconfig/20240223-231440-arnaudb.json |
[production] |
| 22:59 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1244', diff saved to https://phabricator.wikimedia.org/P57854 and previous config saved to /var/cache/conftool/dbconfig/20240223-225933-arnaudb.json |
[production] |
| 22:44 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1244', diff saved to https://phabricator.wikimedia.org/P57853 and previous config saved to /var/cache/conftool/dbconfig/20240223-224427-arnaudb.json |
[production] |
| 22:29 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1244 (T357189)', diff saved to https://phabricator.wikimedia.org/P57852 and previous config saved to /var/cache/conftool/dbconfig/20240223-222920-arnaudb.json |
[production] |
| 21:49 |
<sbassett> |
Deployed updated security mitigation for T336027 |
[production] |
| 21:42 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1244 (T357189)', diff saved to https://phabricator.wikimedia.org/P57850 and previous config saved to /var/cache/conftool/dbconfig/20240223-214211-arnaudb.json |
[production] |
| 21:42 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1244.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:41 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1244.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:41 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1243 (T357189)', diff saved to https://phabricator.wikimedia.org/P57848 and previous config saved to /var/cache/conftool/dbconfig/20240223-214149-arnaudb.json |
[production] |
| 21:26 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1243', diff saved to https://phabricator.wikimedia.org/P57847 and previous config saved to /var/cache/conftool/dbconfig/20240223-212643-arnaudb.json |
[production] |
| 21:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1243', diff saved to https://phabricator.wikimedia.org/P57846 and previous config saved to /var/cache/conftool/dbconfig/20240223-211136-arnaudb.json |
[production] |
| 21:07 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2196.codfw.wmnet with reason: host reimage |
[production] |
| 21:04 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db2196.codfw.wmnet with reason: host reimage |
[production] |
| 20:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1243 (T357189)', diff saved to https://phabricator.wikimedia.org/P57845 and previous config saved to /var/cache/conftool/dbconfig/20240223-205630-arnaudb.json |
[production] |
| 20:48 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host db2196.codfw.wmnet with OS bookworm |
[production] |
| 20:42 |
<isaranto@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . |
[production] |
| 20:23 |
<RhinosF1> |
[relog due to stashbot errors] jhancock@cumin2002 ran cookbook SRE.hardware.upgrade-firmware for hosts db2201/db2204/db2197/db2198/db2202/db2203/db2205 and all END PASS |
[production] |
| 20:10 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2205'] |
[production] |