|
2022-11-14
§
|
| 08:02 |
<marostegui@deploy1002> |
Finished scap: Backport for [[gerrit:855741|Revert "ProductionServices.php: Promote pc2014 to pc1 master"]] (duration: 04m 34s) |
[production] |
| 07:57 |
<marostegui@deploy1002> |
marostegui and marostegui: Backport for [[gerrit:855741|Revert "ProductionServices.php: Promote pc2014 to pc1 master"]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 07:57 |
<marostegui@deploy1002> |
Started scap: Backport for [[gerrit:855741|Revert "ProductionServices.php: Promote pc2014 to pc1 master"]] |
[production] |
| 07:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1121', diff saved to https://phabricator.wikimedia.org/P39380 and previous config saved to /var/cache/conftool/dbconfig/20221114-075131-marostegui.json |
[production] |
| 07:50 |
<moritzm> |
draining ganeti1021 for eventual reimage T311687 |
[production] |
| 07:47 |
<jmm@cumin2002> |
END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti1033.eqiad.wmnet to cluster eqiad and group D |
[production] |
| 07:47 |
<marostegui@deploy1002> |
Finished scap: Backport for [[gerrit:856473|ProductionServices.php: Promote pc2014 to pc1 master (T322295)]] (duration: 05m 14s) |
[production] |
| 07:47 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.addnode for new host ganeti1033.eqiad.wmnet to cluster eqiad and group D |
[production] |
| 07:42 |
<marostegui@deploy1002> |
marostegui and marostegui: Backport for [[gerrit:856473|ProductionServices.php: Promote pc2014 to pc1 master (T322295)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 07:42 |
<marostegui@deploy1002> |
Started scap: Backport for [[gerrit:856473|ProductionServices.php: Promote pc2014 to pc1 master (T322295)]] |
[production] |
| 07:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1121 (T321130)', diff saved to https://phabricator.wikimedia.org/P39379 and previous config saved to /var/cache/conftool/dbconfig/20221114-073624-marostegui.json |
[production] |
| 07:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1121 (T321130)', diff saved to https://phabricator.wikimedia.org/P39378 and previous config saved to /var/cache/conftool/dbconfig/20221114-072203-marostegui.json |
[production] |
| 07:21 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:21 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:21 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1121.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:21 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1121.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:13 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1181.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:13 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1181.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:08 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2118.codfw.wmnet with reason: Maintenance |
[production] |
| 07:08 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db2118.codfw.wmnet with reason: Maintenance |
[production] |
| 06:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1191 (T321130)', diff saved to https://phabricator.wikimedia.org/P39377 and previous config saved to /var/cache/conftool/dbconfig/20221114-065620-marostegui.json |
[production] |
| 06:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1191', diff saved to https://phabricator.wikimedia.org/P39376 and previous config saved to /var/cache/conftool/dbconfig/20221114-064113-marostegui.json |
[production] |
| 06:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1191', diff saved to https://phabricator.wikimedia.org/P39375 and previous config saved to /var/cache/conftool/dbconfig/20221114-062607-marostegui.json |
[production] |
| 06:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1191 (T321130)', diff saved to https://phabricator.wikimedia.org/P39374 and previous config saved to /var/cache/conftool/dbconfig/20221114-061100-marostegui.json |
[production] |
| 06:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1191 (T321130)', diff saved to https://phabricator.wikimedia.org/P39373 and previous config saved to /var/cache/conftool/dbconfig/20221114-060847-marostegui.json |
[production] |
| 06:08 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1191.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:08 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 5:00:00 on db1191.eqiad.wmnet with reason: Maintenance |
[production] |
| 06:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db2173', diff saved to https://phabricator.wikimedia.org/P39372 and previous config saved to /var/cache/conftool/dbconfig/20221114-060207-root.json |
[production] |
|
2022-11-12
§
|
| 23:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179 (T318605)', diff saved to https://phabricator.wikimedia.org/P39371 and previous config saved to /var/cache/conftool/dbconfig/20221112-233420-ladsgroup.json |
[production] |
| 23:19 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P39370 and previous config saved to /var/cache/conftool/dbconfig/20221112-231914-ladsgroup.json |
[production] |
| 23:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179', diff saved to https://phabricator.wikimedia.org/P39369 and previous config saved to /var/cache/conftool/dbconfig/20221112-230407-ladsgroup.json |
[production] |
| 22:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2179 (T318605)', diff saved to https://phabricator.wikimedia.org/P39368 and previous config saved to /var/cache/conftool/dbconfig/20221112-224900-ladsgroup.json |
[production] |
| 22:46 |
<urandom> |
initiating bootstrap, aqs1016-b -- T307802 |
[production] |
| 21:05 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:05 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:05 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1199 (T318605)', diff saved to https://phabricator.wikimedia.org/P39367 and previous config saved to /var/cache/conftool/dbconfig/20221112-210527-ladsgroup.json |
[production] |
| 20:50 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1199', diff saved to https://phabricator.wikimedia.org/P39366 and previous config saved to /var/cache/conftool/dbconfig/20221112-205020-ladsgroup.json |
[production] |
| 20:35 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1199', diff saved to https://phabricator.wikimedia.org/P39365 and previous config saved to /var/cache/conftool/dbconfig/20221112-203514-ladsgroup.json |
[production] |
| 20:20 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1199 (T318605)', diff saved to https://phabricator.wikimedia.org/P39364 and previous config saved to /var/cache/conftool/dbconfig/20221112-202007-ladsgroup.json |
[production] |
| 17:32 |
<volans|off> |
uploaded python3-gjson_0.4.0 to apt.wikimedia.org bullseye-wikimedia |
[production] |
| 17:17 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2179 (T318605)', diff saved to https://phabricator.wikimedia.org/P39363 and previous config saved to /var/cache/conftool/dbconfig/20221112-171705-ladsgroup.json |
[production] |
| 17:16 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2179.codfw.wmnet with reason: Maintenance |
[production] |
| 17:16 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2179.codfw.wmnet with reason: Maintenance |
[production] |
| 17:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T318605)', diff saved to https://phabricator.wikimedia.org/P39362 and previous config saved to /var/cache/conftool/dbconfig/20221112-171643-ladsgroup.json |
[production] |
| 17:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P39361 and previous config saved to /var/cache/conftool/dbconfig/20221112-170137-ladsgroup.json |
[production] |
| 16:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P39360 and previous config saved to /var/cache/conftool/dbconfig/20221112-164630-ladsgroup.json |
[production] |
| 16:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2172 (T318605)', diff saved to https://phabricator.wikimedia.org/P39359 and previous config saved to /var/cache/conftool/dbconfig/20221112-163124-ladsgroup.json |
[production] |
| 14:43 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1199 (T318605)', diff saved to https://phabricator.wikimedia.org/P39358 and previous config saved to /var/cache/conftool/dbconfig/20221112-144302-ladsgroup.json |
[production] |
| 14:42 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1199.eqiad.wmnet with reason: Maintenance |
[production] |
| 14:42 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1199.eqiad.wmnet with reason: Maintenance |
[production] |