3251-3300 of 10000 results (97ms)
2024-04-08 ยง
21:10 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:01 <pfischer@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
21:01 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:01 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1248', diff saved to https://phabricator.wikimedia.org/P59925 and previous config saved to /var/cache/conftool/dbconfig/20240408-205548-arnaudb.json [production]
20:53 <pfischer@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:53 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:53 <pfischer@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:53 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:52 <ebernhardson@deploy1002> Finished scap: Backport for [[gerrit:1015393|cirrus: Restore traffic to codfw clusters]] (duration: 15m 25s) [production]
20:42 <ebernhardson@deploy1002> ebernhardson: Continuing with sync [production]
20:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1248 (T360332)', diff saved to https://phabricator.wikimedia.org/P59924 and previous config saved to /var/cache/conftool/dbconfig/20240408-204041-arnaudb.json [production]
20:39 <ebernhardson@deploy1002> ebernhardson: Backport for [[gerrit:1015393|cirrus: Restore traffic to codfw clusters]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:37 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1248 (T360332)', diff saved to https://phabricator.wikimedia.org/P59923 and previous config saved to /var/cache/conftool/dbconfig/20240408-203723-arnaudb.json [production]
20:37 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1248.eqiad.wmnet with reason: Maintenance [production]
20:37 <ebernhardson@deploy1002> Started scap: Backport for [[gerrit:1015393|cirrus: Restore traffic to codfw clusters]] [production]
20:37 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1248.eqiad.wmnet with reason: Maintenance [production]
20:37 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247 (T360332)', diff saved to https://phabricator.wikimedia.org/P59922 and previous config saved to /var/cache/conftool/dbconfig/20240408-203700-arnaudb.json [production]
20:29 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213 (T360332)', diff saved to https://phabricator.wikimedia.org/P59921 and previous config saved to /var/cache/conftool/dbconfig/20240408-202955-arnaudb.json [production]
20:25 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:1017890|Revert "Mark all autoreviewed edits in PageSaveComplete hook" (T361918 T361940 T361960)]] (duration: 16m 25s) [production]
20:24 <pfischer@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
20:23 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
20:21 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247', diff saved to https://phabricator.wikimedia.org/P59920 and previous config saved to /var/cache/conftool/dbconfig/20240408-202153-arnaudb.json [production]
20:15 <urbanecm@deploy1002> urbanecm and matmarex: Continuing with sync [production]
20:14 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213', diff saved to https://phabricator.wikimedia.org/P59919 and previous config saved to /var/cache/conftool/dbconfig/20240408-201447-arnaudb.json [production]
20:11 <urbanecm@deploy1002> urbanecm and matmarex: Backport for [[gerrit:1017890|Revert "Mark all autoreviewed edits in PageSaveComplete hook" (T361918 T361940 T361960)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:09 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:1017890|Revert "Mark all autoreviewed edits in PageSaveComplete hook" (T361918 T361940 T361960)]] [production]
20:06 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247', diff saved to https://phabricator.wikimedia.org/P59918 and previous config saved to /var/cache/conftool/dbconfig/20240408-200645-arnaudb.json [production]
19:59 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213', diff saved to https://phabricator.wikimedia.org/P59917 and previous config saved to /var/cache/conftool/dbconfig/20240408-195940-arnaudb.json [production]
19:51 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247 (T360332)', diff saved to https://phabricator.wikimedia.org/P59916 and previous config saved to /var/cache/conftool/dbconfig/20240408-195138-arnaudb.json [production]
19:49 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1247 (T360332)', diff saved to https://phabricator.wikimedia.org/P59915 and previous config saved to /var/cache/conftool/dbconfig/20240408-194919-arnaudb.json [production]
19:49 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1247.eqiad.wmnet with reason: Maintenance [production]
19:48 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1247.eqiad.wmnet with reason: Maintenance [production]
19:48 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1245.eqiad.wmnet with reason: Maintenance [production]
19:48 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1245.eqiad.wmnet with reason: Maintenance [production]
19:48 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244 (T360332)', diff saved to https://phabricator.wikimedia.org/P59914 and previous config saved to /var/cache/conftool/dbconfig/20240408-194843-arnaudb.json [production]
19:44 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213 (T360332)', diff saved to https://phabricator.wikimedia.org/P59913 and previous config saved to /var/cache/conftool/dbconfig/20240408-194432-arnaudb.json [production]
19:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2213 (T360332)', diff saved to https://phabricator.wikimedia.org/P59912 and previous config saved to /var/cache/conftool/dbconfig/20240408-194113-arnaudb.json [production]
19:41 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2213.codfw.wmnet with reason: Maintenance [production]
19:40 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2213.codfw.wmnet with reason: Maintenance [production]
19:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211 (T360332)', diff saved to https://phabricator.wikimedia.org/P59911 and previous config saved to /var/cache/conftool/dbconfig/20240408-194050-arnaudb.json [production]
19:33 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244', diff saved to https://phabricator.wikimedia.org/P59910 and previous config saved to /var/cache/conftool/dbconfig/20240408-193336-arnaudb.json [production]
19:25 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211', diff saved to https://phabricator.wikimedia.org/P59909 and previous config saved to /var/cache/conftool/dbconfig/20240408-192543-arnaudb.json [production]
19:18 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244', diff saved to https://phabricator.wikimedia.org/P59908 and previous config saved to /var/cache/conftool/dbconfig/20240408-191828-arnaudb.json [production]
19:10 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211', diff saved to https://phabricator.wikimedia.org/P59907 and previous config saved to /var/cache/conftool/dbconfig/20240408-191035-arnaudb.json [production]
19:03 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244 (T360332)', diff saved to https://phabricator.wikimedia.org/P59905 and previous config saved to /var/cache/conftool/dbconfig/20240408-190319-arnaudb.json [production]
18:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211 (T360332)', diff saved to https://phabricator.wikimedia.org/P59904 and previous config saved to /var/cache/conftool/dbconfig/20240408-185528-arnaudb.json [production]
18:53 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2211 (T360332)', diff saved to https://phabricator.wikimedia.org/P59903 and previous config saved to /var/cache/conftool/dbconfig/20240408-185309-arnaudb.json [production]
18:53 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2211.codfw.wmnet with reason: Maintenance [production]
18:52 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2211.codfw.wmnet with reason: Maintenance [production]