|
2022-11-07
ยง
|
| 08:10 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 08:09 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 08:09 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 08:09 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 08:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1142', diff saved to https://phabricator.wikimedia.org/P38185 and previous config saved to /var/cache/conftool/dbconfig/20221107-080613-marostegui.json |
[production] |
| 08:05 |
<urbanecm@deploy1002> |
urbanecm and daniel: Backport for [[gerrit:845058|Set VisualEditorDefaultParsoidClient for dewiki-beta mad testwiki (T320531)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 08:04 |
<urbanecm@deploy1002> |
Started scap: Backport for [[gerrit:845058|Set VisualEditorDefaultParsoidClient for dewiki-beta mad testwiki (T320531)]] |
[production] |
| 08:03 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 08:02 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 08:02 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 08:01 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 08:00 |
<marostegui@deploy1002> |
Finished scap: Backport for [[gerrit:853910|ProductionServices.php: Promote pc2014 to pc3 master]] (duration: 04m 18s) |
[production] |
| 07:56 |
<marostegui@deploy1002> |
marostegui and marostegui: Backport for [[gerrit:853910|ProductionServices.php: Promote pc2014 to pc3 master]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 07:55 |
<marostegui@deploy1002> |
Started scap: Backport for [[gerrit:853910|ProductionServices.php: Promote pc2014 to pc3 master]] |
[production] |
| 07:51 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1142 (T321123)', diff saved to https://phabricator.wikimedia.org/P38184 and previous config saved to /var/cache/conftool/dbconfig/20221107-075106-marostegui.json |
[production] |
| 07:50 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:50 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1142 (T321123)', diff saved to https://phabricator.wikimedia.org/P38183 and previous config saved to /var/cache/conftool/dbconfig/20221107-074959-marostegui.json |
[production] |
| 07:49 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1142.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:49 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1142.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1141 (T321123)', diff saved to https://phabricator.wikimedia.org/P38182 and previous config saved to /var/cache/conftool/dbconfig/20221107-074938-marostegui.json |
[production] |
| 07:47 |
<marostegui@deploy1002> |
Finished scap: Backport for [[gerrit:853528|Revert "ProductionServices.php: Promote pc1014 to pc1 master"]] (duration: 04m 53s) |
[production] |
| 07:46 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:44 |
<urbanecm> |
Run `time mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php --wiki=frwiki` in a tmux at mwmaint1002 (T318457) |
[production] |
| 07:42 |
<marostegui@deploy1002> |
marostegui and marostegui: Backport for [[gerrit:853528|Revert "ProductionServices.php: Promote pc1014 to pc1 master"]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 07:42 |
<marostegui@deploy1002> |
Started scap: Backport for [[gerrit:853528|Revert "ProductionServices.php: Promote pc1014 to pc1 master"]] |
[production] |
| 07:37 |
<elukey> |
`elukey@aux-k8s-worker1002:~$ sudo systemctl reset-failed ifup@ens13.service` |
[production] |
| 07:34 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1141', diff saved to https://phabricator.wikimedia.org/P38181 and previous config saved to /var/cache/conftool/dbconfig/20221107-073431-marostegui.json |
[production] |
| 07:31 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:28 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:28 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:26 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:20 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1141', diff saved to https://phabricator.wikimedia.org/P38180 and previous config saved to /var/cache/conftool/dbconfig/20221107-071925-marostegui.json |
[production] |
| 07:17 |
<marostegui@deploy1002> |
Finished scap: Backport for [[gerrit:853711|ProductionServices.php: Promote pc1014 to pc1 master (T322295)]] (duration: 04m 29s) |
[production] |
| 07:17 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:17 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 07:13 |
<marostegui@deploy1002> |
marostegui and marostegui: Backport for [[gerrit:853711|ProductionServices.php: Promote pc1014 to pc1 master (T322295)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 07:13 |
<marostegui@deploy1002> |
Started scap: Backport for [[gerrit:853711|ProductionServices.php: Promote pc1014 to pc1 master (T322295)]] |
[production] |
| 07:12 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 07:07 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on pc2011.codfw.wmnet,pc[1011,1014].eqiad.wmnet with reason: Primary switchover |
[production] |
| 07:07 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 1:00:00 on pc2011.codfw.wmnet,pc[1011,1014].eqiad.wmnet with reason: Primary switchover |
[production] |
| 07:05 |
<urbanecm> |
Run `time mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php --wiki=bnwiki` in a tmux at mwmaint1002 (T318457) |
[production] |
| 07:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1141 (T321123)', diff saved to https://phabricator.wikimedia.org/P38179 and previous config saved to /var/cache/conftool/dbconfig/20221107-070418-marostegui.json |
[production] |
| 07:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db1141 (T321123)', diff saved to https://phabricator.wikimedia.org/P38178 and previous config saved to /var/cache/conftool/dbconfig/20221107-070311-marostegui.json |
[production] |
| 07:03 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1141.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:02 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1141.eqiad.wmnet with reason: Maintenance |
[production] |
| 07:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1121 (T321123)', diff saved to https://phabricator.wikimedia.org/P38177 and previous config saved to /var/cache/conftool/dbconfig/20221107-070249-marostegui.json |
[production] |
| 07:02 |
<urbanecm> |
Run `time mwscript extensions/GrowthExperiments/maintenance/updateIsActiveFlagForMentees.php --wiki=cswiki` in a tmux at mwmaint1002 (T318457) |
[production] |