|
2023-01-23
ยง
|
| 15:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1170:3317 (re)pooling @ 5%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43251 and previous config saved to /var/cache/conftool/dbconfig/20230123-151642-root.json |
[production] |
| 15:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 5%: After adding a column', diff saved to https://phabricator.wikimedia.org/P43250 and previous config saved to /var/cache/conftool/dbconfig/20230123-151611-root.json |
[production] |
| 15:09 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:882661|Revert "Enable Linter write namespace tag and template using core config"]] (duration: 07m 28s) |
[production] |
| 15:03 |
<taavi@deploy1002> |
taavi and trainbranchbot: Backport for [[gerrit:882661|Revert "Enable Linter write namespace tag and template using core config"]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 15:02 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:882661|Revert "Enable Linter write namespace tag and template using core config"]] |
[production] |
| 15:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1170:3317', diff saved to https://phabricator.wikimedia.org/P43248 and previous config saved to /var/cache/conftool/dbconfig/20230123-150110-marostegui.json |
[production] |
| 15:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1105:3312', diff saved to https://phabricator.wikimedia.org/P43247 and previous config saved to /var/cache/conftool/dbconfig/20230123-150018-marostegui.json |
[production] |
| 15:00 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:880989|Enable Linter write namespace tag and template using core config (T299612)]] (duration: 07m 56s) |
[production] |
| 14:59 |
<elukey@deploy1002> |
helmfile [staging] DONE helmfile.d/services/changeprop: sync |
[production] |
| 14:59 |
<elukey@deploy1002> |
helmfile [staging] START helmfile.d/services/changeprop: sync |
[production] |
| 14:53 |
<taavi@deploy1002> |
taavi and sbailey: Backport for [[gerrit:880989|Enable Linter write namespace tag and template using core config (T299612)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 14:52 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:880989|Enable Linter write namespace tag and template using core config (T299612)]] |
[production] |
| 14:46 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:882179|SpecialUserrights: Allow updating the expiry of user groups (T327605)]] (duration: 08m 48s) |
[production] |
| 14:42 |
<sukhe> |
rolling out pybal 1.15.10: T321191 |
[production] |
| 14:39 |
<taavi@deploy1002> |
taavi and func: Backport for [[gerrit:882179|SpecialUserrights: Allow updating the expiry of user groups (T327605)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 14:37 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:882179|SpecialUserrights: Allow updating the expiry of user groups (T327605)]] |
[production] |
| 14:37 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] (duration: 11m 24s) |
[production] |
| 14:27 |
<taavi@deploy1002> |
stang and taavi: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 14:26 |
<otto@deploy1002> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/services/flink-app-example: apply |
[production] |
| 14:26 |
<otto@deploy1002> |
helmfile [dse-k8s-eqiad] START helmfile.d/services/flink-app-example: apply |
[production] |
| 14:25 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] |
[production] |
| 14:25 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:882422|bnwikiquote: Update logo (T323131)]], [[gerrit:882425|shnwikibooks: Add project logo (T327380)]] (duration: 09m 22s) |
[production] |
| 14:25 |
<elukey@deploy1002> |
helmfile [staging] DONE helmfile.d/services/changeprop: sync |
[production] |
| 14:25 |
<elukey@deploy1002> |
helmfile [staging] START helmfile.d/services/changeprop: sync |
[production] |
| 14:20 |
<otto@deploy1002> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 14:20 |
<otto@deploy1002> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 14:18 |
<taavi> |
mwscript extensions/WikimediaMaintenance/createExtensionTables.php --wiki=zhwiki pageassessments # T326387 |
[production] |
| 14:17 |
<taavi@deploy1002> |
taavi and stang: Backport for [[gerrit:882422|bnwikiquote: Update logo (T323131)]], [[gerrit:882425|shnwikibooks: Add project logo (T327380)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 14:16 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:882422|bnwikiquote: Update logo (T323131)]], [[gerrit:882425|shnwikibooks: Add project logo (T327380)]] |
[production] |
| 12:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2107 (T323827)', diff saved to https://phabricator.wikimedia.org/P43246 and previous config saved to /var/cache/conftool/dbconfig/20230123-124532-ladsgroup.json |
[production] |
| 12:30 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2107', diff saved to https://phabricator.wikimedia.org/P43245 and previous config saved to /var/cache/conftool/dbconfig/20230123-123025-ladsgroup.json |
[production] |
| 12:15 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2107', diff saved to https://phabricator.wikimedia.org/P43242 and previous config saved to /var/cache/conftool/dbconfig/20230123-121519-ladsgroup.json |
[production] |
| 12:08 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2114.codfw.wmnet with reason: Maintenance |
[production] |
| 12:08 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2114.codfw.wmnet with reason: Maintenance |
[production] |
| 12:06 |
<marostegui> |
dbmaint Reboot db2135 (m5 codfw master) |
[production] |
| 12:06 |
<marostegui> |
dbmaint Reboot db2134 (m3 codfw master) |
[production] |
| 12:05 |
<Emperor> |
removing /usr/local/bin/prometheus-puppet-agent-stats from prometheus crontab on snapshot1014 |
[production] |
| 12:00 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2107 (T323827)', diff saved to https://phabricator.wikimedia.org/P43241 and previous config saved to /var/cache/conftool/dbconfig/20230123-120012-ladsgroup.json |
[production] |
| 11:58 |
<marostegui> |
dbmaint Reboot db2133 (m2 codfw master) |
[production] |
| 11:57 |
<marostegui> |
dbmaint Reboot db2132 (m1 codfw master) |
[production] |
| 11:57 |
<marostegui> |
Reboot db2132 (m1 codfw master) |
[production] |
| 11:35 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'db2113 (re)pooling @ 100%: Maint done', diff saved to https://phabricator.wikimedia.org/P43239 and previous config saved to /var/cache/conftool/dbconfig/20230123-113506-ladsgroup.json |
[production] |
| 11:31 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2114.codfw.wmnet with reason: Maintenance |
[production] |
| 11:31 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2114.codfw.wmnet with reason: Maintenance |
[production] |
| 11:22 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2114.codfw.wmnet with reason: Maintenance |
[production] |
| 11:22 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2114.codfw.wmnet with reason: Maintenance |
[production] |
| 11:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depool db2114 T327644', diff saved to https://phabricator.wikimedia.org/P43236 and previous config saved to /var/cache/conftool/dbconfig/20230123-112134-ladsgroup.json |
[production] |
| 11:20 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'db2113 (re)pooling @ 75%: Maint done', diff saved to https://phabricator.wikimedia.org/P43235 and previous config saved to /var/cache/conftool/dbconfig/20230123-112001-ladsgroup.json |
[production] |
| 11:18 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Promote db2129 to s6 primary T327644', diff saved to https://phabricator.wikimedia.org/P43234 and previous config saved to /var/cache/conftool/dbconfig/20230123-111813-ladsgroup.json |
[production] |
| 11:17 |
<Amir1> |
Starting s6 codfw failover from db2114 to db2129 - T327644 |
[production] |