|
2024-06-26
§
|
| 14:07 |
<jforrester@deploy1002> |
Finished scap: Backport for [[gerrit:1049857|CodeEditor.vue: add watcher for disabled state (T368504)]] (duration: 08m 00s) |
[production] |
| 14:07 |
<cgoubert@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-api-int: apply |
[production] |
| 14:06 |
<cgoubert@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-api-int: apply |
[production] |
| 14:06 |
<cgoubert@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply |
[production] |
| 14:06 |
<root@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1006.eqiad.wmnet with reason: host reimage |
[production] |
| 14:05 |
<jforrester@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply |
[production] |
| 14:05 |
<cgoubert@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-api-int: apply |
[production] |
| 14:04 |
<jforrester@deploy1002> |
helmfile [eqiad] START helmfile.d/services/wikifunctions: apply |
[production] |
| 14:04 |
<cgoubert@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply |
[production] |
| 14:04 |
<jforrester@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply |
[production] |
| 14:02 |
<jforrester@deploy1002> |
helmfile [codfw] START helmfile.d/services/wikifunctions: apply |
[production] |
| 14:02 |
<eevans@cumin1002> |
START - Cookbook sre.cassandra.roll-restart for nodes matching sessionstore[1004-1006].eqiad.wmnet: Apply Cassandra upgrade to 4.1.5 — T354970 - eevans@cumin1002 |
[production] |
| 14:02 |
<cgoubert@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-api-int: apply |
[production] |
| 14:02 |
<eevans@cumin1002> |
END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching sessionstore[2005-2006].codfw.wmnet: Apply Cassandra upgrade to 4.1.5 — T354970 - eevans@cumin1002 |
[production] |
| 14:01 |
<jforrester@deploy1002> |
jforrester: Continuing with sync |
[production] |
| 14:01 |
<jforrester@deploy1002> |
jforrester: Backport for [[gerrit:1049857|CodeEditor.vue: add watcher for disabled state (T368504)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 14:01 |
<jforrester@deploy1002> |
helmfile [staging] DONE helmfile.d/services/wikifunctions: apply |
[production] |
| 14:01 |
<claime> |
Deploying statsd-exporter for mw-api-int - T365265 |
[production] |
| 14:01 |
<fnegri@cumin1002> |
END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database btmwiki (T368066) |
[production] |
| 14:01 |
<jforrester@deploy1002> |
helmfile [staging] START helmfile.d/services/wikifunctions: apply |
[production] |
| 13:59 |
<jforrester@deploy1002> |
Started scap: Backport for [[gerrit:1049857|CodeEditor.vue: add watcher for disabled state (T368504)]] |
[production] |
| 13:56 |
<Lucas_WMDE> |
UTC afternoon backport+config window done (I might deploy a few more patches later out-of-window) |
[production] |
| 13:55 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1049888|[u4cwiki] Enable importing from dewiki/enwiki/metawiki (T368522)]], [[gerrit:1049929|[arbcom_itwiki] Change the logo and a new wordmark and a favicon (T368532)]] (duration: 08m 49s) |
[production] |
| 13:50 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, superpes: Continuing with sync |
[production] |
| 13:49 |
<eevans@cumin1002> |
START - Cookbook sre.cassandra.roll-restart for nodes matching sessionstore[2005-2006].codfw.wmnet: Apply Cassandra upgrade to 4.1.5 — T354970 - eevans@cumin1002 |
[production] |
| 13:49 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, superpes: Backport for [[gerrit:1049888|[u4cwiki] Enable importing from dewiki/enwiki/metawiki (T368522)]], [[gerrit:1049929|[arbcom_itwiki] Change the logo and a new wordmark and a favicon (T368532)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:46 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Started scap: Backport for [[gerrit:1049888|[u4cwiki] Enable importing from dewiki/enwiki/metawiki (T368522)]], [[gerrit:1049929|[arbcom_itwiki] Change the logo and a new wordmark and a favicon (T368532)]] |
[production] |
| 13:42 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1048408|[ltwiki] Add a new 'rollbacker' usergroup (T367993)]] (duration: 08m 48s) |
[production] |
| 13:37 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, superpes: Continuing with sync |
[production] |
| 13:36 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, superpes: Backport for [[gerrit:1048408|[ltwiki] Add a new 'rollbacker' usergroup (T367993)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:35 |
<fnegri@cumin1002> |
START - Cookbook sre.wikireplicas.add-wiki for database btmwiki (T368066) |
[production] |
| 13:33 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Started scap: Backport for [[gerrit:1048408|[ltwiki] Add a new 'rollbacker' usergroup (T367993)]] |
[production] |
| 13:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1209 (T364069)', diff saved to https://phabricator.wikimedia.org/P65481 and previous config saved to /var/cache/conftool/dbconfig/20240626-133239-marostegui.json |
[production] |
| 13:32 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1209.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:32 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1209.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:32 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1203 (T364069)', diff saved to https://phabricator.wikimedia.org/P65480 and previous config saved to /var/cache/conftool/dbconfig/20240626-133216-marostegui.json |
[production] |
| 13:31 |
<fnegri@cumin1002> |
END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) for database btmwiki (T368066) |
[production] |
| 13:30 |
<Lucas_WMDE> |
lucaswerkmeister-wmde@deploy1002 /srv/mediawiki-staging (master $ u=) $ mwscript-k8s namespaceDupes maiwiki -- --fix # T363667, 0 pages/links to fix, i.e. no-op |
[production] |
| 13:28 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1049667|Meta-Wiki: restrict unfuzzy rights to autoconfirmed (T368416)]], [[gerrit:1031533|maiwiki: Remove 'CA' namespace alias (T363667)]] (duration: 10m 50s) |
[production] |
| 13:28 |
<elukey@cumin1002> |
START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1029.eqiad.wmnet |
[production] |
| 13:23 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, dreamrimmer: Continuing with sync |
[production] |
| 13:23 |
<root@cumin1002> |
START - Cookbook sre.hosts.reimage for host cloudcephosd1006.eqiad.wmnet with OS bullseye |
[production] |
| 13:20 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, dreamrimmer: Backport for [[gerrit:1049667|Meta-Wiki: restrict unfuzzy rights to autoconfirmed (T368416)]], [[gerrit:1031533|maiwiki: Remove 'CA' namespace alias (T363667)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:18 |
<fnegri@cumin1002> |
START - Cookbook sre.wikireplicas.add-wiki for database btmwiki (T368066) |
[production] |
| 13:17 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Started scap: Backport for [[gerrit:1049667|Meta-Wiki: restrict unfuzzy rights to autoconfirmed (T368416)]], [[gerrit:1031533|maiwiki: Remove 'CA' namespace alias (T363667)]] |
[production] |
| 13:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P65479 and previous config saved to /var/cache/conftool/dbconfig/20240626-131709-marostegui.json |
[production] |
| 13:15 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1038741|[CheckUser] Stop writing old for event tables migration on group1 (T360685)]] (duration: 12m 09s) |
[production] |
| 13:13 |
<elukey> |
reload nginx on registry* nodes (Docker registry) to pick up new logging changes |
[production] |
| 13:10 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, dreamyjazz: Continuing with sync |
[production] |
| 13:05 |
<logmsgbot> |
lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, dreamyjazz: Backport for [[gerrit:1038741|[CheckUser] Stop writing old for event tables migration on group1 (T360685)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |