|
2026-08-24
ยง
|
| 22:17 |
<ryankemper@cumin2003> |
END (PASS) - Cookbook sre.presto.roll-restart-workers (exit_code=0) for Presto an-presto cluster: Roll restart of all Presto's jvm daemons. |
[production] |
| 22:02 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.ceph.osd.undrain_node |
[admin] |
| 22:01 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.ceph.osd.reactivate (exit_code=0) |
[admin] |
| 22:01 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.ceph.osd.reactivate |
[admin] |
| 22:01 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=0) |
[admin] |
| 22:01 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.ceph.osd.bootstrap_and_add |
[admin] |
| 21:49 |
<ryankemper> |
T435862 [presto] Rolling restart in progress to activate `experimental.spill-enabled=false `from https://gerrit.wikimedia.org/r/c/operations/puppet/+/1328713 |
[production] |
| 21:45 |
<ryankemper> |
T435861 [presto] Kicking off rolling restart to pick up `experimental.spill-enabled=false` changes |
[production] |
| 21:44 |
<ryankemper@cumin2003> |
START - Cookbook sre.presto.roll-restart-workers for Presto an-presto cluster: Roll restart of all Presto's jvm daemons. |
[production] |
| 21:41 |
<sbassett> |
Deployed security fix for T435624 |
[production] |
| 21:21 |
<sbassett> |
Deployed security mitigation for T435455 |
[production] |
| 21:02 |
<arlolra@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1328707|Set a default for wgUseParsoidLinksUpdate in prod (T432760)]] (duration: 07m 07s) |
[production] |
| 20:58 |
<arlolra@deploy1003> |
arlolra: Continuing with deployment |
[production] |
| 20:57 |
<arlolra@deploy1003> |
arlolra: Backport for [[gerrit:1328707|Set a default for wgUseParsoidLinksUpdate in prod (T432760)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:55 |
<arlolra@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1328707|Set a default for wgUseParsoidLinksUpdate in prod (T432760)]] |
[production] |
| 20:48 |
<aaron@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1326410|Mark all Mathoid-based endpoints as deprecated (T431372)]] (duration: 12m 33s) |
[production] |
| 20:44 |
<aaron@deploy1003> |
aaron: Continuing with deployment |
[production] |
| 20:37 |
<aaron@deploy1003> |
aaron: Backport for [[gerrit:1326410|Mark all Mathoid-based endpoints as deprecated (T431372)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:35 |
<aaron@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1326410|Mark all Mathoid-based endpoints as deprecated (T431372)]] |
[production] |
| 20:34 |
<aaron@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1324819|Add configurable RestModuleOverrides (T434267)]] (duration: 09m 35s) |
[production] |
| 20:29 |
<aaron@deploy1003> |
aaron, milazg: Continuing with deployment |
[production] |
| 20:26 |
<aaron@deploy1003> |
aaron, milazg: Backport for [[gerrit:1324819|Add configurable RestModuleOverrides (T434267)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:24 |
<aaron@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1324819|Add configurable RestModuleOverrides (T434267)]] |
[production] |
| 20:16 |
<kemayo@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1328674|Update VE core submodule to master (5e8709982) (T414328 T433533 T435688)]] (duration: 10m 47s) |
[production] |
| 20:09 |
<kemayo@deploy1003> |
kemayo: Continuing with deployment |
[production] |
| 20:07 |
<kemayo@deploy1003> |
kemayo: Backport for [[gerrit:1328674|Update VE core submodule to master (5e8709982) (T414328 T433533 T435688)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:05 |
<kemayo@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1328674|Update VE core submodule to master (5e8709982) (T414328 T433533 T435688)]] |
[production] |
| 19:47 |
<musikanimal@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1328386|CodeMirror: enable JSONC mode (T127682)]] (duration: 07m 04s) |
[production] |
| 19:43 |
<musikanimal@deploy1003> |
musikanimal: Continuing with deployment |
[production] |
| 19:42 |
<musikanimal@deploy1003> |
musikanimal: Backport for [[gerrit:1328386|CodeMirror: enable JSONC mode (T127682)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 19:40 |
<musikanimal@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1328386|CodeMirror: enable JSONC mode (T127682)]] |
[production] |
| 19:10 |
<inflatador> |
bking@krb2002 ktadd -k /tmp/krb1003.keytab host/krb1003.eqiad.wmnet@WIKIMEDIA T435854 |
[production] |
| 19:09 |
<inflatador> |
bking@krb2002 addprinc -randkey host/krb1003.eqiad.wmnet@WIKIMEDIA T435854 |
[production] |
| 18:58 |
<ejegg> |
re-enabled payment attempts queue consumer |
[fundraising] |
| 18:54 |
<urbanecm@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1328660|[Growth] eswiki: Increase mentorship to 85% (T394867)]] (duration: 08m 47s) |
[production] |
| 18:53 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) on eqiad1, with recreate False, for hosts list: ['cloudvirt1077'] |
[cloudvirt-canary] |
| 18:52 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on eqiad1, with recreate False, for hosts list: ['cloudvirt1077'] |
[cloudvirt-canary] |
| 18:52 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T435537) |
[admin] |
| 18:52 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T435537) |
[admin] |
| 18:48 |
<urbanecm@deploy1003> |
urbanecm: Continuing with deployment |
[production] |
| 18:47 |
<urbanecm@deploy1003> |
urbanecm: Backport for [[gerrit:1328660|[Growth] eswiki: Increase mentorship to 85% (T394867)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 18:45 |
<urbanecm@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1328660|[Growth] eswiki: Increase mentorship to 85% (T394867)]] |
[production] |
| 18:34 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) on host 'cloudvirt1077.eqiad.wmnet' (T435537) |
[admin] |
| 18:16 |
<bd808@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply |
[production] |
| 18:14 |
<bd808@deploy1003> |
helmfile [eqiad] START helmfile.d/services/shellbox-video: apply |
[production] |
| 18:14 |
<bd808@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply |
[production] |
| 18:14 |
<bd808@deploy1003> |
helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply |
[production] |
| 18:13 |
<bd808@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply |
[production] |
| 18:13 |
<bd808@deploy1003> |
helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply |
[production] |
| 18:13 |
<bd808@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply |
[production] |