|
2026-09-12
§
|
| 19:40 |
<ladsgroup@cumin1003> |
END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-eqiad |
[production] |
| 19:32 |
<ladsgroup@cumin1003> |
START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-eqiad |
[production] |
| 19:30 |
<ladsgroup@cumin1003> |
END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-codfw |
[production] |
| 19:21 |
<ladsgroup@cumin1003> |
START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-codfw |
[production] |
| 18:23 |
<Southparkfan> |
project-wide puppet hiera: remove puppetmaster::geoip::{fetch_private,use_proxy}, puppetdb_host, profile::puppetmaster::common::{command_broadcast,puppetdb_host,puppetdb_hosts} |
[releng] |
| 18:23 |
<Southparkfan> |
project-wide puppet hiera: remove puppetmaster::geoip::{fetch_private,use_proxy}, puppetdb_host, profile::puppetmaster::common::{command_broadcast,puppetdb_host,puppetdb_hosts} |
[deployment-prep] |
| 18:14 |
<Southparkfan> |
project-wide puppet hiera: remove role::puppetmaster::puppetdb::shared_buffers, obsoleted by profile::puppetdb::database::shared_buffers |
[releng] |
| 18:14 |
<Southparkfan> |
project-wide puppet hiera: remove role::puppetmaster::puppetdb::shared_buffers, obsoleted by profile::puppetdb::database::shared_buffers |
[deployment-prep] |
| 18:09 |
<Southparkfan> |
project-wide puppet hiera: remove profile::puppetdb::master pointing to former deployment-puppetdb02 (overridden in deployment-puppetdb prefix), profile::puppetdb::slaves (by default already empty) + profile::puppetdb::extra_authorized_hosts (variable does not exist, couldn't find it in historic commits either) |
[releng] |
| 18:09 |
<Southparkfan> |
project-wide puppet hiera: remove profile::puppetdb::master pointing to former deployment-puppetdb02 (overridden in deployment-puppetdb prefix), profile::puppetdb::slaves (by default already empty) + profile::puppetdb::extra_authorized_hosts (variable does not exist, couldn't find it in historic commits either) |
[deployment-prep] |
| 02:08 |
<mwpresync@deploy1003> |
Finished scap build-images: Publishing wmf/next image (duration: 07m 35s) |
[production] |
| 02:00 |
<mwpresync@deploy1003> |
Started scap build-images: Publishing wmf/next image |
[production] |
|
2026-09-11
§
|
| 21:51 |
<amastilovic@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/blunderbuss: apply |
[production] |
| 21:50 |
<amastilovic@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/blunderbuss: apply |
[production] |
| 16:47 |
<sbassett@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1339808|Use escaped() for story link parentheses in recent changes (T182213)]], [[gerrit:1339813|Use escaped() for HTML parentheses params in ChangeLineFormatter (T182213)]] (duration: 07m 23s) |
[production] |
| 16:43 |
<sbassett@deploy1003> |
sbassett: Continuing with deployment |
[production] |
| 16:42 |
<sbassett@deploy1003> |
sbassett: Backport for [[gerrit:1339808|Use escaped() for story link parentheses in recent changes (T182213)]], [[gerrit:1339813|Use escaped() for HTML parentheses params in ChangeLineFormatter (T182213)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 16:40 |
<sbassett@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1339808|Use escaped() for story link parentheses in recent changes (T182213)]], [[gerrit:1339813|Use escaped() for HTML parentheses params in ChangeLineFormatter (T182213)]] |
[production] |
| 16:29 |
<dancy> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/1339162 |
[releng] |
| 16:08 |
<bking@cumin2003> |
END (FAIL) - Cookbook sre.hadoop.roll-restart-workers (exit_code=99) restart workers for Hadoop analytics cluster: Roll restart of jvm daemons for openjdk upgrade. |
[production] |
| 14:39 |
<bking@cumin2003> |
START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop analytics cluster: Roll restart of jvm daemons for openjdk upgrade. |
[production] |
| 14:10 |
<ssastry@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply |
[production] |
| 14:10 |
<ssastry@deploy1003> |
helmfile [codfw] START helmfile.d/services/mw-parsoid: apply |
[production] |
| 14:10 |
<ssastry@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply |
[production] |
| 14:09 |
<ssastry@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply |
[production] |
| 13:40 |
<bking@cumin2003> |
END (FAIL) - Cookbook sre.hadoop.roll-restart-workers (exit_code=99) restart workers for Hadoop analytics cluster: Roll restart of jvm daemons for openjdk upgrade. |
[production] |
| 13:33 |
<sfaci@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply |
[production] |
| 13:33 |
<sfaci@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply |
[production] |
| 13:28 |
<bking@cumin2003> |
START - Cookbook sre.hadoop.roll-restart-workers restart workers for Hadoop analytics cluster: Roll restart of jvm daemons for openjdk upgrade. |
[production] |
| 13:16 |
<brouberol@deploy1003> |
helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 13:15 |
<brouberol@deploy1003> |
helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 13:11 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 13:10 |
<brouberol@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 11:20 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1199: db1199 repool |
[production] |
| 11:05 |
<moritzm> |
installing Linux 6.1.187 on Bookworm hosts |
[production] |
| 11:05 |
<jmm@cumin2003> |
DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Jcrespo out of all services on: 2443 hosts |
[production] |
| 10:44 |
<aokoth@cumin1004> |
END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1004.wikimedia.org with reason: Security Release - T437680 |
[production] |
| 10:43 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup2020 in turn |
[production] |
| 10:43 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup2019 in turn |
[production] |
| 10:41 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup2018 in turn |
[production] |
| 10:40 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup2017 in turn |
[production] |
| 10:39 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup2016 in turn |
[production] |
| 10:37 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup2015 in turn |
[production] |
| 10:36 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup1020 in turn |
[production] |
| 10:35 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup1019 in turn |
[production] |
| 10:34 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.pool pool db1199: db1199 repool |
[production] |
| 10:33 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup1018 in turn |
[production] |
| 10:32 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup1017 in turn |
[production] |
| 10:30 |
<Emperor> |
restart versitygw@objectstorage0[0-3].service on backup1016 in turn |
[production] |
| 10:20 |
<Emperor> |
restart versitygw@objectstorage0[1-3].service on backup1015 in turn |
[production] |