|
2023-06-14
ยง
|
| 23:38 |
<mvernon@cumin1001> |
END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe |
[production] |
| 23:30 |
<mvernon@cumin1001> |
START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe |
[production] |
| 22:15 |
<wm-bot> |
<bd808> Updated to d041d0f |
[tools.toolinfo-scraper] |
| 21:39 |
<Krinkle> |
Unroll local patch c7d03fe1d2f5 from 2016 adding `modules/secret/secrets/phabricator/phab_deploy_private_key` in deploy-puppetmaster04:/var/lib/git/labs/private.git. This references https://gerrit.wikimedia.org/r/c/operations/puppet/+/432528 which is "no longer needed as of 2019". |
[releng] |
| 21:38 |
<mutante> |
phabricator - made dancy (https://phabricator.wikimedia.org/people/manage/25411/) and administrator (T339174) |
[production] |
| 21:38 |
<Krinkle> |
Unroll local patch 772ed98a88eaca2 from 2018 adding `modules/secret/secrets/phabricator/swift_key` in deploy-puppetmaster04:/var/lib/git/labs/private.git. This references https://gerrit.wikimedia.org/r/c/operations/puppet/+/432528 which is "no longer needed as of 2019". |
[releng] |
| 21:20 |
<Krinkle> |
Unroll local patch from 2018 for `profile::cache::kafka::certificate::ssl_key_password` in deploy-puppetmaster04:/var/lib/git/labs/private.git. This was already obsoleted by a (different) value in Horizon. |
[releng] |
| 21:17 |
<Krinkle> |
Unroll local patch from 2018 for `profile::maps::osm_master::kartotherian_pass` in deploy-puppetmaster04:/var/lib/git/labs/private.git, move to Horizon |
[releng] |
| 21:09 |
<Krinkle> |
Unroll local patch from 2017 to disable `puppetdb::password::replication` in deploy-puppetmaster04:/var/lib/git/labs/private.git, move to Horizon |
[releng] |
| 21:06 |
<Krinkle> |
Unroll 5 local patches for global_account_keys in deploy-puppetmaster04:/var/lib/git/labs/private.git, move to Horizon and rename to match prod. Fixes T336191. |
[releng] |
| 21:05 |
<Krinkle> |
Unroll local patch for osm_password in deploy-puppetmaster04:/var/lib/git/labs/private.git, move to Horizon. |
[releng] |
| 21:04 |
<Krinkle> |
Unroll local patch for puppetdb::password::rw in deploy-puppetmaster04:/var/lib/git/labs/private.git, move to Horizon. |
[releng] |
| 21:02 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:930249|Fix thumb styling on file description page (T337804)]] (duration: 10m 44s) |
[production] |
| 20:54 |
<taavi@deploy1002> |
arlolra and taavi: Backport for [[gerrit:930249|Fix thumb styling on file description page (T337804)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 20:52 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:930249|Fix thumb styling on file description page (T337804)]] |
[production] |
| 20:34 |
<aokoth@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on otrs1001.eqiad.wmnet with reason: Replacing Host |
[production] |
| 20:34 |
<aokoth@cumin1001> |
START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on otrs1001.eqiad.wmnet with reason: Replacing Host |
[production] |
| 20:29 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:929742|Enable mobile page tabs for everyone in ptwikisource. (T338974)]] (duration: 10m 23s) |
[production] |
| 20:21 |
<taavi@deploy1002> |
taavi and albertoleoncio: Backport for [[gerrit:929742|Enable mobile page tabs for everyone in ptwikisource. (T338974)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet |
[production] |
| 20:19 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:929742|Enable mobile page tabs for everyone in ptwikisource. (T338974)]] |
[production] |
| 20:18 |
<milimetric> |
reran mediawiki_history_reduced druid load task after deploying Joseph's fix |
[analytics] |
| 20:12 |
<taavi@deploy1002> |
Finished scap: Backport for [[gerrit:930206|simplewiki: Remove "changetags" from registered user (T339124)]] (duration: 08m 55s) |
[production] |
| 20:10 |
<mutante> |
https://ticket.wikimedia.org down for migration |
[production] |
| 20:06 |
<taavi@deploy1002> |
taavi and stang: Backport for [[gerrit:930206|simplewiki: Remove "changetags" from registered user (T339124)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 20:03 |
<taavi@deploy1002> |
Started scap: Backport for [[gerrit:930206|simplewiki: Remove "changetags" from registered user (T339124)]] |
[production] |
| 20:01 |
<aokoth@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on otrs1001.eqiad.wmnet with reason: Replacing Host |
[production] |
| 20:01 |
<aokoth@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on otrs1001.eqiad.wmnet with reason: Replacing Host |
[production] |
| 18:29 |
<milimetric@deploy1002> |
Finished deploy [airflow-dags/analytics@3d6caed]: Deploying mostly to rerun druid loading for mediawiki history reduced (duration: 00m 09s) |
[production] |
| 18:29 |
<milimetric@deploy1002> |
Started deploy [airflow-dags/analytics@3d6caed]: Deploying mostly to rerun druid loading for mediawiki history reduced |
[production] |
| 18:11 |
<moritzm> |
installing libssh security updates on buster |
[production] |
| 17:04 |
<otto@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/eventgate-main: apply |
[production] |
| 17:03 |
<otto@deploy1002> |
helmfile [eqiad] START helmfile.d/services/eventgate-main: apply |
[production] |
| 17:03 |
<otto@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/eventgate-main: apply |
[production] |
| 17:02 |
<otto@deploy1002> |
helmfile [codfw] START helmfile.d/services/eventgate-main: apply |
[production] |
| 17:01 |
<otto@deploy1002> |
helmfile [staging] DONE helmfile.d/services/eventgate-main: apply |
[production] |
| 17:01 |
<otto@deploy1002> |
helmfile [staging] START helmfile.d/services/eventgate-main: apply |
[production] |
| 16:52 |
<bking@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs2021.codfw.wmnet with OS bullseye |
[production] |
| 16:51 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1492.mgmt.eqiad.wmnet with reboot policy FORCED |
[production] |
| 16:48 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.provision for host mw1492.mgmt.eqiad.wmnet with reboot policy FORCED |
[production] |
| 16:26 |
<hashar> |
integration-agent-docker-1025: rm -fR /srv/jenkins/workspace/* |
[releng] |
| 16:26 |
<hashar> |
integration-agent-docker-1037: rm -fR /srv/jenkins/workspace/* |
[releng] |
| 15:42 |
<jnuche@deploy1002> |
Finished deploy [releng/jenkins-deploy@0c82f2d] (releasing): (no justification provided) (duration: 02m 03s) |
[production] |
| 15:40 |
<jnuche@deploy1002> |
Started deploy [releng/jenkins-deploy@0c82f2d] (releasing): (no justification provided) |
[production] |
| 15:00 |
<lucaswerkmeister-wmde:> |
Deployed security patch for T250720 |
[production] |
| 14:53 |
<lucaswerkmeister-wmde:> |
Deployed security patch for T250720 |
[production] |
| 14:36 |
<Amir1> |
mwscript findBadBlobs.php --wiki=nlwiki --revisions 880583,880584,880585,880586,880587,880588,880589,880590,880591,880592,880593,880594,880595,880596,880597,880598,880599,880600,880601,880602 --mark "T128154" |
[production] |
| 14:33 |
<hnowlan@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/thumbor: apply |
[production] |
| 14:32 |
<hnowlan@deploy1002> |
helmfile [codfw] START helmfile.d/services/thumbor: apply |
[production] |
| 14:32 |
<hnowlan@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/thumbor: apply |
[production] |
| 14:30 |
<hnowlan@deploy1002> |
helmfile [eqiad] START helmfile.d/services/thumbor: apply |
[production] |