5701-5750 of 10000 results (110ms)
2024-08-08 §
07:40 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1219', diff saved to https://phabricator.wikimedia.org/P67249 and previous config saved to /var/cache/conftool/dbconfig/20240808-074005-marostegui.json [production]
07:32 <dcausse> T371401: reindexing testwikidatawiki to index mul labels [production]
07:24 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1219 (T367856)', diff saved to https://phabricator.wikimedia.org/P67248 and previous config saved to /var/cache/conftool/dbconfig/20240808-072458-marostegui.json [production]
07:19 <hashar> Restarted CI Jenkins for upgrade and plugin update # T371976 [production]
07:11 <dcausse@deploy1003> Finished scap: Backport for [[gerrit:1060430|search: index stems for mul labels (T371401)]] (duration: 09m 03s) [production]
07:06 <dcausse@deploy1003> dcausse: Continuing with sync [production]
07:04 <dcausse@deploy1003> dcausse: Backport for [[gerrit:1060430|search: index stems for mul labels (T371401)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
07:02 <dcausse@deploy1003> Started scap sync-world: Backport for [[gerrit:1060430|search: index stems for mul labels (T371401)]] [production]
06:57 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
06:57 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
06:57 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
06:57 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
06:51 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-wikifunctions: apply [production]
06:51 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-wikifunctions: apply [production]
06:51 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-wikifunctions: apply [production]
06:51 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-wikifunctions: apply [production]
06:51 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
06:51 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
06:51 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
06:51 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
06:51 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-misc: apply [production]
06:51 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-misc: apply [production]
06:51 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-misc: apply [production]
06:51 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-misc: apply [production]
06:51 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-api-int: apply [production]
06:50 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-api-int: apply [production]
06:50 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply [production]
06:50 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-api-int: apply [production]
06:50 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-api-ext: apply [production]
06:50 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-api-ext: apply [production]
06:50 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply [production]
06:50 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply [production]
06:49 <jelto@cumin1002> END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1004.wikimedia.org with reason: Upgrade GitLab Replica to new version [production]
06:48 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-web: apply [production]
06:48 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-web: apply [production]
06:48 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-web: apply [production]
06:47 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-web: apply [production]
06:43 <jelto@cumin1002> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1004.wikimedia.org with reason: Upgrade GitLab Replica to new version [production]
06:42 <hashar> restarting Gerrit [production]
06:42 <jelto@cumin1002> END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Upgrade GitLab Replica to new version [production]
06:41 <oblivian@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-jobrunner: apply [production]
06:41 <oblivian@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-jobrunner: apply [production]
06:41 <oblivian@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-jobrunner: apply [production]
06:41 <oblivian@deploy1003> helmfile [codfw] START helmfile.d/services/mw-jobrunner: apply [production]
06:35 <jelto@cumin1002> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Upgrade GitLab Replica to new version [production]
02:19 <cstone> civicrm upgraded from d1f1d7bd to 686c7c5f [production]
00:30 <rzl@deploy1003> Finished scap: https://gerrit.wikimedia.org/r/1060184 (duration: 02m 33s) [production]
00:29 <rzl@deploy1003> Started scap sync-world: https://gerrit.wikimedia.org/r/1060184 [production]
2024-08-07 §
21:23 <cstone> payments-wiki upgraded from 88500664 to a7f3301a [production]
21:19 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host vrts1003.eqiad.wmnet with OS bookworm [production]