7401-7450 of 10000 results (135ms)
2023-04-27 ยง
14:58 <cgoubert@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2330.codfw.wmnet [production]
14:58 <cgoubert@cumin1001> START - Cookbook sre.hosts.remove-downtime for mw2330.codfw.wmnet [production]
14:56 <Lucas_WMDE> UTC afternoon backport+config window done [production]
14:55 <lucaswerkmeister-wmde@deploy1002> Finished scap: Backport for [[gerrit:912290|Add language codes cal and tpv to wmgExtraLanguageNames (T308062)]] (duration: 07m 55s) [production]
14:49 <lucaswerkmeister-wmde@deploy1002> lucaswerkmeister-wmde and noa: Backport for [[gerrit:912290|Add language codes cal and tpv to wmgExtraLanguageNames (T308062)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet [production]
14:47 <lucaswerkmeister-wmde@deploy1002> Started scap: Backport for [[gerrit:912290|Add language codes cal and tpv to wmgExtraLanguageNames (T308062)]] [production]
14:46 <ejegg> payments-wiki upgraded from f30bc859 to 7fa25437 [production]
14:46 <lucaswerkmeister-wmde@deploy1002> Finished scap: Backport for [[gerrit:912815|lowiki: Use Western style (0-9) numerals (T335345)]] (duration: 08m 53s) [production]
14:38 <lucaswerkmeister-wmde@deploy1002> lucaswerkmeister-wmde and stang: Backport for [[gerrit:912815|lowiki: Use Western style (0-9) numerals (T335345)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet [production]
14:37 <ejegg> disabled fundraising job ingenico_recurring_fill_scheme_ids (it's all done) [production]
14:37 <lucaswerkmeister-wmde@deploy1002> Started scap: Backport for [[gerrit:912815|lowiki: Use Western style (0-9) numerals (T335345)]] [production]
14:36 <vgutierrez> restarting varnish on cp5020 and cp5028 to drop port 80 - T322774 [production]
14:35 <lucaswerkmeister-wmde@deploy1002> Finished scap: Backport for [[gerrit:911308|Close cnwikimedia (T274083)]] (duration: 11m 05s) [production]
14:29 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-timeline: apply [production]
14:28 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-timeline: apply [production]
14:28 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
14:28 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
14:27 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-media: apply [production]
14:27 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-media: apply [production]
14:27 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply [production]
14:26 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-constraints: apply [production]
14:26 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox: apply [production]
14:26 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox: apply [production]
14:25 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox: apply [production]
14:25 <lucaswerkmeister-wmde@deploy1002> lucaswerkmeister-wmde and stang: Backport for [[gerrit:911308|Close cnwikimedia (T274083)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
14:25 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox: apply [production]
14:25 <moritzm> restarting apache/FPM on mw canaries to pick up curl update [production]
14:24 <lucaswerkmeister-wmde@deploy1002> Started scap: Backport for [[gerrit:911308|Close cnwikimedia (T274083)]] [production]
14:20 <lucaswerkmeister-wmde@deploy1002> Finished scap: Backport for [[gerrit:912337|labtestwiki: disable cirrus completion index]] (duration: 09m 31s) [production]
14:13 <moritzm> installing curl security updates on buster [production]
14:12 <lucaswerkmeister-wmde@deploy1002> dcausse and lucaswerkmeister-wmde: Backport for [[gerrit:912337|labtestwiki: disable cirrus completion index]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
14:11 <lucaswerkmeister-wmde@deploy1002> Started scap: Backport for [[gerrit:912337|labtestwiki: disable cirrus completion index]] [production]
14:09 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ml-cache1003.eqiad.wmnet with OS bullseye [production]
14:05 <samtar@deploy1002> Finished scap: Backport for [[gerrit:910056|Enable $wgCampaignEventsEnableMultipleOrganizers in production (T334088)]] (duration: 38m 35s) [production]
14:00 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
13:59 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
13:59 <legoktm@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox: apply [production]
13:59 <legoktm@deploy1002> helmfile [staging] START helmfile.d/services/shellbox: apply [production]
13:48 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ml-cache1003.eqiad.wmnet with reason: host reimage [production]
13:45 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on ml-cache1003.eqiad.wmnet with reason: host reimage [production]
13:35 <akosiaris@deploy1002> helmfile [staging] DONE helmfile.d/services/machinetranslation: apply [production]
13:33 <elukey@cumin1001> START - Cookbook sre.hosts.reimage for host ml-cache1003.eqiad.wmnet with OS bullseye [production]
13:31 <akosiaris@deploy1002> helmfile [staging] START helmfile.d/services/machinetranslation: apply [production]
13:30 <akosiaris@deploy1002> helmfile [staging] DONE helmfile.d/services/machinetranslation: apply [production]
13:28 <samtar@deploy1002> samtar and cmelo: Backport for [[gerrit:910056|Enable $wgCampaignEventsEnableMultipleOrganizers in production (T334088)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet [production]
13:26 <samtar@deploy1002> Started scap: Backport for [[gerrit:910056|Enable $wgCampaignEventsEnableMultipleOrganizers in production (T334088)]] [production]
13:20 <samtar@deploy1002> Finished scap: Backport for [[gerrit:910055|metawiki: Give campaignevents-organize-events to campaignevents-beta-tester only (T334088)]] (duration: 15m 07s) [production]
13:20 <akosiaris@deploy1002> helmfile [staging] START helmfile.d/services/machinetranslation: apply [production]
13:06 <samtar@deploy1002> samtar and cmelo: Backport for [[gerrit:910055|metawiki: Give campaignevents-organize-events to campaignevents-beta-tester only (T334088)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
13:05 <samtar@deploy1002> Started scap: Backport for [[gerrit:910055|metawiki: Give campaignevents-organize-events to campaignevents-beta-tester only (T334088)]] [production]