6051-6100 of 10000 results (145ms)
2024-03-26 ยง
15:32 <gmodena@deploy1002> helmfile [staging] DONE helmfile.d/services/eventstreams: apply [production]
15:32 <gmodena@deploy1002> helmfile [staging] START helmfile.d/services/eventstreams: apply [production]
15:30 <gmodena@deploy1002> helmfile [staging] DONE helmfile.d/services/eventstreams-internal: apply [production]
15:30 <gmodena@deploy1002> helmfile [staging] START helmfile.d/services/eventstreams-internal: apply [production]
15:30 <jebe@deploy1002> Started deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] [production]
15:28 <gmodena@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventstreams: apply [production]
15:27 <gmodena@deploy1002> helmfile [codfw] START helmfile.d/services/eventstreams: apply [production]
15:27 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dbprov2005.mgmt.codfw.wmnet with reboot policy FORCED [production]
15:26 <gmodena@deploy1002> helmfile [staging] DONE helmfile.d/services/eventstreams: apply [production]
15:26 <gmodena@deploy1002> helmfile [staging] START helmfile.d/services/eventstreams: apply [production]
15:25 <brouberol@cumin1002> START - Cookbook sre.hosts.decommission for hosts an-coord1002.eqiad.wmnet [production]
15:23 <brouberol@cumin1002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts an-coord1001.eqiad.wmnet [production]
15:23 <brouberol@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:23 <brouberol@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-coord1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin1002" [production]
15:22 <brouberol@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-coord1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin1002" [production]
15:18 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host dbprov2005.mgmt.codfw.wmnet with reboot policy FORCED [production]
15:15 <brouberol@cumin1002> START - Cookbook sre.dns.netbox [production]
15:09 <brouberol@cumin1002> START - Cookbook sre.hosts.decommission for hosts an-coord1001.eqiad.wmnet [production]
14:56 <jnuche@deploy1002> Installation of scap version "4.73.2" completed for 364 hosts [production]
14:55 <jnuche@deploy1002> Installing scap version "4.73.2" for 364 hosts [production]
14:26 <jayme@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
14:25 <jayme@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
14:25 <jayme@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
14:24 <jayme@deploy1002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
14:24 <jayme@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
14:22 <jayme@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
14:21 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
14:20 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
14:20 <claime> Deploying split listener for 10% of backend restbase traffic to mw-api-int - T358213 [production]
14:19 <claime> enabling and running puppet on P:restbase - T358213 [production]
14:19 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
14:19 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
14:18 <TheresNoTime> UTC afternoon backport window done [production]
14:15 <samtar@deploy1002> Finished scap: Backport for [[gerrit:1014509|dewiki: Enable mobile page tabs for everyone (T360246)]], [[gerrit:1011097|frwiki: update legacy vector logo (T359741)]] (duration: 17m 23s) [production]
14:13 <vriley@cumin1002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts parse1014.eqiad.wmnet [production]
14:04 <samtar@deploy1002> anzx and samtar: Continuing with sync [production]
14:03 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
14:02 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
14:02 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
14:01 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
14:01 <vriley@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts parse1014.eqiad.wmnet [production]
14:00 <samtar@deploy1002> anzx and samtar: Backport for [[gerrit:1014509|dewiki: Enable mobile page tabs for everyone (T360246)]], [[gerrit:1011097|frwiki: update legacy vector logo (T359741)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:57 <samtar@deploy1002> Started scap: Backport for [[gerrit:1014509|dewiki: Enable mobile page tabs for everyone (T360246)]], [[gerrit:1011097|frwiki: update legacy vector logo (T359741)]] [production]
13:55 <samtar@deploy1002> Finished scap: Backport for [[gerrit:1010881|knwikisource, knwiktionary: update logo, wordmark (T360022)]] (duration: 15m 53s) [production]
13:43 <samtar@deploy1002> anzx and samtar: Continuing with sync [production]
13:41 <samtar@deploy1002> anzx and samtar: Backport for [[gerrit:1010881|knwikisource, knwiktionary: update logo, wordmark (T360022)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:39 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1158 (T352010)', diff saved to https://phabricator.wikimedia.org/P58921 and previous config saved to /var/cache/conftool/dbconfig/20240326-133932-ladsgroup.json [production]
13:39 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
13:39 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
13:39 <samtar@deploy1002> Started scap: Backport for [[gerrit:1010881|knwikisource, knwiktionary: update logo, wordmark (T360022)]] [production]