1-50 of 10000 results (21ms)
2026-05-14 ยง
19:15 <vriley@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1274.eqiad.wmnet with OS bookworm [production]
19:14 <vriley@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" [production]
19:06 <ejegg> fundraising civicrm upgraded from 950908ec to e25fa223 [fundraising]
19:06 <vriley@cumin1003> START - Cookbook sre.hosts.reimage for host db1281.eqiad.wmnet with OS bookworm [production]
19:04 <ejegg> disabled fundraising scheduled jobs for CiviCRM deployment [fundraising]
18:58 <jforrester@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
18:57 <jforrester@deploy1003> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
18:53 <wmftkbot> Test Kitchen edge-unique experiments (poll 239209) - adds: none; removes: synth-aa-ncs-1; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD [analytics]
18:49 <wm-bot2> Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25878752531 (https://github.com/cluebotng/component-configs/commits/21e928fa1870ddaf5fae15afc6f92aa3cb3fb970) [tools.cluebotng-review]
18:48 <wm-bot2> Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25878752565 (https://github.com/cluebotng/component-configs/commits/21e928fa1870ddaf5fae15afc6f92aa3cb3fb970) [tools.cluebotng]
18:47 <vriley@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" [production]
18:40 <vriley@cumin1003> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host db1281.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
18:29 <vriley@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1274.eqiad.wmnet with reason: host reimage [production]
18:29 <wm-bot2> Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25877799217 (https://github.com/cluebotng/component-configs/commits/df1444522160198ee3ab31965936679052be618b) [tools.cluebotng-monitoring]
18:25 <vriley@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on db1274.eqiad.wmnet with reason: host reimage [production]
18:17 <vriley@cumin1003> START - Cookbook sre.hosts.provision for host db1281.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
18:16 <vriley@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:14 <vriley@cumin1003> START - Cookbook sre.dns.netbox [production]
18:09 <vriley@cumin1003> START - Cookbook sre.hosts.reimage for host db1274.eqiad.wmnet with OS bookworm [production]
17:32 <jforrester@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
17:31 <jforrester@deploy1003> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
17:23 <aokoth@cumin1003> END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T426298 [production]
17:17 <bd808@deploy1003> helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply [production]
17:17 <bd808@deploy1003> helmfile [eqiad] START helmfile.d/services/developer-portal: apply [production]
17:16 <bd808@deploy1003> helmfile [codfw] DONE helmfile.d/services/developer-portal: apply [production]
17:16 <bd808@deploy1003> helmfile [codfw] START helmfile.d/services/developer-portal: apply [production]
17:16 <bd808@deploy1003> helmfile [staging] DONE helmfile.d/services/developer-portal: apply [production]
17:15 <bd808@deploy1003> helmfile [staging] START helmfile.d/services/developer-portal: apply [production]
17:14 <aokoth@cumin1003> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T426298 [production]
17:10 <cmooney@dns2005> END - running authdns-update [production]
17:09 <cmooney@dns2005> START - running authdns-update [production]
17:06 <aokoth@cumin1003> END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T426298 [production]
17:02 <wm-bot2> Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/25873443162 (https://github.com/cluebotng/component-configs/commits/815c3361fd913b83b92ecd2a0d61ae298b921dda) [tools.cluebotng-trainer]
16:58 <aokoth@cumin1003> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T426298 [production]
16:49 <atsuko@deploy1003> helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply [production]
16:49 <atsuko@deploy1003> helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply [production]
16:48 <ejegg> standalone (IPN listener) SmashPig upgraded from bb833986 to a2a3a015 [fundraising]
16:36 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply [production]
16:36 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply [production]
16:35 <elukey@cumin1003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host kafka-logging1006.eqiad.wmnet with OS trixie [production]
16:31 <topranks> disable core router direct link at esams now that traffic is flowing via switches T424611 [production]
16:25 <topranks> disable core router direct link at drmrs now that traffic is flowing via switches T424611 [production]
16:21 <topranks> disable core router direct link at magru now that traffic is flowing via switches T424611 [production]
16:20 <rzl@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-cron: apply [production]
16:20 <rzl@deploy1003> helmfile [codfw] START helmfile.d/services/mw-cron: apply [production]
16:19 <rzl@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply [production]
16:19 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers [tools]
16:18 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers [tools]
16:17 <rzl@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-cron: apply [production]
16:16 <vriley@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db1289.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]