1401-1450 of 10000 results (85ms)
2023-10-30 §
07:22 <marostegui@deploy2002> Started scap: Backport for [[gerrit:969360|Revert "ProductionServices.php: Promote pc1014 to pc1 master"]] [production]
07:22 <marostegui@deploy2002> Finished scap: Backport for [[gerrit:969533|ProductionServices.php: Promote pc1014 to pc1 master]] (duration: 14m 04s) [production]
07:18 <elukey> arm keyholder on acmechief2002 and deploy1002 [production]
07:16 <marostegui@deploy2002> marostegui: Continuing with sync [production]
07:16 <marostegui@deploy2002> marostegui: Backport for [[gerrit:969533|ProductionServices.php: Promote pc1014 to pc1 master]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
07:08 <marostegui@deploy2002> Started scap: Backport for [[gerrit:969533|ProductionServices.php: Promote pc1014 to pc1 master]] [production]
2023-10-28 §
21:25 <fabfur> re-pooled cp1089 and cp3069 [production]
21:05 <fabfur> depooled cp1089 and cp3069 to restart varnish|haproxy and let purged process incoming messages [production]
20:20 <fabfur> restarted purged on cp1089, cp6005, cp3069 [production]
19:46 <fabfur> restarted purged on cp1078 [production]
2023-10-27 §
22:47 <rzl> reprepro -C main include bullseye-wikimedia k8s-controller-sidecars_1.0.2-1_source.changes [production]
22:05 <ejegg> fundraising civicrm upgraded from 74781efd to 2c79475e [production]
15:38 <cmooney@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sretest2004.codfw.wmnet with OS bullseye [production]
15:38 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - cmooney@cumin1001" [production]
15:21 <herron> power cycled titan1001 [production]
14:59 <cmooney@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - cmooney@cumin1001" [production]
14:42 <cmooney@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest2004.codfw.wmnet with reason: host reimage [production]
14:39 <cmooney@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on sretest2004.codfw.wmnet with reason: host reimage [production]
14:19 <topranks> announcing internal core routes to esams asw's to test policy T344547 [production]
14:19 <jayme@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
14:18 <jayme@deploy2002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
14:12 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host sretest1004.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:12 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host sretest1004.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:04 <jayme@deploy2002> helmfile [codfw] DONE helmfile.d/services/developer-portal: apply [production]
14:04 <jayme@deploy2002> helmfile [codfw] START helmfile.d/services/developer-portal: apply [production]
14:04 <jayme@deploy2002> helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply [production]
14:03 <jayme@deploy2002> helmfile [eqiad] START helmfile.d/services/developer-portal: apply [production]
14:03 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/developer-portal: apply [production]
14:02 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/developer-portal: apply [production]
13:38 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host acmechief2002.codfw.wmnet [production]
13:38 <cmooney@cumin1001> START - Cookbook sre.hosts.reimage for host sretest2004.codfw.wmnet with OS bullseye [production]
13:37 <cmooney@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2004.codfw.wmnet with OS bullseye [production]
13:36 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:36 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: change sretest2004 DNS - cmooney@cumin1001" [production]
13:35 <cmooney@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: change sretest2004 DNS - cmooney@cumin1001" [production]
13:33 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
13:31 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-host for host acmechief2002.codfw.wmnet [production]
13:27 <jbond@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host acmechief2002.codfw.wmnet [production]
13:27 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host acmechief2002.codfw.wmnet with OS bookworm [production]
13:00 <cmooney@cumin1001> START - Cookbook sre.hosts.reimage for host sretest2004.codfw.wmnet with OS bullseye [production]
12:55 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
12:54 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:41 <jayme> updated mwdebug1001 to icu67 - T345561 [production]
12:17 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on acmechief2002.codfw.wmnet with reason: host reimage [production]
12:14 <jbond@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on acmechief2002.codfw.wmnet with reason: host reimage [production]
11:52 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp1102.eqiad.wmnet with OS bullseye [production]
11:34 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1102.eqiad.wmnet with reason: host reimage [production]
11:31 <fabfur@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp1102.eqiad.wmnet with reason: host reimage [production]
11:31 <jbond@cumin1001> START - Cookbook sre.hosts.reimage for host acmechief2002.codfw.wmnet with OS bookworm [production]
11:30 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM acmechief2002.codfw.wmnet - jbond@cumin1001" [production]