1801-1850 of 10000 results (72ms)
2023-11-21 ยง
16:03 <taavi@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:03 <taavi@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: allocate cloud-private svc ips to wiki replicas - taavi@cumin1001" [production]
16:02 <taavi@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: allocate cloud-private svc ips to wiki replicas - taavi@cumin1001" [production]
15:59 <taavi@cumin1001> START - Cookbook sre.dns.netbox [production]
15:55 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-worker1168.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:51 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: insetup::wmcs [production]
15:47 <fabfur> repooled cp1088 [production]
15:44 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: insetup::wmcs [production]
15:43 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: wmcs::cloudgw [production]
15:37 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: wmcs::cloudgw [production]
15:34 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: wmcs::cloudlb [production]
15:27 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host an-worker1168.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:26 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1016.eqiad.wmnet with reason: host reimage [production]
15:25 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: wmcs::cloudlb [production]
15:24 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: wmcs::db::wikireplicas::analytics_multiinstance [production]
15:23 <eevans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1016.eqiad.wmnet with reason: host reimage [production]
15:21 <fabfur> depooled cp1113 [production]
15:14 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: wmcs::db::wikireplicas::analytics_multiinstance [production]
15:13 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: wmcs::db::wikireplicas::web_multiinstance [production]
15:12 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host aqs1016.eqiad.wmnet with OS bullseye [production]
15:07 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: wmcs::db::wikireplicas::web_multiinstance [production]
15:06 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: dumps::distribution::server [production]
15:05 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for aqs1015.eqiad.wmnet [production]
15:04 <eevans@cumin1001> START - Cookbook sre.hosts.remove-downtime for aqs1015.eqiad.wmnet [production]
15:04 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1015.eqiad.wmnet with OS bullseye [production]
15:00 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: dumps::distribution::server [production]
14:57 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: cluster::cloud_management [production]
14:53 <hnowlan@deploy2002> helmfile [codfw] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
14:53 <hnowlan@deploy2002> helmfile [codfw] [canary] DONE helmfile.d/services/mw-jobrunner : sync [production]
14:52 <hnowlan@deploy2002> helmfile [codfw] [canary] START helmfile.d/services/mw-jobrunner : sync [production]
14:52 <hnowlan@deploy2002> helmfile [codfw] [main] START helmfile.d/services/mw-jobrunner : sync [production]
14:52 <hnowlan@deploy2002> helmfile [eqiad] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
14:52 <hnowlan@deploy2002> helmfile [eqiad] [canary] DONE helmfile.d/services/mw-jobrunner : sync [production]
14:52 <hnowlan@deploy2002> helmfile [eqiad] [main] START helmfile.d/services/mw-jobrunner : sync [production]
14:52 <hnowlan@deploy2002> helmfile [eqiad] [canary] START helmfile.d/services/mw-jobrunner : sync [production]
14:49 <jbond@cumin1001> START - Cookbook sre.puppet.migrate-role for role: cluster::cloud_management [production]
14:45 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1015.eqiad.wmnet with reason: host reimage [production]
14:45 <Lucas_WMDE> T350224 maintenance script finished (8m46s real time) [production]
14:44 <fabfur> swapped cp1113 <-> cp1088 (T349244) [production]
14:44 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-worker1174.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:43 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-worker1173.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:43 <eevans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1015.eqiad.wmnet with reason: host reimage [production]
14:43 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp1113.eqiad.wmnet [production]
14:43 <fabfur@cumin1001> START - Cookbook sre.hosts.remove-downtime for cp1113.eqiad.wmnet [production]
14:42 <fabfur> swapped cp1112 <-> cp1087 (T349244) [production]
14:41 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-worker1171.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:39 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp1112.eqiad.wmnet [production]
14:39 <fabfur@cumin1001> START - Cookbook sre.hosts.remove-downtime for cp1112.eqiad.wmnet [production]
14:38 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on P{lvs[3008-3009].esams.wmnet} and A:lvs (T351069) [production]
14:36 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-worker1172.mgmt.eqiad.wmnet with reboot policy FORCED [production]