101-150 of 10000 results (19ms)
2025-01-09 §
16:54 <elukey@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:54 <elukey@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker2022 - elukey@cumin1002" [production]
16:54 <elukey@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker2022 - elukey@cumin1002" [production]
16:50 <elukey@cumin1002> START - Cookbook sre.dns.netbox [production]
16:50 <elukey@cumin1002> START - Cookbook sre.hosts.move-vlan for host wikikube-worker2022 [production]
11:29 <elukey@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wikikube-worker2022.codfw.wmnet with OS bookworm [production]
11:24 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker2022 [production]
11:24 <elukey@cumin1002> START - Cookbook sre.hosts.move-vlan for host wikikube-worker2022 [production]
11:23 <elukey@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker2022.codfw.wmnet with OS bookworm [production]
2025-01-08 §
14:53 <elukey@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host wikikube-worker2022.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
14:40 <elukey> elukey@puppetserver1001:~$ sudo puppetserver ca clean --certname kubernetes1061.eqiad.wmnet [production]
14:33 <elukey@cumin1002> START - Cookbook sre.hosts.provision for host wikikube-worker2022.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
13:37 <elukey> elukey@puppetserver1001:~$ sudo puppetserver ca clean --certname kubernetes1021.eqiad.wmnet [production]
08:36 <elukey> kill hanging processes on stat1011 to allow puppet to properly clean up absented users [production]
08:23 <elukey> destroy puppet cert for cloudelastic1011.eqiad.wmnet on puppetmaster1001 (cruft from old/wrong reimages) [production]
2025-01-07 §
16:06 <elukey> reloaded postgres config on puppetdb1003 to pick up new wal size settings [production]
14:34 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
14:33 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
14:02 <elukey> re-enable puppet fleetwide after maintenance [production]
13:57 <elukey> start postgres and puppetdb on puppetdb2003 [production]
13:52 <elukey@cumin1002> END (FAIL) - Cookbook sre.postgresql.postgres-init (exit_code=99) [production]
13:41 <elukey@cumin1002> START - Cookbook sre.postgresql.postgres-init [production]
13:39 <elukey> stop puppetdb and postgres on puppetdb2003 - T383114 [production]
13:35 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on puppetdb2003.codfw.wmnet with reason: Resync postgres [production]
13:35 <elukey@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on puppetdb2003.codfw.wmnet with reason: Resync postgres [production]
13:32 <elukey> disable puppet fleetwide to allow maintenance on puppetdb2003 [production]
2024-12-19 §
10:42 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1043.eqiad.wmnet with OS bookworm [production]
10:42 <elukey@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - elukey@cumin1002" [production]
10:36 <elukey@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - elukey@cumin1002" [production]
10:18 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1043.eqiad.wmnet with reason: host reimage [production]
10:14 <elukey@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on es1043.eqiad.wmnet with reason: host reimage [production]
09:55 <elukey@cumin1002> START - Cookbook sre.hosts.reimage for host es1043.eqiad.wmnet with OS bookworm [production]
09:53 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host es1043.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
09:50 <elukey@cumin1002> START - Cookbook sre.hosts.provision for host es1043.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
2024-12-18 §
09:28 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
09:18 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
2024-12-17 §
14:34 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
14:34 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
14:02 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
14:02 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
2024-12-16 §
15:21 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
15:21 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
15:02 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
14:52 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
14:51 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
14:41 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
14:20 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
14:10 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]
14:01 <elukey@deploy2002> helmfile [staging] DONE helmfile.d/services/kartotherian: sync [production]
13:59 <elukey@deploy2002> helmfile [staging] START helmfile.d/services/kartotherian: sync [production]