1951-2000 of 10000 results (108ms)
2024-12-23 §
10:41 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1012.eqiad.wmnet with reason: host reimage [production]
10:41 <jayme@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1025.eqiad.wmnet with reason: host reimage [production]
10:38 <jayme@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1012.eqiad.wmnet with reason: host reimage [production]
10:22 <jayme@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1025.eqiad.wmnet with OS bookworm [production]
10:21 <jayme@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1012.eqiad.wmnet with OS bookworm [production]
10:20 <jayme@cumin1002> START - Cookbook sre.k8s.roll-reimage-nodes rolling reimage on P{wikikube-worker[1025-1027].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) [production]
10:20 <jayme@cumin1002> START - Cookbook sre.k8s.roll-reimage-nodes rolling reimage on P{wikikube-worker[1012-1014].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) [production]
10:04 <fabfur@cumin1002> conftool action : set/pooled=yes; selector: name=cp3066.esams.wmnet [production]
10:03 <fabfur@cumin1002> conftool action : set/pooled=no; selector: name=cp3066.esams.wmnet [production]
09:50 <Emperor> repool ms-fe2010 [production]
09:45 <Emperor> depool ms-fe2010 to attempt swap clearance [production]
09:44 <mvernon@cumin2002> END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling reboot on P{ms-fe2009*} and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) [production]
09:38 <mvernon@cumin2002> START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling reboot on P{ms-fe2009*} and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) [production]
09:38 <moritzm> installing gtk+3.0 security updates [production]
08:39 <akosiaris@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-ctrl1004.eqiad.wmnet with OS bookworm [production]
08:38 <akosiaris@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-ctrl1004.eqiad.wmnet with OS bookworm [production]
08:15 <akosiaris@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-ctrl1004.eqiad.wmnet with OS bookworm [production]
08:02 <akosiaris@cumin1002> END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from wikikube-worker1290 to wikikube-ctrl1004 [production]
08:02 <moritzm> installing libxstream-java security updates [production]
08:02 <akosiaris@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-ctrl1004 [production]
08:02 <akosiaris@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-ctrl1004 [production]
08:02 <akosiaris@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
08:02 <akosiaris@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming wikikube-worker1290 to wikikube-ctrl1004 - akosiaris@cumin1002" [production]
07:57 <akosiaris@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming wikikube-worker1290 to wikikube-ctrl1004 - akosiaris@cumin1002" [production]
07:53 <akosiaris@cumin1002> START - Cookbook sre.dns.netbox [production]
07:53 <akosiaris@cumin1002> START - Cookbook sre.hosts.rename from wikikube-worker1290 to wikikube-ctrl1004 [production]
07:46 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker1290.eqiad.wmnet [production]
07:46 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker1290.eqiad.wmnet [production]
07:03 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10 days, 0:00:00 on dbproxy1029.eqiad.wmnet with reason: maintenance [production]
07:03 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 10 days, 0:00:00 on dbproxy1029.eqiad.wmnet with reason: maintenance [production]
07:03 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10 days, 0:00:00 on dbproxy1028.eqiad.wmnet with reason: maintenance [production]
07:03 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 10 days, 0:00:00 on dbproxy1028.eqiad.wmnet with reason: maintenance [production]
2024-12-22 §
11:43 <mvernon@cumin2002> END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-codfw [production]
11:40 <mvernon@cumin2002> START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-codfw [production]
10:52 <Emperor> restart swift-object on ms-be2082 [production]
10:49 <mvernon@cumin2002> END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-codfw [production]
10:46 <mvernon@cumin2002> START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-codfw [production]
2024-12-21 §
00:00 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd2004-dev.codfw.wmnet with OS bookworm [production]
00:00 <andrew@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - andrew@cumin1002" [production]
2024-12-20 §
22:12 <andrew@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - andrew@cumin1002" [production]
21:52 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd2004-dev.codfw.wmnet with reason: host reimage [production]
21:48 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd2004-dev.codfw.wmnet with reason: host reimage [production]
21:29 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudcephosd2004-dev.codfw.wmnet with OS bookworm [production]
21:24 <andrew@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd2004-dev.codfw.wmnet with OS bookworm [production]
21:13 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudcephosd2004-dev.codfw.wmnet with OS bookworm [production]
18:43 <marostegui@cumin1002> dbctl commit (dc=all): 'db2168 (re)pooling @ 100%: Repooling after upgrade', diff saved to https://phabricator.wikimedia.org/P71741 and previous config saved to /var/cache/conftool/dbconfig/20241220-184307-root.json [production]
18:35 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.roll-reimage-nodes (exit_code=0) rolling reimage on P{wikikube-worker[1023-1024].eqiad.wmnet} and (A:wikikube-master-eqiad or A:wikikube-worker-eqiad) [production]
18:35 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1024.eqiad.wmnet with OS bookworm [production]
18:28 <marostegui@cumin1002> dbctl commit (dc=all): 'db2168 (re)pooling @ 75%: Repooling after upgrade', diff saved to https://phabricator.wikimedia.org/P71739 and previous config saved to /var/cache/conftool/dbconfig/20241220-182801-root.json [production]
18:15 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1024.eqiad.wmnet with reason: host reimage [production]