551-600 of 6182 results (20ms)
2024-02-06 §
02:09 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.openstack.restart_openstack (exit_code=97) [admin]
02:09 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
2024-02-05 §
21:29 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
21:29 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
20:45 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
20:45 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
20:03 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
20:03 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
20:02 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
20:01 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
18:55 <andrewbogott> rebuilt bookworm base image in eqiad1 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/992677 [admin]
2024-02-02 §
13:54 <arturo> [codfw1dev] cleanup /etc/network/interfaces on cloudlb2003-dev from puppet leftovers [admin]
2024-02-01 §
10:54 <taavi> invite aborrero to /repos/cloud, /toolforge-repos, /cloudvps-repos on gitlab [admin]
2024-01-29 §
12:54 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
12:54 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
2024-01-26 §
09:01 <taavi> joining cloudrabbit1001/2 to the cluster on 1003 T345610 [admin]
2024-01-25 §
16:48 <andrewbogott> taavi just moved all rabbitmq traffic to cloudrabbit1003 as part of T345610 [admin]
16:45 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
16:40 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
13:27 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster [admin]
13:27 <wmbot~taavi@runko> Added a new k8s worker-nfs tools-k8s-worker-nfs-2.tools.eqiad1.wikimedia.cloud to the cluster [admin]
13:15 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster [admin]
13:15 <wmbot~taavi@runko> Added a new k8s worker-nfs tools-k8s-worker-nfs-1.tools.eqiad1.wikimedia.cloud to the cluster [admin]
12:48 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster [admin]
12:48 <wmbot~taavi@runko> Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-1.toolsbeta.eqiad1.wikimedia.cloud to the cluster [admin]
2024-01-24 §
11:37 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the toolsbeta cluster [admin]
11:37 <taavi@cloudcumin1001> Added a new k8s worker toolsbeta-test-k8s-worker-10.toolsbeta.eqiad1.wikimedia.cloud to the cluster [admin]
2024-01-22 §
16:03 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
15:56 <andrewbogott> restarting openstack sevices on eqiad1 to clean up from the mariadb restarts [admin]
15:56 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
15:07 <andrewbogott> merging https://gerrit.wikimedia.org/r/c/operations/puppet/+/992192 and resetting galera cluster in eqiad1 [admin]
2024-01-21 §
05:19 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
05:12 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
2024-01-19 §
23:36 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
23:33 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
2024-01-18 §
20:10 <taavi> mysql:labsdbaccounts@m5-master.eqiad.wmnet [labsdbaccounts]> update account_host set status = 'absent' where id = 137613; [admin]
12:38 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
12:38 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-101.tools.eqiad1.wikimedia.cloud to the cluster [admin]
2024-01-17 §
15:17 <andrewbogott> "systemctl restart mariadb@s4.service mariadb@s6.service" on clouddb1015. System is in danger of oom and there are no obvious long queries running [admin]
2024-01-16 §
14:25 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) [admin]
14:25 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
14:24 <wmbot~taavi@runko> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) for cloudvirt1060.eqiad.wmnet [admin]
14:23 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot for cloudvirt1060.eqiad.wmnet [admin]
14:23 <wmbot~taavi@runko> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) for cloudvirt1060.eqiad.wmnet [admin]
14:23 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot for cloudvirt1060.eqiad.wmnet [admin]
13:55 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) [admin]
13:55 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
13:54 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) [admin]
13:54 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
13:53 <wmbot~taavi@runko> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=99) [admin]