301-350 of 5899 results (30ms)
2024-01-19 §
23:36 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
23:33 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
2024-01-18 §
20:10 <taavi> mysql:labsdbaccounts@m5-master.eqiad.wmnet [labsdbaccounts]> update account_host set status = 'absent' where id = 137613; [admin]
12:38 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
12:38 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-101.tools.eqiad1.wikimedia.cloud to the cluster [admin]
2024-01-17 §
15:17 <andrewbogott> "systemctl restart mariadb@s4.service mariadb@s6.service" on clouddb1015. System is in danger of oom and there are no obvious long queries running [admin]
2024-01-16 §
14:25 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) [admin]
14:25 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
14:24 <wmbot~taavi@runko> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) for cloudvirt1060.eqiad.wmnet [admin]
14:23 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot for cloudvirt1060.eqiad.wmnet [admin]
14:23 <wmbot~taavi@runko> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) for cloudvirt1060.eqiad.wmnet [admin]
14:23 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot for cloudvirt1060.eqiad.wmnet [admin]
13:55 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) [admin]
13:55 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
13:54 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) [admin]
13:54 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
13:53 <wmbot~taavi@runko> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=99) [admin]
13:53 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
13:52 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) [admin]
13:52 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance [admin]
11:35 <wmbot~taavi@runko> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) for cloudvirt1060.eqiad.wmnet [admin]
11:34 <wmbot~taavi@runko> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot for cloudvirt1060.eqiad.wmnet [admin]
11:22 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) (T355061) [admin]
11:02 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot (T355061) [admin]
09:41 <taavi> drop dbproxy1018/9 grants from all clouddb hosts T346947 [admin]
09:24 <taavi> move cloudvirt2004-dev from 'failed' to 'active' in netbox - seems like that was for T348531 which is now resolved [admin]
2024-01-15 §
14:37 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) (T355061) [admin]
14:32 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot (T355061) [admin]
14:32 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=99) (T355061) [admin]
14:27 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.safe_reboot (T355061) [admin]
12:41 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
12:37 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
2024-01-11 §
10:51 <dcaro> restarting striker.service on cloudweb1003 as it seems non-responsive [admin]
2024-01-10 §
17:43 <bd808> Blocking Developer accounts connected to invalid/legacy wikimedia.org email addresses (T218239) [admin]
16:48 <wmbot~fran@wmf3169> END (PASS) - Cookbook wmcs.do_log_msg (exit_code=0) (T346631) [admin]
16:48 <wmbot~fran@wmf3169> test message3 from local cookbook (T346631) [admin]
16:48 <wmbot~fran@wmf3169> START - Cookbook wmcs.do_log_msg (T346631) [admin]
2024-01-09 §
17:50 <wmbot~fran@wmf3169> END (PASS) - Cookbook wmcs.do_log_msg (exit_code=0) (T346631) [admin]
17:49 <wmbot~fran@wmf3169> test message2 from local cookbook (T346631) [admin]
17:49 <wmbot~fran@wmf3169> START - Cookbook wmcs.do_log_msg (T346631) [admin]
17:43 <wmbot~fran@wmf3169> %(message)s (T346631) [admin]
17:42 <wmbot~fran@wmf3169> %(message)s (T346631) [admin]
17:42 <wmbot~fran@wmf3169> %(message)s (T346631) [admin]
2024-01-08 §
15:52 <taavi> verify wmcloud.org, wmflabs.org and toolforge.org in gmail postmaster console to figure out how much google likes us (T354112) [admin]
2024-01-07 §
19:34 <andrewbogott> removed cloudvirt1063 from 'ceph' aggregate, added to 'maintenance' aggregate T353408 [admin]
19:34 <andrewbogott> evacuating all VMs from cloudvirt1063. T353408 [admin]
2024-01-02 §
16:18 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=99) (T353408) [admin]
16:18 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T353408) [admin]
10:22 <wm-bot2> fran@wmf3169 END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99) [admin]
10:22 <wm-bot2> fran@wmf3169 START - Cookbook wmcs.openstack.cloudvirt.vm_console [admin]