401-450 of 5537 results (15ms)
2023-11-01 §
23:13 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (348643) [admin]
21:29 <taavi> re-enable puppet on cloudcontrol2006-dev.codfw.wmnet which has fallen off of puppetdb [admin]
21:15 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
21:15 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
21:15 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
21:14 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
20:26 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
20:11 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (348643) [admin]
19:55 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (348643) [admin]
15:10 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
14:55 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
14:52 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (348643) [admin]
14:52 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
14:50 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) (348643) [admin]
14:49 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node (348643) [admin]
14:49 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) (348643) [admin]
14:48 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node (348643) [admin]
14:47 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) (348643) [admin]
14:47 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node (348643) [admin]
09:04 <taavi> reset local cookbook changes on cloudcumin1001 which were causing issues with puppet runs [admin]
09:02 <taavi> restart nova-fullstack which had had some issues after yesterday's cloudcontrol1007 reimage [admin]
02:17 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
00:58 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
00:52 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
2023-10-31 §
23:46 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
18:41 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
14:12 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) (348643) [admin]
14:11 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node (348643) [admin]
14:10 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
10:16 <dhinus> upgrading mariadb-server in cloudcontrol1005 (T345811) [admin]
10:04 <dhinus> upgrading mariadb-server in cloudcontrol1006 (T345811) [admin]
09:51 <dhinus> upgrading mariadb-server in cloudcontrol1007, second attempt (T345811) [admin]
06:16 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (348643) [admin]
05:20 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
02:27 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) (348643) [admin]
02:27 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node (348643) [admin]
02:27 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.ceph.osd.drain_node (exit_code=97) (348643) [admin]
02:26 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
01:59 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
01:59 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
01:07 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
01:07 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
01:07 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (348643) [admin]
2023-10-30 §
17:40 <andrewbogott> rebooting tools-db-1.tools.eqiad1.wikimedia.cloud for yet another oom death [admin]
17:09 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
17:04 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
17:02 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]
16:56 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (348643) [admin]
16:55 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (348643) [admin]
16:55 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (348643) [admin]