151-200 of 372 results (16ms)
2019-11-18 §
18:44 <andrewbogott> depooling labsdb1011 and killing remaining user queries T238480 [admin]
18:42 <andrewbogott> repooled labsdb1009 and 1010 T238480 [admin]
18:19 <andrewbogott> running maintain-views --all-databases --replace-all —clean on labsdb1010 T238480 [admin]
18:18 <andrewbogott> depooling labsdb1010, killing remaining user queries [admin]
17:46 <andrewbogott> running maintain-views --all-databases --replace-all —clean on labsdb1009 T238480 [admin]
17:38 <andrewbogott> depooling labsdb1009, killing remaining user queries [admin]
16:54 <andrewbogott> running maintain-views --all-databases --replace-all —clean on labsdb1012 T237509 [admin]
2019-11-15 §
20:04 <andrewbogott> repool labdb1011 (T237509) [admin]
19:29 <andrewbogott> running maintain-views --all-databases --replace-all —clean on labsdb1011 [admin]
19:25 <andrewbogott> depooling labsdb1011, killing remaining queries [admin]
19:25 <andrewbogott> repooling labsdb1010 [admin]
18:59 <andrewbogott> running maintain-views --all-databases --replace-all —clean on labsdb1012 [admin]
18:57 <andrewbogott> running maintain-views --all-databases --replace-all —clean on labsdb1010 [admin]
18:54 <andrewbogott> depooling labsdb1010, killing remaining user queries [admin]
18:54 <andrewbogott> depooled labsdb1009, ran maintain-views —clean —all-databases —replace-all, repooled [admin]
2019-11-11 §
13:10 <arturo> cloudweb2001-dev: disable puppet and redirect stderr in the loadExitNodes.php cron script to prevent cronspam while we investigate the cause of the issue (T237971) [admin]
2019-11-05 §
11:59 <arturo> icinga downtime for 1h cloudcontrol1004, cloudnet1003, cloudvirt1017/1020/1022 for PDU operations in the rack T227542 [admin]
2019-11-04 §
21:55 <andrewbogott> deleting a ton of wikitech hiera pages that were either no-ops or refer to nonexistent VMs or prefixes [admin]
2019-10-31 §
11:01 <arturo> icinga-downtimed cloudvirt1030 and cloudservices1003 for 1h due to PDU upgrade operations T227543 [admin]
2019-10-30 §
22:43 <jeh> reboot cloud-bootstrapvz-stretch to resolve bad bootstrapvz build [admin]
2019-10-29 §
10:52 <arturo> icinga downtime cloudvirt1001/1002/1024/1018/1012/1009/1015/1008 for 1h T227538 [admin]
2019-10-25 §
10:45 <arturo> icinga downtime toolschecker for 1 to upgrade clouddb1002 mariadb (toolsdb secondary) (T236384 , T236420) [admin]
2019-10-24 §
12:30 <arturo> starting cloudvirt1019, PDU operations ended (T227540) [admin]
11:58 <arturo> icinga downtime for 2h (T227540) cloudvirt1019 [admin]
11:15 <arturo> poweroff cloudvirt1019 during the PDU operations (T227540) [admin]
11:10 <arturo> icinga downtime for 2h (T227540) toolschecker [admin]
10:58 <arturo> icinga downtime for 1h (T227540) cloudvirt100[3-7], cloudvirt1019, cloudvirt1016, cloudvirt1021, cloudvirt1013, cloudnet1004 [admin]
2019-10-23 §
09:23 <arturo> cloudvirt1026 reboot ended OK [admin]
09:12 <arturo> rebooting cloudvirt1026 for kernel upgrade [admin]
09:09 <arturo> cloudvirt1025 reboot ended OK [admin]
09:00 <arturo> rebooting cloudvirt1025 for kernel upgrade [admin]
08:51 <arturo> icinga downtime cloudvirt1025/1026 for reboots [admin]
2019-10-18 §
16:01 <arturo> created the `eqiad1.wikimedia.cloud` DNS zone (T235846) [admin]
14:27 <andrewbogott> deleted a bunch of leaked VMS from earlier today from the admin-monitoring project. Fullstack leaks due to an api outage, maybe? [admin]
10:44 <arturo> double max_message_size from 40KB to 80KB in the cloud-admin mailing list. A simple email with a couple of quotes can go over the 40KB limit. [admin]
2019-10-16 §
21:59 <jeh> resync wiki replica tool and user accounts T235697 [admin]
09:40 <arturo> reboot of cloudvirt1030 went fine [admin]
09:28 <arturo> reboot of cloudvirt1029 went fine [admin]
09:28 <arturo> rebooting cloudvirt1030 for kernel updates [admin]
09:12 <arturo> rebooting cloudvirt1029 for kernel updates [admin]
09:11 <arturo> reboot of cloudvirt1028 went fine [admin]
09:00 <arturo> rebooting cloudvirt1028 for kernel updates [admin]
08:56 <arturo> icinga downtime cloudvirt[1028-1030].eqiad.wmnet for 1h for reboots [admin]
2019-10-15 §
13:30 <jeh> creating indexes and views for banwiki T234770 [admin]
2019-10-10 §
18:55 <bd808> Created indexes and views for nqowiki (T230543) [admin]
11:59 <arturo> network switch hardware is down affecting cloudvirt1025/1026 (T227536) VMs are supposed to be online but unreachable [admin]
2019-10-09 §
10:44 <arturo> cloudvirt1013 rebooted well [admin]
10:32 <arturo> cloudvirt1013 is rebooting [admin]
10:32 <arturo> cloudvirt1012 rebooted just fine (very slow, 35 VMs) [admin]
10:20 <arturo> cloudvirt1012 is rebooting [admin]