8251-8300 of 9339 results (45ms)
2021-06-09 §
13:21 <wm-bot> Rebooting the nodes cloudcephmon2002-dev,cloudcephmon2003-dev,cloudcephmon2004-dev (T281248) - cookbook ran by dcaro@vulcanus [admin]
13:01 <wm-bot> Rebooting node cloudcephmon2002-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
13:01 <wm-bot> Rebooting the nodes cloudcephmon2002-dev,cloudcephmon2003-dev,cloudcephmon2004-dev (T281248) - cookbook ran by dcaro@vulcanus [admin]
12:53 <wm-bot> Rebooting node cloudcephmon2002-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
12:53 <wm-bot> Rebooting the nodes cloudcephmon2002-dev,cloudcephmon2003-dev,cloudcephmon2004-dev (T281248) - cookbook ran by dcaro@vulcanus [admin]
2021-06-08 §
23:19 <bd808> Downtimed cloudmetrics1002 in icinga until 2021-06-30 23:59:01 (T281881) [admin]
21:08 <bstorm> downtiming grafana-labs for maintenance [admin]
16:28 <wm-bot> Finished rebooting the nodes ['cloudcephosd2001-dev', 'cloudcephosd2002-dev', 'cloudcephosd2003-dev'] (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:27 <wm-bot> Finished rebooting node cloudcephosd2003-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:24 <wm-bot> Rebooting node cloudcephosd2003-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:24 <wm-bot> Finished rebooting node cloudcephosd2002-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:22 <wm-bot> Rebooting node cloudcephosd2002-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:21 <wm-bot> Finished rebooting node cloudcephosd2001-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:18 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:18 <wm-bot> Rebooting the nodes ['cloudcephosd2001-dev', 'cloudcephosd2002-dev', 'cloudcephosd2003-dev'] (T281248) - cookbook ran by dcaro@vulcanus [admin]
16:17 <wm-bot> Rebooting the nodes ['cloudcephosd2001-dev', 'cloudcephosd2002-dev', 'cloudcephosd2003-dev'] (T281248) - cookbook ran by dcaro@vulcanus [admin]
15:03 <wm-bot> Finished rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
14:59 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
14:59 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
14:57 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
14:57 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
14:29 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
14:23 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
14:18 <wm-bot> Rebooting node cloudcephosd2001-dev.codfw.wmnet - cookbook ran by dcaro@vulcanus [admin]
2021-06-07 §
14:26 <andrewbogott> moving cloudvirt1040 from 'maintenance' aggregate to 'ceph' aggregate T281399 [admin]
2021-06-01 §
13:12 <dcaro> Changed the ceph osd_memory_target on eqiad pool to 6Gi (we were reaching the limit, swapping at some points) [admin]
09:57 <arturo> fix PTR record for 185.15.56.1 (T284025) [admin]
09:56 <arturo> fix PTR record for 185.15.56.1 (T248025) [admin]
2021-05-27 §
14:58 <wm-bot> Testing - cookbook ran by dcaro@vulcanus [admin]
2021-05-26 §
19:10 <andrewbogott> reimaging cloudvirt1018 to support local VM storage [admin]
18:07 <andrewbogott> draining cloudvirt1018, converting it to a local-storage host like cloudvirt1019 and 1020 -- T283296 [admin]
14:36 <dcaro> Enabled syslog logging for osd.55 on eqiad ceph cluster for testing (T281247) [admin]
14:36 <dcaro> Enabled syslog logging on codfw ceph cluster (mon/osd/mgr) (T281247) [admin]
11:26 <arturo> [codfw1dev] purge old kernel packages in cloudvirt200[12]-dev [admin]
11:03 <arturo> created public flavor `g3.cores16.ram36.disk20` (even though it was requested as private in T283293, but may be useful for others) [admin]
2021-05-25 §
16:14 <bd808> Closed #wikimedia-cloud-admin on f***node [admin]
16:11 <bd808> Closed #wikimedia-cloud-feed on f***node [admin]
15:19 <dcaro> rebooted cloudvirt1020, starting VMs (T275893) [admin]
15:13 <dcaro> rebooting cloudvirt1020 (T275893) [admin]
14:42 <dcaro> taking cloudvirt1020 out for maintenance (openstack wise) so no new VMs are scheduled on it (T275893) [admin]
2021-05-24 §
22:32 <andrewbogott> changing the default ttl for eqiad1.wikimedia.cloud. from 3600 to 60; this should help us avoid madness when re-using hostnames. [admin]
11:20 <arturo> created `g3.cores2.ram80.disk40.private` for the wmf-research-tools project, to allow resizing a 40G disk instance [admin]
2021-05-22 §
02:14 <bstorm> downtiming SMART alerts on dumps server labstore1007 for the weekend because it has been flapping T281045 [admin]
2021-05-13 §
21:25 <bstorm> converted the maps and scratch volumes on cloudstore1008 (standby) to drbd T224747 [admin]
15:45 <bstorm> re-running wikireplicas-dns after refactor of config to make sure it doesn't change anything [admin]
2021-05-12 §
14:22 <arturo> [codfw1dev] cleanup old unused agents (bgp, ovs) [admin]
11:37 <arturo> [codfw1dev] replacing cloudnet2003-dev with cloudnet2004-dev (T281381) [admin]
2021-05-11 §
18:00 <andrewbogott> adding 'trove' service project in advance of deploying trove in eqiad1 [admin]
10:22 <arturo> rebooted cloudgw1002 (active) thus causing a failover to cloudgw1001 [admin]
2021-05-09 §
10:53 <arturo> icinga-downtime cloudmetrics1002 for 3 months (T275605) [admin]