26 results (8ms)
2024-04-23 §
11:47 <cgoubert@cumin1002> conftool action : set/weight=10:pooled=yes; selector: name=(mw1414.eqiad.wmnet|mw1415.eqiad.wmnet|mw1416.eqiad.wmnet|mw1448.eqiad.wmnet|mw1449.eqiad.wmnet),cluster=kubernetes,service=kubesvc [production]
11:47 <claime> Pooling and uncordoning mw1414.eqiad.wmnet,mw1415.eqiad.wmnet,mw1416.eqiad.wmnet,mw1448.eqiad.wmnet,mw1449.eqiad.wmnet - T351074 [production]
11:38 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1415.eqiad.wmnet with OS bullseye [production]
11:21 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1415.eqiad.wmnet with reason: host reimage [production]
11:06 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1415.eqiad.wmnet with reason: host reimage [production]
10:53 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host mw1415.eqiad.wmnet with OS bullseye [production]
10:45 <claime> Depooling mw1414.eqiad.wmnet,mw1415.eqiad.wmnet,mw1416.eqiad.wmnet,mw1448.eqiad.wmnet,mw1449.eqiad.wmnet for reimage to kubernetes - T351074 [production]
2023-02-28 §
18:23 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1415.eqiad.wmnet [production]
18:03 <ladsgroup@cumin1001> conftool action : set/pooled=no; selector: name=mw1415.eqiad.wmnet [production]
2022-06-08 §
21:41 <mutante> repooled mw1415 after restarting apache and php-fpm, seeing all Icinga alerts recover etc T307755 T310225 [production]
21:40 <dzahn@cumin2002> conftool action : set/pooled=yes; selector: dc=eqiad,name=mw1415.eqiad.wmnet [production]
21:17 <dzahn@cumin2002> conftool action : set/pooled=no; selector: dc=eqiad,name=mw1415.eqiad.wmnet [production]
21:13 <mutante> mw1415 - scap pull, restart apache, /usr/local/sbin/restart-php7.2-fpm (INFO: The server is depooled from all services. Restarting the service directly) [production]
20:42 <Krinkle> krinkle@mw1415: Run `scap pull` manually ref T310225 [production]
2022-05-09 §
08:03 <ladsgroup@cumin1001> conftool action : set/pooled=inactive; selector: name=mw1415.eqiad.wmnet [production]
08:02 <ladsgroup@cumin1001> conftool action : set/pooled=no; selector: name=mw1415.eqiad.wmnet [production]
2022-05-06 §
01:51 <dzahn@cumin2002> conftool action : set/pooled=no; selector: dc=eqiad,name=mw1415.eqiad.wmnet [production]
01:50 <dzahn@cumin2002> conftool action : set/pooled=no; selector: dc=codfw,name=mw1415.eqiad.wmnet [production]
2022-03-31 §
18:43 <duesen> removing /var/run/php/use-config-schema from canaries mw1415, mw1438, and mw1448 to disable config schema loading (T304460) [production]
16:33 <duesen> creating /var/run/php/use-config-schema on canaries mw1415, mw1438, and mw1448 to enable config schema loading (T304460) [production]
14:22 <duesen> (late) about 5 hours ago, I removed /var/run/php/use-config-schema from mw1415 to disable config schema loading (T304460) [production]
09:16 <duesen> created /var/run/php/use-config-schema on canary mw1415 to enable config schema loading (T304460) [production]
2022-01-17 §
15:54 <mutante> mw1414,mw1415,mw1416,mw1417,mw1418,mw1447,mw1448,mw1449,mw1450,mw1437,mw1438 (all canaries eqiad) - apt-get remove --purge fonts*; apt-get remove --purge xfonts* (T294378) [production]
2021-07-19 §
13:01 <dzahn@cumin1001> conftool action : set/weight=1; selector: name=mw1415.eqiad.wmnet,service=canary [production]
2021-06-23 §
23:11 <robh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1415.eqiad.wmnet with reason: REIMAGE [production]
23:09 <robh@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1415.eqiad.wmnet with reason: REIMAGE [production]