1501-1550 of 10000 results (93ms)
2024-05-05 §
08:42 <taavi> taavi@gerrit1003 ~ $ sudo systemctl restart apache2 [production]
2024-05-04 §
13:41 <jayme> doubled the number of eventgate-main replicas in eqiad to 16 [production]
07:39 <taavi@cumin1002> END (PASS) - Cookbook sre.wikireplicas.update-views (exit_code=0) [production]
07:33 <taavi@cumin1002> START - Cookbook sre.wikireplicas.update-views [production]
03:07 <denisse> Restarting `status curator_actions_cluster_wide.service` to log with DEBUGG level on logstash2026 - T364190 [production]
03:06 <denisse> Enable log level DEBUG for curator on logstash2026 - T364190 [production]
01:33 <bblack@cumin1002> conftool action : set/weight=100; selector: name=dns7.* [production]
01:24 <bblack> lvs7001 - restart pybal [production]
01:23 <bblack> lvs7003 - restart pybal [production]
2024-05-03 §
21:38 <ryankemper@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6 days, 0:00:00 on wdqs2023.codfw.wmnet with reason: T362920 [production]
21:38 <ryankemper@cumin2002> START - Cookbook sre.hosts.downtime for 6 days, 0:00:00 on wdqs2023.codfw.wmnet with reason: T362920 [production]
21:27 <ryankemper> T362920 [wdqs] Depooled `wdqs2023` in preparation to switch it to a graph split host [production]
19:02 <sukhe> cleaning up stale confd template files for magru related reimaging [production]
18:44 <brett@cumin2002> conftool action : set/pooled=yes; selector: name=ncredir7002.magru.wmnet,service=nginx [production]
18:43 <brett@cumin2002> conftool action : set/pooled=yes; selector: name=ncredir7001.magru.wmnet,service=nginx [production]
18:38 <brett@cumin2002> conftool action : set/pooled=no; selector: name=ncredir7001.magru.wmnet,service=nginx [production]
18:38 <brett@cumin2002> conftool action : set/pooled=no; selector: name=ncredir7002.magru.wmnet,service=nginx [production]
18:29 <brett@cumin2002> conftool action : set/pooled=yes; selector: name=ncredir7002.magru.wmnet,service=nginx [production]
18:29 <brett@cumin2002> conftool action : set/weight=1; selector: name=ncredir7002.magru.wmnet,service=nginx [production]
18:29 <brett@cumin2002> conftool action : set/pooled=yes; selector: name=ncredir7001.magru.wmnet,service=nginx [production]
18:28 <brett@cumin2002> conftool action : set/weight=1; selector: name=ncredir7001.magru.wmnet,service=nginx [production]
17:45 <dcausse> repooling wdqs1012 [production]
17:27 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2200.codfw.wmnet with reason: Maintenance [production]
17:27 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2200.codfw.wmnet with reason: Maintenance [production]
17:14 <brett@cumin2002> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host ncredir7002.magru.wmnet [production]
17:14 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir7002.magru.wmnet with OS bookworm [production]
17:13 <denisse> Run `sudo mdadm --add /dev/md1 /dev/sdg` on `centrallog1002` - T363660 [production]
17:01 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2198.codfw.wmnet with reason: Maintenance [production]
17:00 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2198.codfw.wmnet with reason: Maintenance [production]
17:00 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2195 (T361627)', diff saved to https://phabricator.wikimedia.org/P61862 and previous config saved to /var/cache/conftool/dbconfig/20240503-170054-marostegui.json [production]
16:47 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir7002.magru.wmnet with reason: host reimage [production]
16:45 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2195', diff saved to https://phabricator.wikimedia.org/P61860 and previous config saved to /var/cache/conftool/dbconfig/20240503-164546-marostegui.json [production]
16:44 <brett@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir7002.magru.wmnet with reason: host reimage [production]
16:30 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2195', diff saved to https://phabricator.wikimedia.org/P61859 and previous config saved to /var/cache/conftool/dbconfig/20240503-163039-marostegui.json [production]
16:18 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host ncredir7002.magru.wmnet with OS bookworm [production]
16:15 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2195 (T361627)', diff saved to https://phabricator.wikimedia.org/P61858 and previous config saved to /var/cache/conftool/dbconfig/20240503-161531-marostegui.json [production]
15:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2195 (T361627)', diff saved to https://phabricator.wikimedia.org/P61857 and previous config saved to /var/cache/conftool/dbconfig/20240503-155432-marostegui.json [production]
15:54 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2195.codfw.wmnet with reason: Maintenance [production]
15:54 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2195.codfw.wmnet with reason: Maintenance [production]
15:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2181 (T361627)', diff saved to https://phabricator.wikimedia.org/P61856 and previous config saved to /var/cache/conftool/dbconfig/20240503-155409-marostegui.json [production]
15:42 <brett@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ncredir7002.magru.wmnet - brett@cumin2002" [production]
15:41 <brett@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ncredir7002.magru.wmnet - brett@cumin2002" [production]
15:40 <brett@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ncredir7002.magru.wmnet on all recursors [production]
15:40 <brett@cumin2002> START - Cookbook sre.dns.wipe-cache ncredir7002.magru.wmnet on all recursors [production]
15:40 <brett@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:40 <brett@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ncredir7002.magru.wmnet - brett@cumin2002" [production]
15:39 <brett@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ncredir7002.magru.wmnet - brett@cumin2002" [production]
15:39 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P61855 and previous config saved to /var/cache/conftool/dbconfig/20240503-153901-marostegui.json [production]
15:34 <brett@cumin2002> START - Cookbook sre.dns.netbox [production]
15:34 <brett@cumin2002> START - Cookbook sre.ganeti.makevm for new host ncredir7002.magru.wmnet [production]