1-50 of 164 results (36ms)
2026-06-27 §
20:09 <Krinkle> Reboot codesearch10, appears to be down since ~2h ago [codesearch]
2026-06-24 §
17:57 <mutante> - created second webproxy codesearch-backend and pointed to port 3002 on instance codesearch10 to fix search [codesearch]
2026-06-22 §
21:30 <mutante> - upgraded to Debian trixie [codesearch]
20:57 <mutante> - re-created on trixie! oof:)) should be working - https://codesearch.wmcloud.org/search/ T429828 [codesearch]
19:56 <mutante> creating instance codesearch11 on bookworm - hound services failed to start on trixie - verify if it's distro related - T429828 T429496 [codesearch]
19:29 <mutante> deleted instance codesearch9 by accident :( - creating instance codesearch10 on trixie - T429828 T429496 [codesearch]
18:53 <mutante> deleting proxy codesearch-sourcebot.wmcloud.org - T429496 [codesearch]
18:52 <mutante> shutting down instance sourcebot1 - T429496 [codesearch]
2026-06-09 §
10:25 <taavi> hard reboot codesearch9 [codesearch]
2026-05-29 §
17:32 <mutante> - adding "confd::ensure: absent" to Horizon Hiera at Project Puppet level - T417458 [codesearch]
2026-03-03 §
23:56 <mutante> nevermind - puppet restarts it (debug) (T417458) [codesearch]
23:49 <mutante> codesearch9 systemctl mask confd (T417458) [codesearch]
2026-02-14 §
00:01 <mutante> codesearch9: systemctl restart logrotate with new config without * - T413739 [codesearch]
2026-02-13 §
21:39 <Krinkle> Manually pruning codesearch9:/var/log/account/pacct, ref T417397 T413739 [codesearch]
21:29 <Krinkle> Reboot codesearch9 instance, ref T417397 [codesearch]
2025-12-29 §
21:35 <andrewbogott> rebooting codesearch9 instance; it's unresponsive, even via the console [codesearch]
2025-12-05 §
18:35 <mutante> re-enabled puppet on codesearch9 - service up; has double disk space vs before incident - T411047 T411728 [codesearch]
2025-12-03 §
23:22 <mutante> - shut down instance codesearch9; extending volume "data2" to double its size T411728 T411047 [codesearch]
2025-12-01 §
12:52 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T411062) [codesearch]
12:52 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase (T411062) [codesearch]
2025-11-25 §
20:22 <Krinkle> Remove confusing duplicate Horizon hieradata for codesearch9.codesearch.eqiad1.wikimedia.cloud. Was committed in https://gerrit.wikimedia.org/r/1196670. Ref T335407 [codesearch]
2025-09-10 §
08:55 <volans> force-reboot codesearch9 VM because it's unresponsive to ssh and console and the services are down T404163 [codesearch]
08:53 <volans@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [codesearch]
08:52 <volans@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [codesearch]
08:52 <volans@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.vm_console [codesearch]
08:51 <volans@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.vm_console [codesearch]
2025-08-09 §
08:09 <addshore> rebooted codesearch9.codesearch.eqiad1.wikimedia.cloud T401513 [codesearch]
08:09 <addshore> rebooted [codesearch]
2025-06-05 §
16:50 <mutante> resizing instance sourcebot1 from g4.cores4.ram8.disk20 to g4.cores4.ram16.disk20 [codesearch]
2025-06-04 §
23:21 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T396073) [codesearch]
23:21 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase (T396073) [codesearch]
23:20 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T396073) [codesearch]
23:20 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase (T396073) [codesearch]
2025-05-19 §
21:44 <mutante> soft rebooting instance codesearch9, web UI was down and could not get shell [codesearch]
2025-03-28 §
19:39 <Krinkle> Reboot codesearch9, web service was down and ssh not responding/timing out. Alert fired 41min ago about instance being perceiced as down. [codesearch]
2024-12-11 §
08:12 <fnegri@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T381911) [codesearch]
08:12 <fnegri@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase (T381911) [codesearch]
2024-10-15 §
18:10 <mutante> - re-enabled puppet on codesearch9 [codesearch]
2024-10-10 §
19:35 <mutante> stopping and restarting all hound services [codesearch]
17:24 <mutante> rebooting codesearch9 again [codesearch]
16:08 <mutante> running commands listed in commit message of gerrit:920243 [codesearch]
16:02 <mutante> adding new project member eoghan and granting member role, also granting member role to aokoth [codesearch]
2024-10-09 §
16:52 <mutante> soft rebooting instance codesearch 9 which was unreachable again [codesearch]
2024-10-03 §
17:16 <mutante> - service not reachable - cant ssh to instance codesearch9, attempting to soft reboot VM [codesearch]
2024-06-18 §
14:54 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T367878) [codesearch]
14:54 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.quota_increase (T367878) [codesearch]
2024-06-14 §
16:59 <mutante> createing instance codesearch9 with bookworm to replace codesearch8 (T367479) [codesearch]
2024-04-19 §
21:22 <Krinkle> Attempt to fix puppet failure since 48h `codesearch8 puppet-agent[21081]: (/Stage[main]/Codesearch/Git::Clone[labs/codesearch]/Exec[git_pull_labs/codesearch]/returns) error: insufficient permission for adding an object to repository database .git/objects` [codesearch]
2024-04-03 §
02:46 <Krinkle> Reboot host https://grafana.wmcloud.org/d/0g9N-7pVz/cloud-vps-project-board?orgId=1&var-project=codesearch&var-instance=All&from=now-3h&to=now [codesearch]
2024-03-15 §
03:53 <Krinkle> Deploying fix for T321347 [codesearch]