5651-5700 of 10000 results (110ms)
2023-10-02 §
07:32 <taavi@cumin1001> START - Cookbook sre.dns.netbox [production]
07:31 <taavi@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
07:31 <taavi@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: assign new IPs to cloudcontrol1006 - taavi@cumin1001" [production]
07:30 <taavi@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: assign new IPs to cloudcontrol1006 - taavi@cumin1001" [production]
07:28 <taavi@cumin1001> START - Cookbook sre.dns.netbox [production]
05:11 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
05:10 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
2023-10-01 §
01:38 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2172 (T343198)', diff saved to https://phabricator.wikimedia.org/P52799 and previous config saved to /var/cache/conftool/dbconfig/20231001-013851-arnaudb.json [production]
01:23 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P52798 and previous config saved to /var/cache/conftool/dbconfig/20231001-012344-arnaudb.json [production]
01:08 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2172', diff saved to https://phabricator.wikimedia.org/P52797 and previous config saved to /var/cache/conftool/dbconfig/20231001-010838-arnaudb.json [production]
00:53 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2172 (T343198)', diff saved to https://phabricator.wikimedia.org/P52796 and previous config saved to /var/cache/conftool/dbconfig/20231001-005332-arnaudb.json [production]
2023-09-30 §
19:44 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depooling db2172 (T343198)', diff saved to https://phabricator.wikimedia.org/P52795 and previous config saved to /var/cache/conftool/dbconfig/20230930-194448-arnaudb.json [production]
19:44 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2172.codfw.wmnet with reason: Maintenance [production]
19:44 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2172.codfw.wmnet with reason: Maintenance [production]
19:44 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2155 (T343198)', diff saved to https://phabricator.wikimedia.org/P52794 and previous config saved to /var/cache/conftool/dbconfig/20230930-194427-arnaudb.json [production]
19:29 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P52793 and previous config saved to /var/cache/conftool/dbconfig/20230930-192920-arnaudb.json [production]
19:14 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P52792 and previous config saved to /var/cache/conftool/dbconfig/20230930-191414-arnaudb.json [production]
18:59 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2155 (T343198)', diff saved to https://phabricator.wikimedia.org/P52791 and previous config saved to /var/cache/conftool/dbconfig/20230930-185908-arnaudb.json [production]
14:20 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depooling db2155 (T343198)', diff saved to https://phabricator.wikimedia.org/P52790 and previous config saved to /var/cache/conftool/dbconfig/20230930-142054-arnaudb.json [production]
14:20 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
14:20 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
14:20 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2155.codfw.wmnet with reason: Maintenance [production]
14:20 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2155.codfw.wmnet with reason: Maintenance [production]
14:20 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2147 (T343198)', diff saved to https://phabricator.wikimedia.org/P52789 and previous config saved to /var/cache/conftool/dbconfig/20230930-142017-arnaudb.json [production]
14:05 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P52788 and previous config saved to /var/cache/conftool/dbconfig/20230930-140510-arnaudb.json [production]
13:50 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P52787 and previous config saved to /var/cache/conftool/dbconfig/20230930-135004-arnaudb.json [production]
13:34 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2147 (T343198)', diff saved to https://phabricator.wikimedia.org/P52786 and previous config saved to /var/cache/conftool/dbconfig/20230930-133458-arnaudb.json [production]
09:46 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on ldap-rw[1001,2001].wikimedia.org with reason: setup in progress [production]
09:46 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on ldap-rw[1001,2001].wikimedia.org with reason: setup in progress [production]
08:47 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depooling db2147 (T343198)', diff saved to https://phabricator.wikimedia.org/P52785 and previous config saved to /var/cache/conftool/dbconfig/20230930-084720-arnaudb.json [production]
08:47 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2147.codfw.wmnet with reason: Maintenance [production]
08:47 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2147.codfw.wmnet with reason: Maintenance [production]
08:46 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2140 (T343198)', diff saved to https://phabricator.wikimedia.org/P52784 and previous config saved to /var/cache/conftool/dbconfig/20230930-084658-arnaudb.json [production]
08:31 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2140', diff saved to https://phabricator.wikimedia.org/P52783 and previous config saved to /var/cache/conftool/dbconfig/20230930-083152-arnaudb.json [production]
08:16 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2140', diff saved to https://phabricator.wikimedia.org/P52782 and previous config saved to /var/cache/conftool/dbconfig/20230930-081645-arnaudb.json [production]
08:01 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2140 (T343198)', diff saved to https://phabricator.wikimedia.org/P52781 and previous config saved to /var/cache/conftool/dbconfig/20230930-080139-arnaudb.json [production]
02:56 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depooling db2140 (T343198)', diff saved to https://phabricator.wikimedia.org/P52780 and previous config saved to /var/cache/conftool/dbconfig/20230930-025624-arnaudb.json [production]
02:56 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2140.codfw.wmnet with reason: Maintenance [production]
02:56 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2140.codfw.wmnet with reason: Maintenance [production]
2023-09-29 §
23:41 <fabfur@cumin1001> END (PASS) - Cookbook sre.cdn.roll-restart-varnish (exit_code=0) rolling restart of Varnish on 8 hosts matching query A:cp-text_esams [production]
23:40 <fabfur@cumin1001> END (PASS) - Cookbook sre.cdn.roll-restart-varnish (exit_code=0) rolling restart of Varnish on 8 hosts matching query A:cp-upload_esams [production]
22:44 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
22:44 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
22:44 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314 (T343198)', diff saved to https://phabricator.wikimedia.org/P52779 and previous config saved to /var/cache/conftool/dbconfig/20230929-224409-arnaudb.json [production]
22:29 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314', diff saved to https://phabricator.wikimedia.org/P52778 and previous config saved to /var/cache/conftool/dbconfig/20230929-222902-arnaudb.json [production]
22:13 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314', diff saved to https://phabricator.wikimedia.org/P52777 and previous config saved to /var/cache/conftool/dbconfig/20230929-221356-arnaudb.json [production]
21:58 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314 (T343198)', diff saved to https://phabricator.wikimedia.org/P52776 and previous config saved to /var/cache/conftool/dbconfig/20230929-215849-arnaudb.json [production]
21:00 <fabfur@cumin1001> START - Cookbook sre.cdn.roll-restart-varnish rolling restart of Varnish on 8 hosts matching query A:cp-text_esams [production]
20:59 <fabfur@cumin1001> START - Cookbook sre.cdn.roll-restart-varnish rolling restart of Varnish on 8 hosts matching query A:cp-upload_esams [production]
20:35 <fabfur@cumin1001> END (PASS) - Cookbook sre.cdn.roll-restart-varnish (exit_code=0) rolling restart of Varnish on 8 hosts matching query A:cp-text_drmrs [production]