3801-3850 of 10000 results (96ms)
2024-02-28 ยง
14:25 <daniel@deploy2002> daniel: Backport for [[gerrit:994212|Configure parser cache filters for parsoid-pcache (T346765 T355375)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:24 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1233', diff saved to https://phabricator.wikimedia.org/P58077 and previous config saved to /var/cache/conftool/dbconfig/20240228-142445-arnaudb.json [production]
14:24 <jiji@deploy2002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
14:24 <jiji@deploy2002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
14:23 <daniel@deploy2002> Started scap: Backport for [[gerrit:994212|Configure parser cache filters for parsoid-pcache (T346765 T355375)]] [production]
14:23 <jiji@deploy2002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
14:22 <jiji@deploy2002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
14:22 <jiji@deploy2002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
14:14 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2190 (T352010)', diff saved to https://phabricator.wikimedia.org/P58076 and previous config saved to /var/cache/conftool/dbconfig/20240228-141413-ladsgroup.json [production]
14:14 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2190.codfw.wmnet with reason: Maintenance [production]
14:13 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2190.codfw.wmnet with reason: Maintenance [production]
14:09 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1233 (T357189)', diff saved to https://phabricator.wikimedia.org/P58075 and previous config saved to /var/cache/conftool/dbconfig/20240228-140938-arnaudb.json [production]
14:06 <marostegui@cumin1002> dbctl commit (dc=all): 'db1232 (re)pooling @ 100%: After running optimize', diff saved to https://phabricator.wikimedia.org/P58074 and previous config saved to /var/cache/conftool/dbconfig/20240228-140626-root.json [production]
14:03 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1233 (T357189)', diff saved to https://phabricator.wikimedia.org/P58073 and previous config saved to /var/cache/conftool/dbconfig/20240228-140346-arnaudb.json [production]
14:03 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1233.eqiad.wmnet with reason: Maintenance [production]
14:03 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1233.eqiad.wmnet with reason: Maintenance [production]
14:03 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1229 (T357189)', diff saved to https://phabricator.wikimedia.org/P58072 and previous config saved to /var/cache/conftool/dbconfig/20240228-140323-arnaudb.json [production]
13:53 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:53 <ayounsi@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: sretest2003 - ayounsi@cumin1002" [production]
13:52 <ayounsi@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: sretest2003 - ayounsi@cumin1002" [production]
13:51 <marostegui@cumin1002> dbctl commit (dc=all): 'db1232 (re)pooling @ 75%: After running optimize', diff saved to https://phabricator.wikimedia.org/P58071 and previous config saved to /var/cache/conftool/dbconfig/20240228-135121-root.json [production]
13:49 <ayounsi@cumin1002> START - Cookbook sre.dns.netbox [production]
13:48 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1229', diff saved to https://phabricator.wikimedia.org/P58070 and previous config saved to /var/cache/conftool/dbconfig/20240228-134817-arnaudb.json [production]
13:41 <ayounsi@cumin1002> END (FAIL) - Cookbook sre.hosts.move-vlan (exit_code=99) for host <spicerack.netbox.NetboxServer object at 0x7f3aaebfffa0> [production]
13:41 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) 6.0.0.0.0.1.0.0.2.9.1.0.0.1.0.0.b.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
13:41 <ayounsi@cumin1002> START - Cookbook sre.dns.wipe-cache 6.0.0.0.0.1.0.0.2.9.1.0.0.1.0.0.b.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
13:41 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) 6.10.192.10.in-addr.arpa on all recursors [production]
13:41 <ayounsi@cumin1002> START - Cookbook sre.dns.wipe-cache 6.10.192.10.in-addr.arpa on all recursors [production]
13:41 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) sretest2003.codfw.wmnet on all recursors [production]
13:41 <ayounsi@cumin1002> START - Cookbook sre.dns.wipe-cache sretest2003.codfw.wmnet on all recursors [production]
13:40 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:40 <btullis@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbstore1007.eqiad.wmnet with OS bookworm [production]
13:40 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1168 (T352010)', diff saved to https://phabricator.wikimedia.org/P58069 and previous config saved to /var/cache/conftool/dbconfig/20240228-133959-ladsgroup.json [production]
13:39 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
13:39 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
13:39 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1165 (T352010)', diff saved to https://phabricator.wikimedia.org/P58068 and previous config saved to /var/cache/conftool/dbconfig/20240228-133937-ladsgroup.json [production]
13:39 <ayounsi@cumin1002> START - Cookbook sre.dns.netbox [production]
13:39 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) 5.5.2.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
13:39 <ayounsi@cumin1002> START - Cookbook sre.dns.wipe-cache 5.5.2.0.0.0.0.0.2.9.1.0.0.1.0.0.1.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
13:39 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) 255.0.192.10.in-addr.arpa on all recursors [production]
13:39 <ayounsi@cumin1002> START - Cookbook sre.dns.wipe-cache 255.0.192.10.in-addr.arpa on all recursors [production]
13:39 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) sretest2003.codfw.wmnet on all recursors [production]
13:39 <ayounsi@cumin1002> START - Cookbook sre.dns.wipe-cache sretest2003.codfw.wmnet on all recursors [production]
13:39 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:39 <ayounsi@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host sretest2003 - ayounsi@cumin1002" [production]
13:38 <ayounsi@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host sretest2003 - ayounsi@cumin1002" [production]
13:36 <marostegui@cumin1002> dbctl commit (dc=all): 'db1232 (re)pooling @ 50%: After running optimize', diff saved to https://phabricator.wikimedia.org/P58066 and previous config saved to /var/cache/conftool/dbconfig/20240228-133616-root.json [production]
13:36 <ayounsi@cumin1002> START - Cookbook sre.dns.netbox [production]
13:33 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1229', diff saved to https://phabricator.wikimedia.org/P58065 and previous config saved to /var/cache/conftool/dbconfig/20240228-133311-arnaudb.json [production]
13:33 <ayounsi@cumin1002> START - Cookbook sre.hosts.move-vlan for host <spicerack.netbox.NetboxServer object at 0x7f3aaebfffa0> [production]