4501-4550 of 10000 results (88ms)
2023-05-30 ยง
07:21 <kartik@deploy1002> kartik: Backport for [[gerrit:924050|testwiki: Enable Section Translation for 9 Wikipedia (T337290)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
07:19 <kartik@deploy1002> Started scap: Backport for [[gerrit:924050|testwiki: Enable Section Translation for 9 Wikipedia (T337290)]] [production]
07:17 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) testvm2006.codfw.wmnet on all recursors [production]
07:17 <slyngshede@cumin1001> START - Cookbook sre.dns.wipe-cache testvm2006.codfw.wmnet on all recursors [production]
07:17 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
07:17 <slyngshede@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
07:16 <kartik@deploy1002> Finished scap: Backport for [[gerrit:923527|Undeploy Special:Contribute from unsupported skins (T337366)]] (duration: 11m 49s) [production]
07:16 <moritzm> update bookworm installer to rc4 T330495 [production]
07:16 <slyngshede@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'db2110 (re)pooling @ 50%: Repooling after maintenance', diff saved to https://phabricator.wikimedia.org/P48630 and previous config saved to /var/cache/conftool/dbconfig/20230530-071436-root.json [production]
07:10 <slyngshede@cumin1001> START - Cookbook sre.dns.netbox [production]
07:10 <slyngshede@cumin1001> START - Cookbook sre.ganeti.makevm for new host testvm2006.codfw.wmnet [production]
07:10 <slyngshede@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host testvm2006.codfw.wmnet [production]
07:10 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) testvm2006.codfw.wmnet on all recursors [production]
07:10 <slyngshede@cumin1001> START - Cookbook sre.dns.wipe-cache testvm2006.codfw.wmnet on all recursors [production]
07:10 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
07:10 <slyngshede@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
07:09 <slyngshede@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
07:07 <slyngshede@cumin1001> START - Cookbook sre.dns.netbox [production]
07:07 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) testvm2006.codfw.wmnet on all recursors [production]
07:07 <slyngshede@cumin1001> START - Cookbook sre.dns.wipe-cache testvm2006.codfw.wmnet on all recursors [production]
07:07 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
07:07 <slyngshede@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
07:06 <kartik@deploy1002> kartik: Backport for [[gerrit:923527|Undeploy Special:Contribute from unsupported skins (T337366)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
07:06 <slyngshede@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
07:04 <kartik@deploy1002> Started scap: Backport for [[gerrit:923527|Undeploy Special:Contribute from unsupported skins (T337366)]] [production]
07:04 <slyngshede@cumin1001> START - Cookbook sre.dns.netbox [production]
07:03 <slyngshede@cumin1001> START - Cookbook sre.ganeti.makevm for new host testvm2006.codfw.wmnet [production]
07:02 <slyngshede@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=2) for new host testvm2006.codfw.wmnet [production]
07:02 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) testvm2006.codfw.wmnet on all recursors [production]
07:02 <slyngshede@cumin1001> START - Cookbook sre.dns.wipe-cache testvm2006.codfw.wmnet on all recursors [production]
07:02 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
07:02 <slyngshede@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
07:01 <slyngshede@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
06:59 <marostegui@cumin1001> dbctl commit (dc=all): 'db2110 (re)pooling @ 25%: Repooling after maintenance', diff saved to https://phabricator.wikimedia.org/P48629 and previous config saved to /var/cache/conftool/dbconfig/20230530-065932-root.json [production]
06:58 <slyngshede@cumin1001> START - Cookbook sre.dns.netbox [production]
06:58 <slyngshede@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
06:57 <slyngshede@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
06:51 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) testvm2006.codfw.wmnet on all recursors [production]
06:51 <slyngshede@cumin1001> START - Cookbook sre.dns.wipe-cache testvm2006.codfw.wmnet on all recursors [production]
06:51 <slyngshede@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
06:51 <slyngshede@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
06:50 <slyngshede@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM testvm2006.codfw.wmnet - slyngshede@cumin1001" [production]
06:48 <slyngshede@cumin1001> START - Cookbook sre.dns.netbox [production]
06:48 <slyngshede@cumin1001> START - Cookbook sre.ganeti.makevm for new host testvm2006.codfw.wmnet [production]
06:44 <marostegui@cumin1001> dbctl commit (dc=all): 'db2110 (re)pooling @ 10%: Repooling after maintenance', diff saved to https://phabricator.wikimedia.org/P48628 and previous config saved to /var/cache/conftool/dbconfig/20230530-064427-root.json [production]
06:29 <marostegui@cumin1001> dbctl commit (dc=all): 'db2110 (re)pooling @ 5%: Repooling after maintenance', diff saved to https://phabricator.wikimedia.org/P48625 and previous config saved to /var/cache/conftool/dbconfig/20230530-062922-root.json [production]
06:14 <marostegui@cumin1001> dbctl commit (dc=all): 'db2110 (re)pooling @ 3%: Repooling after maintenance', diff saved to https://phabricator.wikimedia.org/P48624 and previous config saved to /var/cache/conftool/dbconfig/20230530-061417-root.json [production]
05:59 <marostegui@cumin1001> dbctl commit (dc=all): 'db2110 (re)pooling @ 1%: Repooling after maintenance', diff saved to https://phabricator.wikimedia.org/P48623 and previous config saved to /var/cache/conftool/dbconfig/20230530-055913-root.json [production]
05:43 <ayounsi@cumin1001> END (ERROR) - Cookbook sre.network.peering (exit_code=97) with action 'configure' for AS: 62597 [production]