251-300 of 10000 results (71ms)
2023-02-14 ยง
23:07 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
23:05 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
23:03 <cwhite@deploy1002> Started deploy [releng/phatality@eaa4c16]: T314098 [production]
23:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P44651 and previous config saved to /var/cache/conftool/dbconfig/20230214-230025-ladsgroup.json [production]
22:51 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2443.codfw.wmnet with reason: host reimage [production]
22:48 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2442.codfw.wmnet with reason: host reimage [production]
22:46 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2443.codfw.wmnet with reason: host reimage [production]
22:45 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2442.codfw.wmnet with reason: host reimage [production]
22:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2157 (T328255)', diff saved to https://phabricator.wikimedia.org/P44650 and previous config saved to /var/cache/conftool/dbconfig/20230214-224519-ladsgroup.json [production]
22:39 <eoghan@cumin2002> END (PASS) - Cookbook sre.ganeti.reimage (exit_code=0) for host aphlict2001.codfw.wmnet with OS bullseye [production]
22:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2157 (T328255)', diff saved to https://phabricator.wikimedia.org/P44649 and previous config saved to /var/cache/conftool/dbconfig/20230214-223931-ladsgroup.json [production]
22:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
22:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
22:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3315 (T328255)', diff saved to https://phabricator.wikimedia.org/P44648 and previous config saved to /var/cache/conftool/dbconfig/20230214-223910-ladsgroup.json [production]
22:28 <eoghan@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aphlict2001.codfw.wmnet with reason: host reimage [production]
22:26 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on an-airflow1005.eqiad.wmnet with reason: new OS but some puppet stuff doesn't work yet [production]
22:26 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on an-airflow1005.eqiad.wmnet with reason: new OS but some puppet stuff doesn't work yet [production]
22:26 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host mw2443.codfw.wmnet with OS buster [production]
22:25 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host mw2442.codfw.wmnet with OS buster [production]
22:25 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2441.codfw.wmnet with OS buster [production]
22:25 <pt1979@cumin2002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
22:25 <eoghan@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on aphlict2001.codfw.wmnet with reason: host reimage [production]
22:25 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw2440.codfw.wmnet with OS buster [production]
22:25 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
22:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3315', diff saved to https://phabricator.wikimedia.org/P44647 and previous config saved to /var/cache/conftool/dbconfig/20230214-222403-ladsgroup.json [production]
22:10 <eoghan@cumin2002> START - Cookbook sre.ganeti.reimage for host aphlict2001.codfw.wmnet with OS bullseye [production]
22:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3315', diff saved to https://phabricator.wikimedia.org/P44646 and previous config saved to /var/cache/conftool/dbconfig/20230214-220857-ladsgroup.json [production]
22:04 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
21:59 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
21:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3315 (T328255)', diff saved to https://phabricator.wikimedia.org/P44645 and previous config saved to /var/cache/conftool/dbconfig/20230214-215351-ladsgroup.json [production]
21:47 <dancy@deploy1002> Finished scap: Backport for [[gerrit:889220|Change linter maintenance scripts to use existing config varaibles (T329342)]] (duration: 07m 44s) [production]
21:47 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2441.codfw.wmnet with reason: host reimage [production]
21:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2137:3315 (T328255)', diff saved to https://phabricator.wikimedia.org/P44644 and previous config saved to /var/cache/conftool/dbconfig/20230214-214642-ladsgroup.json [production]
21:46 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2137.codfw.wmnet with reason: Maintenance [production]
21:46 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db2137.codfw.wmnet with reason: Maintenance [production]
21:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128 (T328255)', diff saved to https://phabricator.wikimedia.org/P44643 and previous config saved to /var/cache/conftool/dbconfig/20230214-214621-ladsgroup.json [production]
21:44 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2441.codfw.wmnet with reason: host reimage [production]
21:42 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2440.codfw.wmnet with reason: host reimage [production]
21:41 <dancy@deploy1002> dancy and sbailey: Backport for [[gerrit:889220|Change linter maintenance scripts to use existing config varaibles (T329342)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
21:39 <dancy@deploy1002> Started scap: Backport for [[gerrit:889220|Change linter maintenance scripts to use existing config varaibles (T329342)]] [production]
21:39 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw2440.codfw.wmnet with reason: host reimage [production]
21:35 <dancy@deploy1002> Backport cancelled. [production]
21:34 <dancy@deploy1002> Finished scap: Backport for [[gerrit:888764|Enable Page Tools for logged in users across all wikis (T328692)]] (duration: 12m 50s) [production]
21:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P44642 and previous config saved to /var/cache/conftool/dbconfig/20230214-213115-ladsgroup.json [production]
21:23 <dancy@deploy1002> dancy and bwang: Backport for [[gerrit:888764|Enable Page Tools for logged in users across all wikis (T328692)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet [production]
21:21 <dancy@deploy1002> Started scap: Backport for [[gerrit:888764|Enable Page Tools for logged in users across all wikis (T328692)]] [production]
21:20 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host mw2441.codfw.wmnet with OS buster [production]
21:20 <dancy@deploy1002> Backport cancelled. [production]
21:19 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host mw2440.codfw.wmnet with OS buster [production]
21:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P44640 and previous config saved to /var/cache/conftool/dbconfig/20230214-211608-ladsgroup.json [production]