551-600 of 10000 results (52ms)
2022-04-28 ยง
23:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138 (T298565)', diff saved to https://phabricator.wikimedia.org/P26949 and previous config saved to /var/cache/conftool/dbconfig/20220428-231701-ladsgroup.json [production]
23:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3312', diff saved to https://phabricator.wikimedia.org/P26948 and previous config saved to /var/cache/conftool/dbconfig/20220428-231550-ladsgroup.json [production]
23:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138', diff saved to https://phabricator.wikimedia.org/P26947 and previous config saved to /var/cache/conftool/dbconfig/20220428-230156-ladsgroup.json [production]
23:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3312', diff saved to https://phabricator.wikimedia.org/P26946 and previous config saved to /var/cache/conftool/dbconfig/20220428-230045-ladsgroup.json [production]
22:54 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host aqs2004.mgmt.codfw.wmnet with reboot policy FORCED [production]
22:54 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host aqs2003.mgmt.codfw.wmnet with reboot policy FORCED [production]
22:48 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2002.mgmt.codfw.wmnet with reboot policy FORCED [production]
22:48 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs2001.mgmt.codfw.wmnet with reboot policy FORCED [production]
22:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138', diff saved to https://phabricator.wikimedia.org/P26945 and previous config saved to /var/cache/conftool/dbconfig/20220428-224650-ladsgroup.json [production]
22:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3312 (T306560)', diff saved to https://phabricator.wikimedia.org/P26944 and previous config saved to /var/cache/conftool/dbconfig/20220428-224540-ladsgroup.json [production]
22:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1170:3312 (T306560)', diff saved to https://phabricator.wikimedia.org/P26943 and previous config saved to /var/cache/conftool/dbconfig/20220428-224426-ladsgroup.json [production]
22:44 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
22:44 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
22:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T306560)', diff saved to https://phabricator.wikimedia.org/P26942 and previous config saved to /var/cache/conftool/dbconfig/20220428-224417-ladsgroup.json [production]
22:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138 (T298565)', diff saved to https://phabricator.wikimedia.org/P26941 and previous config saved to /var/cache/conftool/dbconfig/20220428-223145-ladsgroup.json [production]
22:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P26940 and previous config saved to /var/cache/conftool/dbconfig/20220428-222912-ladsgroup.json [production]
22:21 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host aqs2002.mgmt.codfw.wmnet with reboot policy FORCED [production]
22:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1138 (T298565)', diff saved to https://phabricator.wikimedia.org/P26939 and previous config saved to /var/cache/conftool/dbconfig/20220428-222035-ladsgroup.json [production]
22:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1138.eqiad.wmnet with reason: Maintenance [production]
22:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1138.eqiad.wmnet with reason: Maintenance [production]
22:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138 (T298565)', diff saved to https://phabricator.wikimedia.org/P26938 and previous config saved to /var/cache/conftool/dbconfig/20220428-222022-ladsgroup.json [production]
22:20 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host aqs2001.mgmt.codfw.wmnet with reboot policy FORCED [production]
22:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P26937 and previous config saved to /var/cache/conftool/dbconfig/20220428-221407-ladsgroup.json [production]
22:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138', diff saved to https://phabricator.wikimedia.org/P26936 and previous config saved to /var/cache/conftool/dbconfig/20220428-220517-ladsgroup.json [production]
22:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T306560)', diff saved to https://phabricator.wikimedia.org/P26935 and previous config saved to /var/cache/conftool/dbconfig/20220428-220450-ladsgroup.json [production]
21:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T306560)', diff saved to https://phabricator.wikimedia.org/P26934 and previous config saved to /var/cache/conftool/dbconfig/20220428-215902-ladsgroup.json [production]
21:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1122 (T306560)', diff saved to https://phabricator.wikimedia.org/P26933 and previous config saved to /var/cache/conftool/dbconfig/20220428-215547-ladsgroup.json [production]
21:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
21:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
21:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138', diff saved to https://phabricator.wikimedia.org/P26932 and previous config saved to /var/cache/conftool/dbconfig/20220428-215012-ladsgroup.json [production]
21:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P26931 and previous config saved to /var/cache/conftool/dbconfig/20220428-214945-ladsgroup.json [production]
21:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1138 (T298565)', diff saved to https://phabricator.wikimedia.org/P26930 and previous config saved to /var/cache/conftool/dbconfig/20220428-213507-ladsgroup.json [production]
21:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P26929 and previous config saved to /var/cache/conftool/dbconfig/20220428-213440-ladsgroup.json [production]
21:27 <tgr> UTC late deploys done [production]
21:26 <tgr@deploy1002> Synchronized php-1.39.0-wmf.9/extensions/GrowthExperiments/includes/VariantHooks.php: Backport: [[gerrit:787461|Video landing page: Record campaign parameter for control users (T303785)]] (duration: 00m 54s) [production]
21:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1138 (T298565)', diff saved to https://phabricator.wikimedia.org/P26928 and previous config saved to /var/cache/conftool/dbconfig/20220428-212331-ladsgroup.json [production]
21:23 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1138.eqiad.wmnet with reason: Maintenance [production]
21:23 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1138.eqiad.wmnet with reason: Maintenance [production]
21:20 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T306560)', diff saved to https://phabricator.wikimedia.org/P26927 and previous config saved to /var/cache/conftool/dbconfig/20220428-211934-ladsgroup.json [production]
21:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1168 (T306560)', diff saved to https://phabricator.wikimedia.org/P26926 and previous config saved to /var/cache/conftool/dbconfig/20220428-211727-ladsgroup.json [production]
21:17 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
21:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
21:15 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
21:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T306560)', diff saved to https://phabricator.wikimedia.org/P26925 and previous config saved to /var/cache/conftool/dbconfig/20220428-211507-ladsgroup.json [production]
21:13 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:12 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:12 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:11 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:08 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]