2151-2200 of 10000 results (62ms)
2022-06-28 ยง
21:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1184 (T298560)', diff saved to https://phabricator.wikimedia.org/P30596 and previous config saved to /var/cache/conftool/dbconfig/20220628-213806-ladsgroup.json [production]
21:38 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
21:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
21:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132 (T298560)', diff saved to https://phabricator.wikimedia.org/P30595 and previous config saved to /var/cache/conftool/dbconfig/20220628-213735-ladsgroup.json [production]
21:31 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host clouddb2002-dev.codfw.wmnet with OS bullseye [production]
21:30 <pt1979@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host clouddb2002-dev.codfw.wmnet with OS bullseye [production]
21:25 <cjming> end of UTC late backport window [production]
21:22 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132', diff saved to https://phabricator.wikimedia.org/P30594 and previous config saved to /var/cache/conftool/dbconfig/20220628-212230-ladsgroup.json [production]
21:20 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:808056|Enable title above tabs on group 1 and group 0 wikis (1/2) (T310054)]] (duration: 03m 34s) [production]
21:19 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:16 <cjming@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/VisualEditor: Backport: [[gerrit:809249|Prevent skinStyles from applying to the Vector 2022 skin. (T310197)]] (duration: 03m 38s) [production]
21:13 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:12 <cjming@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/VisualEditor: Backport: [[gerrit:809249|Prevent skinStyles from applying to the Vector 2022 skin. (T310197)]] (duration: 03m 33s) [production]
21:12 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:12 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:08 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host clouddb2002-dev.codfw.wmnet with OS bullseye [production]
21:07 <volans@cumin2002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts sretest2001.codfw.wmnet [production]
21:07 <volans@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132', diff saved to https://phabricator.wikimedia.org/P30592 and previous config saved to /var/cache/conftool/dbconfig/20220628-210725-ladsgroup.json [production]
21:05 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:04 <cjming@deploy1002> Synchronized php-1.39.0-wmf.17/extensions/VisualEditor: Backport: [[gerrit:809245|Prevent skinStyles from applying to the Vector 2022 skin. (T310197)]] (duration: 03m 27s) [production]
21:03 <volans@cumin2002> START - Cookbook sre.dns.netbox [production]
21:02 <volans> deployed spicerack 3.0.0 to cumin1001 [production]
21:00 <cjming@deploy1002> Synchronized php-1.39.0-wmf.17/extensions/VisualEditor/modules: Backport: [[gerrit:808071|Rename `data-ve-target-container` attribute to `data-mw-ve-target-container` (T310197)]] (duration: 03m 33s) [production]
21:00 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:59 <volans@cumin2002> START - Cookbook sre.hosts.decommission for hosts sretest2001.codfw.wmnet [production]
20:57 <volans@cumin2002> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host sretest2001.codfw.wmnet [production]
20:56 <cjming@deploy1002> Synchronized php-1.39.0-wmf.17/skins/Vector/includes/templates/skin.mustache: Backport: [[gerrit:808068|Rename `data-ve-target-container` attribute to `data-mw-ve-target-container` (T310197)]] (duration: 03m 33s) [production]
20:56 <mforns@deploy1002> Finished deploy [analytics/refinery@2f5987d]: Regular analytics weekly train [analytics/refinery@2f5987d] (duration: 00m 26s) [production]
20:55 <mforns@deploy1002> Started deploy [analytics/refinery@2f5987d]: Regular analytics weekly train [analytics/refinery@2f5987d] [production]
20:54 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:54 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:53 <mforns@deploy1002> deploy aborted: Regular analytics weekly train [analytics/refinery@2f5987d] (duration: 21m 55s) [production]
20:53 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:52 <volans@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) sretest2001.codfw.wmnet on all recursors [production]
20:52 <volans@cumin2002> START - Cookbook sre.dns.wipe-cache sretest2001.codfw.wmnet on all recursors [production]
20:52 <volans@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132 (T298560)', diff saved to https://phabricator.wikimedia.org/P30591 and previous config saved to /var/cache/conftool/dbconfig/20220628-205220-ladsgroup.json [production]
20:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
20:52 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
20:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T298560)', diff saved to https://phabricator.wikimedia.org/P30590 and previous config saved to /var/cache/conftool/dbconfig/20220628-205206-ladsgroup.json [production]
20:48 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:48 <volans@cumin2002> START - Cookbook sre.dns.netbox [production]
20:48 <volans@cumin2002> START - Cookbook sre.ganeti.makevm for new host sretest2001.codfw.wmnet [production]
20:47 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:47 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:47 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]