2501-2550 of 10000 results (67ms)
2022-09-13 ยง
22:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P34657 and previous config saved to /var/cache/conftool/dbconfig/20220913-224244-ladsgroup.json [production]
22:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2169:3317', diff saved to https://phabricator.wikimedia.org/P34656 and previous config saved to /var/cache/conftool/dbconfig/20220913-223241-ladsgroup.json [production]
22:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2149 (T314041)', diff saved to https://phabricator.wikimedia.org/P34655 and previous config saved to /var/cache/conftool/dbconfig/20220913-223025-ladsgroup.json [production]
22:30 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
22:30 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
22:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1202 (T314041)', diff saved to https://phabricator.wikimedia.org/P34654 and previous config saved to /var/cache/conftool/dbconfig/20220913-222738-ladsgroup.json [production]
22:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
22:19 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
22:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
22:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
22:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2169:3317 (T314041)', diff saved to https://phabricator.wikimedia.org/P34653 and previous config saved to /var/cache/conftool/dbconfig/20220913-221734-ladsgroup.json [production]
22:16 <dancy> dancy@deploy1002$ rm /var/lib/deploy-mwdebug/pause [production]
22:15 <dancy@deploy1002> Sync cancelled. [production]
22:15 <dancy@deploy1002> dancy: testing synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
22:14 <dancy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
22:14 <dancy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
22:14 <dancy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
22:13 <dancy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
22:12 <dancy@deploy1002> Started scap: testing [production]
22:12 <dancy@deploy1002> Sync cancelled. [production]
22:11 <dancy@deploy1002> dancy: testing synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
22:11 <dancy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
22:11 <dancy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
22:10 <dancy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
22:08 <dancy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
22:07 <dancy@deploy1002> Started scap: testing [production]
22:07 <dancy@deploy1002> Sync cancelled. [production]
22:07 <dancy@deploy1002> dancy: testing synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
22:06 <dancy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
22:06 <dancy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
22:05 <dancy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
22:03 <dancy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
22:02 <dancy@deploy1002> Started scap: testing [production]
22:01 <dancy@deploy1002> Sync cancelled. [production]
22:01 <dancy@deploy1002> dancy: testing synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
22:01 <dancy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:58 <dancy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:58 <dancy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:55 <dancy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:55 <dancy@deploy1002> Started scap: testing [production]
21:55 <dancy@deploy1002> Sync cancelled. [production]
21:54 <dancy@deploy1002> dancy: testing synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet [production]
21:54 <dancy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:50 <dancy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:50 <dancy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:48 <dancy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:47 <dancy@deploy1002> Started scap: testing [production]
21:37 <dancy@deploy1002> scap failed: CalledProcessError Command 'sudo -u mwbuilder /usr/bin/make -C /srv/mwbuilder/release/make-container-image -f Makefile build-and-push-all-images http_proxy=http://webproxy.eqiad.wmnet:8080 https_proxy=http://webproxy.eqiad.wmnet:8080 GIT_BASE=https://gerrit.wikimedia.org/r/ MW_CONFIG_BRANCH=master workdir_volume=/srv/mediawiki-staging mv_image_name=docker-registry.discovery.wmnet/restric [production]
21:36 <dancy@deploy1002> Started scap: testing [production]
21:18 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]