6101-6150 of 10000 results (81ms)
2024-03-06 ยง
12:02 <cgoubert@cumin2002> START - Cookbook sre.hosts.reimage for host mw2371.codfw.wmnet with OS bullseye [production]
12:01 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
12:01 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:59 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
11:59 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:54 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
11:54 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:53 <moritzm> restarting Exim on the MXes to pick up new GNU TLS [production]
11:52 <jmm@cumin2002> END (PASS) - Cookbook sre.ldap.roll-restart-reboot-replica (exit_code=0) rolling restart_daemons on A:ldap-replicas-eqiad [production]
11:51 <jmm@cumin2002> START - Cookbook sre.ldap.roll-restart-reboot-replica rolling restart_daemons on A:ldap-replicas-eqiad [production]
11:48 <jmm@cumin2002> END (PASS) - Cookbook sre.ldap.roll-restart-reboot-replica (exit_code=0) rolling restart_daemons on A:ldap-replicas-codfw [production]
11:47 <jmm@cumin2002> START - Cookbook sre.ldap.roll-restart-reboot-replica rolling restart_daemons on A:ldap-replicas-codfw [production]
11:43 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
11:43 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:42 <claime> Depooling mw2371.codfw.wmnet,mw2372.codfw.wmnet,mw2373.codfw.wmnet,mw2374.codfw.wmnet,mw2375.codfw.wmnet,mw2376.codfw.wmnet for reimage to kubernetes - T351074 [production]
11:41 <cgoubert@cumin2002> conftool action : set/weight=30; selector: cluster=api_appserver,service=canary,dc=codfw [production]
11:41 <cgoubert@cumin2002> conftool action : set/pooled=yes; selector: cluster=api_appserver,service=canary,dc=codfw [production]
11:40 <claime> pooling new canaries - T351074 [production]
11:37 <claime> Enabling and running puppet on deployment servers - T351074 [production]
11:33 <claime> Enabling and running puppet on new canaries mw2283.codfw.wmnet,mw2284.codfw.wmnet - T351074 [production]
11:31 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
11:31 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
11:31 <claime> Disabling puppet on mw2374.codfw.wmnet,mw2376.codfw.wmnet,mw2283.codfw.wmnet,mw2284.codfw.wmnet,mw2371.codfw.wmnet,mw2372.codfw.wmnet,mw2373.codfw.wmnet,mw2375.codfw.wmnet for canary api_appserver move - T351074 [production]
11:28 <claime> Disabling puppet on deployment servers for canary api_appserver move - T351074 [production]
11:21 <jgiannelos@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply [production]
11:21 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.clone (exit_code=0) Will create a clone of db2106.codfw.wmnet onto db2206.codfw.wmnet [production]
11:21 <jgiannelos@deploy2002> helmfile [eqiad] START helmfile.d/services/mobileapps: apply [production]
11:20 <jgiannelos@deploy2002> helmfile [codfw] DONE helmfile.d/services/mobileapps: apply [production]
11:19 <jgiannelos@deploy2002> helmfile [codfw] START helmfile.d/services/mobileapps: apply [production]
11:17 <jgiannelos@deploy2002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
11:17 <jgiannelos@deploy2002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
11:10 <akosiaris@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
11:10 <akosiaris@deploy2002> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
11:08 <akosiaris@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
11:08 <akosiaris@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
11:04 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.clone (exit_code=0) Will create a clone of db2108.codfw.wmnet onto db2208.codfw.wmnet [production]
10:50 <arnaudb@cumin1002> dbctl commit (dc=all): 'Host has already been cloned, there was 2 candidate master', diff saved to https://phabricator.wikimedia.org/P58580 and previous config saved to /var/cache/conftool/dbconfig/20240306-105007-arnaudb.json [production]
10:49 <jiji@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-mcrouter: sync [production]
10:49 <jiji@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-mcrouter: sync [production]
10:48 <jiji@deploy2002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
10:47 <jiji@deploy2002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
10:46 <jiji@deploy2002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
10:46 <jiji@deploy2002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
10:38 <akosiaris@cumin1002> conftool action : set/weight=10:pooled=yes; selector: name=(parse2008.codfw.wmnet|parse2009.codfw.wmnet|parse2010.codfw.wmnet|parse2011.codfw.wmnet|parse2012.codfw.wmnet|parse2013.codfw.wmnet|parse2014.codfw.wmnet|parse2015.codfw.wmnet),cluster=kubernetes,service=kubesvc [production]
10:37 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
10:37 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
10:34 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.clone (exit_code=0) Will create a clone of db2105.codfw.wmnet onto db2205.codfw.wmnet [production]
10:24 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2204 (re)pooling @ 100%: Cloning done', diff saved to https://phabricator.wikimedia.org/P58579 and previous config saved to /var/cache/conftool/dbconfig/20240306-102404-arnaudb.json [production]
10:24 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2203 (re)pooling @ 100%: Cloning done', diff saved to https://phabricator.wikimedia.org/P58578 and previous config saved to /var/cache/conftool/dbconfig/20240306-102402-arnaudb.json [production]
10:23 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2196 (re)pooling @ 100%: Cloning done', diff saved to https://phabricator.wikimedia.org/P58577 and previous config saved to /var/cache/conftool/dbconfig/20240306-102357-arnaudb.json [production]