1951-2000 of 10000 results (37ms)
2022-02-22 ยง
21:25 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: ee7608c7b56b579e2aaa50b504b6c2e28b63058e: Deploy the fawiki test safety survey to production (T297629) (duration: 00m 51s) [production]
21:19 <cwhite> end opensearch upgrade (codfw) T299168 [production]
21:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:12 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:12 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:12 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host elastic2076.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:06 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcontrol1004.wikimedia.org with OS bullseye [production]
21:05 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:03 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be2069.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:55 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:49 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:49 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:42 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:36 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudcontrol1004.wikimedia.org with OS bullseye [production]
20:29 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host ms-be2069.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:27 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be2068.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:26 <cwhite> begin opensearch upgrade (codfw) T299168 [production]
20:17 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:13 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:13 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:11 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:10 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host ms-be2068.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:09 <ryankemper> T302340 [WCQS] Seeing `0.3.104` running on the hosts now [production]
20:09 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be2067.mgmt.codfw.wmnet with reboot policy FORCED [production]
20:08 <ryankemper@deploy1002> Finished deploy [wdqs/wdqs@5d384a5] (wcqs): Deploy 0.3.104 to WCQS (duration: 02m 33s) [production]
20:07 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.38.0-wmf.23 refs T300199 [production]
20:06 <ryankemper@deploy1002> Started deploy [wdqs/wdqs@5d384a5] (wcqs): Deploy 0.3.104 to WCQS [production]
20:06 <ryankemper> T302340 [WCQS] Forgot to fetch & rebase `deploy1002:/srv/deployment/wdqs/wdqs` before deploy, so `0.3.104` did not actually deploy (still on `0.3.103`). Re-rolling deploy... [production]
20:00 <ryankemper@deploy1002> Finished deploy [wdqs/wdqs@f0d05eb] (wcqs): Deploy 0.3.104 to WCQS (duration: 03m 00s) [production]
19:58 <ryankemper> T302340 `scap deploy -v --environment wcqs 'Deploy 0.3.104 to WCQS'` [production]
19:57 <ryankemper@deploy1002> Started deploy [wdqs/wdqs@f0d05eb] (wcqs): Deploy 0.3.104 to WCQS [production]
19:56 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
19:50 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
19:50 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
19:49 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host ms-be2067.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:44 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
19:43 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be2066.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:39 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
19:32 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
19:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
19:25 <ryankemper> T302330 `ryankemper@cumin1001:~$ sudo -E cumin '*mwmaint*' 'run-puppet-agent'` (getting https://gerrit.wikimedia.org/r/c/operations/puppet/+/764875 out) [production]
19:25 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
19:24 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts logstash[2004-2006].codfw.wmnet [production]
19:20 <dduvall@deploy1002> Pruned MediaWiki: 1.38.0-wmf.21 (duration: 03m 50s) [production]
19:16 <dduvall@deploy1002> Finished scap: testwikis wikis to 1.38.0-wmf.23 refs T300199 (duration: 49m 17s) [production]
19:11 <herron@cumin1001> START - Cookbook sre.hosts.decommission for hosts logstash[2004-2006].codfw.wmnet [production]
19:10 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts logstash[1007-1009].eqiad.wmnet [production]
19:07 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host ms-be2066.mgmt.codfw.wmnet with reboot policy FORCED [production]
18:58 <ssastry@deploy1002> helmfile [eqiad] DONE helmfile.d/services/proton: apply [production]
18:56 <ssastry@deploy1002> helmfile [eqiad] START helmfile.d/services/proton: apply [production]