4801-4850 of 10000 results (67ms)
2022-05-09 ยง
15:34 <sukhe> depool doh6001 (as part of T307427) [production]
14:05 <taavi> UTC afternoon backport window done [production]
14:05 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:04 <taavi@deploy1002> Synchronized php-1.39.0-wmf.10/extensions/ContentTranslation/app: Backport: [[gerrit:790328|CX3 Build 0.2.0+20220509 (T306643)]] (duration: 00m 51s) [production]
14:04 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:04 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:03 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:58 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:57 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:57 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:56 <jhathaway@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx2001.wikimedia.org with reason: old kernel :( [production]
13:56 <jhathaway@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on mx2001.wikimedia.org with reason: old kernel :( [production]
13:54 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:52 <jhathaway@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx1001.wikimedia.org with reason: old kernel :( [production]
13:52 <jhathaway@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on mx1001.wikimedia.org with reason: old kernel :( [production]
13:49 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:49 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host ganeti-test2001.codfw.wmnet [production]
13:48 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:48 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:47 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:780874|Newcomer tasks: deploy AND topic selection to pilot wikis (T305399)]] (duration: 00m 49s) [production]
13:45 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:41 <filippo@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host centrallog2002.codfw.wmnet [production]
13:41 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host centrallog2002.codfw.wmnet [production]
13:40 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:39 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:39 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:39 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet [production]
13:38 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:37 <taavi@deploy1002> Synchronized php-1.39.0-wmf.10/extensions/ContentTranslation/modules/entrypoints: Backport: [[gerrit:789832|ULS entrypoint: Do not show current language, fix domain redirects (T307745 T298032)]] (duration: 00m 50s) [production]
13:36 <taavi@deploy1002> Synchronized docroot/wwwportal/w/search-redirect.php: Config: [[gerrit:789972|search-redirect.php: Make sure the family is lowercased (T304629)]] (duration: 00m 51s) [production]
13:33 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:32 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:29 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:29 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:789974|rowiki: Fix canonical namespaces (T127607)]] (duration: 00m 51s) [production]
13:26 <moritzm> failover ganeti master in codfw/test to ganeti-test2003 [production]
13:24 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:24 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:24 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:23 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2003.codfw.wmnet [production]
13:21 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:789889|ptwiki: Revoke 500KB uploading limitation (T307813)]] (duration: 00m 50s) [production]
13:21 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:17 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2003.codfw.wmnet [production]
13:16 <taavi@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:788777|Set log level to 'debug' for mediamoderation (T303312)]] (duration: 00m 50s) [production]
13:06 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet [production]
12:59 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet [production]
12:54 <moritzm> installing perf updates on stretch/buster hosts [production]
12:46 <moritzm> installing perf updates on bullseye hosts [production]
12:45 <klausman@deploy1002> Finished deploy [ores/deploy@98a1b2e]: (no justification provided) (duration: 00m 07s) [production]
12:45 <klausman@deploy1002> Started deploy [ores/deploy@98a1b2e]: (no justification provided) [production]