2101-2150 of 10000 results (53ms)
2022-04-06 ยง
13:39 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on db2125.codfw.wmnet with reason: Rebooting for T303174 [production]
13:39 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on db2125.codfw.wmnet with reason: Rebooting for T303174 [production]
13:38 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:38 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:37 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:36 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:36 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
13:36 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
13:34 <mmandere@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4021.ulsfo.wmnet with reason: host reimage [production]
13:33 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host urldownloader1001.wikimedia.org [production]
13:31 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:31 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on db2107.codfw.wmnet with reason: Rebooting for T303174 [production]
13:31 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on db2107.codfw.wmnet with reason: Rebooting for T303174 [production]
13:31 <mmandere@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp4021.ulsfo.wmnet with reason: host reimage [production]
13:30 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host urldownloader1001.wikimedia.org [production]
13:30 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:29 <kartik@deploy1002> Synchronized php-1.39.0-wmf.6/extensions/Translate/tag/PageTranslationHooks.php: Backport: [[gerrit:777767|Revert "PageTranslationHooks: Don't kick in during interface message parsing" (T305531)]] (duration: 00m 57s) [production]
13:26 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
13:26 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
13:25 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
13:24 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:24 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:23 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:23 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on db2095.codfw.wmnet with reason: Rebooting for T303174 [production]
13:23 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on db2095.codfw.wmnet with reason: Rebooting for T303174 [production]
13:20 <kartik@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:776254|Start writing to $wmgUsingKubernetes the same value as to $wmfUsingKubernetes (T45956)]] (duration: 00m 55s) [production]
13:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
13:18 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on db2094.codfw.wmnet with reason: Rebooting for T303174 [production]
13:18 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on db2094.codfw.wmnet with reason: Rebooting for T303174 [production]
13:16 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host urldownloader2002.wikimedia.org [production]
13:15 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
13:15 <mmandere@cumin1001> START - Cookbook sre.hosts.reimage for host cp4021.ulsfo.wmnet with OS buster [production]
13:13 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
13:13 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host urldownloader2002.wikimedia.org [production]
13:11 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
13:11 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
13:11 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
13:10 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
13:07 <mmandere> depool cp4021 for reimage - T290005 [production]
13:03 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
12:53 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
12:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1146:3312 (T297189)', diff saved to https://phabricator.wikimedia.org/P24158 and previous config saved to /var/cache/conftool/dbconfig/20220406-125117-marostegui.json [production]
12:51 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
12:51 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
12:49 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
12:45 <mmandere> pool cp4033 with HAProxy as TLS termination layer - T290005 [production]
12:42 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
12:38 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]