401-450 of 10000 results (95ms)
2023-11-09 ยง
11:08 <hnowlan@deploy2002> helmfile [staging] DONE helmfile.d/services/edit-analytics: apply [production]
11:08 <hnowlan@deploy2002> helmfile [staging] START helmfile.d/services/edit-analytics: apply [production]
11:07 <mvolz@deploy2002> helmfile [eqiad] DONE helmfile.d/services/zotero: apply [production]
11:06 <mvolz@deploy2002> helmfile [eqiad] START helmfile.d/services/zotero: apply [production]
11:06 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/editor-analytics: apply [production]
11:06 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/editor-analytics: apply [production]
11:06 <hnowlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/editor-analytics: apply [production]
11:05 <hnowlan@deploy2002> helmfile [eqiad] START helmfile.d/services/editor-analytics: apply [production]
11:05 <hnowlan@deploy2002> helmfile [staging] DONE helmfile.d/services/editor-analytics: apply [production]
11:05 <mvolz@deploy2002> helmfile [codfw] DONE helmfile.d/services/zotero: apply [production]
11:05 <hnowlan@deploy2002> helmfile [staging] START helmfile.d/services/editor-analytics: apply [production]
11:04 <mvolz@deploy2002> helmfile [codfw] START helmfile.d/services/zotero: apply [production]
11:04 <mvolz@deploy2002> helmfile [staging] DONE helmfile.d/services/zotero: apply [production]
11:03 <mvolz@deploy2002> helmfile [staging] START helmfile.d/services/zotero: apply [production]
10:32 <jiji@deploy2002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
10:31 <jiji@deploy2002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
10:28 <jiji@deploy2002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
10:27 <jiji@deploy2002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
09:55 <btullis@deploy2002> helmfile [eqiad] DONE helmfile.d/services/datahub: sync on main [production]
09:52 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host urldownloader2003.wikimedia.org [production]
09:51 <btullis@deploy2002> helmfile [eqiad] START helmfile.d/services/datahub: sync on main [production]
09:51 <btullis@deploy2002> helmfile [codfw] DONE helmfile.d/services/datahub: sync on main [production]
09:41 <btullis@deploy2002> helmfile [codfw] START helmfile.d/services/datahub: sync on main [production]
09:41 <jnuche@deploy2002> rebuilt and synchronized wikiversions files: Deploy 1.42.0-wmf.4 to group2 (labswiki staying at 1.42.0-wmf.3 due to T350836) [production]
09:35 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: insetup::infrastructure_foundations [production]
09:24 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: insetup::infrastructure_foundations [production]
09:18 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: insetup::data_engineering [production]
09:08 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: insetup::data_engineering [production]
08:47 <godog> add 50G to prometheus/ml-serve in codfw [production]
08:35 <Emperor> restart vopsbot.service on alert1001 [production]
08:25 <stevemunene@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on an-airflow1007.eqiad.wmnet with reason: Downtime as we setup the new WMDE Airflow instance [production]
08:25 <stevemunene@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on an-airflow1007.eqiad.wmnet with reason: Downtime as we setup the new WMDE Airflow instance [production]
08:21 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: etcd::v3::dse_k8s_etcd [production]
08:20 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubestagetcd2001.codfw.wmnet [production]
08:19 <kharlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
08:19 <kharlan@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
08:16 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host kubestagetcd2001.codfw.wmnet [production]
08:15 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: etcd::v3::kubernetes::staging [production]
08:14 <kharlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
08:13 <kharlan@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
08:07 <oblivian@deploy2002> helmfile [codfw] [canary] DONE helmfile.d/services/mw-jobrunner : sync [production]
08:07 <oblivian@deploy2002> helmfile [codfw] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
08:07 <oblivian@deploy2002> helmfile [codfw] [canary] START helmfile.d/services/mw-jobrunner : sync [production]
08:07 <oblivian@deploy2002> helmfile [codfw] [main] START helmfile.d/services/mw-jobrunner : sync [production]
07:55 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: etcd::v3::kubernetes::staging [production]
07:35 <moritzm> installing openjdk-8 security updates [production]
07:15 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2112.codfw.wmnet with reason: Maintenance [production]
07:15 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2112.codfw.wmnet with reason: Maintenance [production]
07:09 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depool db2112 T350142', diff saved to https://phabricator.wikimedia.org/P53177 and previous config saved to /var/cache/conftool/dbconfig/20231109-070936-arnaudb.json [production]
07:04 <arnaudb@cumin1001> dbctl commit (dc=all): 'Promote db2103 to s1 primary and set section read-write T350142', diff saved to https://phabricator.wikimedia.org/P53176 and previous config saved to /var/cache/conftool/dbconfig/20231109-070410-arnaudb.json [production]