4801-4850 of 10000 results (109ms)
2023-03-15 ยง
14:54 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply [production]
14:54 <Emperor> depool moss-fe1001 as rate of token denial is too high [production]
14:54 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-api-int: apply [production]
14:54 <cgoubert@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-api-int: apply [production]
14:54 <cgoubert@deploy2002> helmfile [codfw] START helmfile.d/services/mw-api-int: apply [production]
14:54 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply [production]
14:53 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply [production]
14:53 <cgoubert@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-api-ext: apply [production]
14:53 <cgoubert@deploy2002> helmfile [codfw] START helmfile.d/services/mw-api-ext: apply [production]
14:53 <claime> Redeploying mw-on-k8s for php7.4 update T330270 [production]
14:52 <hnowlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/thumbor: apply [production]
14:49 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:46 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
14:41 <hnowlan@deploy2002> helmfile [eqiad] START helmfile.d/services/thumbor: apply [production]
14:41 <cgoubert@deploy2002> Started scap: (no justification provided) [production]
14:41 <claime> Rebuilding mw-on-k8s images - T330270 [production]
14:38 <claime> Updating php7.4 production images [production]
14:36 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:34 <hnowlan@deploy2002> helmfile [eqiad] START helmfile.d/services/thumbor: apply [production]
14:31 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on doh4002.wikimedia.org with reason: host reimage [production]
14:27 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on doh4002.wikimedia.org with reason: host reimage [production]
14:24 <daniel@deploy2002> Finished scap: Backport for [[gerrit:898795|Always write parsoid output to parser cache. (T320534)]] (duration: 09m 57s) [production]
14:22 <jbond@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) pki.discovery.wmnet on all recursors [production]
14:22 <jbond@cumin1001> START - Cookbook sre.dns.wipe-cache pki.discovery.wmnet on all recursors [production]
14:22 <jbond@cumin1001> conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=pki [production]
14:22 <jbond> switch pki to be active active [production]
14:20 <jbond@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) pki.discovery.wmnet on all recursors [production]
14:20 <jbond@cumin1001> START - Cookbook sre.dns.wipe-cache pki.discovery.wmnet on all recursors [production]
14:19 <jbond> update pki to use discovery record [production]
14:16 <jbond@cumin1001> conftool action : set/pooled=true; selector: name=codfw,dnsdisc=pki [production]
14:15 <daniel@deploy2002> daniel: Backport for [[gerrit:898795|Always write parsoid output to parser cache. (T320534)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
14:14 <sukhe@cumin2002> START - Cookbook sre.ganeti.reimage for host doh4002.wikimedia.org with OS bullseye [production]
14:14 <daniel@deploy2002> Started scap: Backport for [[gerrit:898795|Always write parsoid output to parser cache. (T320534)]] [production]
14:12 <sukhe> [correction] depool _doh4002_ for reimaging to bullseye: T321309 [production]
14:12 <sukhe> depool dns4002 for reimaging to bullseye: T321309 [production]
14:00 <moritzm> nodejs security updates on buster [production]
13:51 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-logging1003.eqiad.wmnet with OS bullseye [production]
13:50 <sukhe> reprepro -C component/pdns-recursor include bullseye-wikimedia pdns-recursor_4.6.2-1+wmf11u1_amd64.changes: T321309 [production]
13:49 <moritzm> installing graphite-web security updates [production]
13:32 <jayme@deploy2002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
13:32 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-logging1003.eqiad.wmnet with reason: host reimage [production]
13:30 <jayme@deploy2002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
13:30 <jayme@deploy2002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
13:28 <jayme@deploy2002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
13:28 <jayme@deploy2002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:28 <hnowlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/thumbor: apply [production]
13:27 <jayme@deploy2002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. [production]
13:27 <jayme@deploy2002> helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. [production]
13:27 <herron@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-logging1003.eqiad.wmnet with reason: host reimage [production]
13:26 <jayme@deploy2002> helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. [production]