2801-2850 of 10000 results (85ms)
2023-02-01 §
23:39 <zabe@deploy1002> zabe: Backport for [[gerrit:885908|Stop writing to cuc_user and cuc_user_text in group1 wikis (T233004)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
23:37 <zabe@deploy1002> Started scap: Backport for [[gerrit:885908|Stop writing to cuc_user and cuc_user_text in group1 wikis (T233004)]] [production]
23:31 <rzl@cumin2002> dbctl commit (dc=all): 'Depool db2181', diff saved to https://phabricator.wikimedia.org/P43574 and previous config saved to /var/cache/conftool/dbconfig/20230201-233140-rzl.json [production]
23:31 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp5022.eqsin.wmnet with reason: host reimage [production]
23:27 <brett@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp5022.eqsin.wmnet with reason: host reimage [production]
23:19 <dzahn@cumin1001> END (FAIL) - Cookbook sre.gitlab.upgrade (exit_code=99) on GitLab host gitlab2002.wikimedia.org with reason: security release [production]
23:17 <dancy@deploy1002> Synchronized php: group1 wikis to 1.40.0-wmf.21 refs T325584 (duration: 06m 57s) [production]
23:10 <dancy@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.40.0-wmf.21 refs T325584 [production]
23:01 <zabe@deploy1002> Finished scap: Backport for [[gerrit:885781|CachingKartographerEmbeddingHandler: Fall back to Special:BlankPage title (T328601)]] (duration: 07m 45s) [production]
22:55 <zabe@deploy1002> zabe: Backport for [[gerrit:885781|CachingKartographerEmbeddingHandler: Fall back to Special:BlankPage title (T328601)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet [production]
22:54 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host cp5022.eqsin.wmnet with OS bullseye [production]
22:53 <zabe@deploy1002> Started scap: Backport for [[gerrit:885781|CachingKartographerEmbeddingHandler: Fall back to Special:BlankPage title (T328601)]] [production]
22:49 <zabe@deploy1002> Finished scap: Backport for [[gerrit:885898|Stop writing to cuc_comment_id in group0 wikis (T233004)]] (duration: 13m 03s) [production]
22:47 <dzahn@cumin1001> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: security release [production]
22:40 <brett@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cp5022.eqsin.wmnet with OS bullseye [production]
22:38 <zabe@deploy1002> zabe: Backport for [[gerrit:885898|Stop writing to cuc_comment_id in group0 wikis (T233004)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
22:36 <zabe@deploy1002> Started scap: Backport for [[gerrit:885898|Stop writing to cuc_comment_id in group0 wikis (T233004)]] [production]
22:32 <kindrobot> close UTC late backport window [production]
22:31 <kindrobot@deploy1002> Finished scap: Backport for [[gerrit:885841|Enable client preferences for group1 (T327979)]] (duration: 10m 37s) [production]
22:22 <kindrobot@deploy1002> nray and kindrobot: Backport for [[gerrit:885841|Enable client preferences for group1 (T327979)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
22:20 <kindrobot@deploy1002> Started scap: Backport for [[gerrit:885841|Enable client preferences for group1 (T327979)]] [production]
22:14 <kindrobot@deploy1002> Finished scap: Backport for [[gerrit:885852|Enable Linter write namespace, tag and template for all wikis (T299612)]] (duration: 18m 14s) [production]
21:57 <kindrobot@deploy1002> kindrobot and sbailey: Backport for [[gerrit:885852|Enable Linter write namespace, tag and template for all wikis (T299612)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
21:57 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching sessionstore100*: Applying new TLS certificates — T327675 - eevans@cumin1001 [production]
21:56 <kindrobot@deploy1002> Started scap: Backport for [[gerrit:885852|Enable Linter write namespace, tag and template for all wikis (T299612)]] [production]
21:53 <aokoth@cumin1001> END (FAIL) - Cookbook sre.gitlab.upgrade (exit_code=99) on GitLab host gitlab2002.wikimedia.org with reason: Security Release [production]
21:52 <kindrobot@deploy1002> Finished scap: Backport for [[gerrit:885358|Disable write old for CheckUserLog reason on group 0 (T233004)]] (duration: 14m 53s) [production]
21:43 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host cp5022.eqsin.wmnet with OS bullseye [production]
21:39 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching sessionstore100*: Applying new TLS certificates — T327675 - eevans@cumin1001 [production]
21:39 <kindrobot@deploy1002> dreamyjazz and kindrobot: Backport for [[gerrit:885358|Disable write old for CheckUserLog reason on group 0 (T233004)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
21:37 <kindrobot@deploy1002> Started scap: Backport for [[gerrit:885358|Disable write old for CheckUserLog reason on group 0 (T233004)]] [production]
21:32 <kindrobot@deploy1002> Finished scap: Backport for [[gerrit:865214|Disable wgParserEnableLegacyMediaDOM on group1 wikis (T314318)]] (duration: 13m 56s) [production]
21:26 <eevans@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=sessionstore,name=codfw [production]
21:26 <eevans@puppetmaster1001> conftool action : get/pooled=true; selector: dnsdisc=sessionstore,name=codfw [production]
21:26 <eevans@puppetmaster1001> conftool action : get/pooled=true; selector: dnsdisc=sessionstore,name=codfw [production]
21:24 <aokoth@cumin1001> START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: Security Release [production]
21:20 <kindrobot@deploy1002> arlolra and kindrobot: Backport for [[gerrit:865214|Disable wgParserEnableLegacyMediaDOM on group1 wikis (T314318)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet [production]
21:19 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching sessionstore200*: Applying new TLS certificates — T327675 - eevans@cumin1001 [production]
21:18 <kindrobot@deploy1002> Started scap: Backport for [[gerrit:865214|Disable wgParserEnableLegacyMediaDOM on group1 wikis (T314318)]] [production]
21:14 <brett@cumin2002> conftool action : set/pooled=yes; selector: name=cp3065.esams.wmnet [production]
21:10 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp3065.esams.wmnet with OS bullseye [production]
21:03 <kindrobot> start UTC late backport deployment window [production]
21:02 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching sessionstore200*: Applying new TLS certificates — T327675 - eevans@cumin1001 [production]
20:46 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp3065.esams.wmnet with reason: host reimage [production]
20:44 <eevans@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=sessionstore,name=codfw [production]
20:43 <urandom> depooling sessionstore —codfw— in preparation for Cassandra restarts — T327675 [production]
20:42 <brett@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp3065.esams.wmnet with reason: host reimage [production]
20:40 <brett@cumin2002> conftool action : set/pooled=yes; selector: name=cp3064.esams.wmnet [production]
20:38 <eevans@puppetmaster1001> conftool action : get/pooled; selector: dnsdisc=$SERVICE,name=$DC [production]
20:33 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp3064.esams.wmnet with OS bullseye [production]