2501-2550 of 10000 results (86ms)
2024-01-31 §
03:29 <eileen> tools upgraded from 02281338 to c823e692 [production]
03:05 <fab@deploy2002> Finished deploy [airflow-dags/research@6a97a34]: (no justification provided) (duration: 00m 23s) [production]
03:05 <fab@deploy2002> Started deploy [airflow-dags/research@6a97a34]: (no justification provided) [production]
2024-01-30 §
23:54 <mutante> LDAP - added aklapper to group releng T356043 [production]
23:07 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for sessionstore1006.eqiad.wmnet [production]
23:07 <eevans@cumin1002> START - Cookbook sre.hosts.remove-downtime for sessionstore1006.eqiad.wmnet [production]
22:49 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on sessionstore1006.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
22:48 <eevans@cumin1002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on sessionstore1006.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
22:40 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
22:20 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for sessionstore1005.eqiad.wmnet [production]
22:20 <eevans@cumin1002> START - Cookbook sre.hosts.remove-downtime for sessionstore1005.eqiad.wmnet [production]
22:10 <cjming> end of UTC late backport window [production]
22:09 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994254|[eswiki] Add 13 namespaces to $wgExemptFromUserRobotsControl (T355033)]] (duration: 08m 24s) [production]
22:02 <cjming@deploy2002> cjming and superpes: Continuing with sync [production]
22:02 <cjming@deploy2002> cjming and superpes: Backport for [[gerrit:994254|[eswiki] Add 13 namespaces to $wgExemptFromUserRobotsControl (T355033)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:00 <cjming@deploy2002> Started scap: Backport for [[gerrit:994254|[eswiki] Add 13 namespaces to $wgExemptFromUserRobotsControl (T355033)]] [production]
21:59 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994211|[ukwiki] Change autoconfirmed setting (T355972)]], [[gerrit:994214|[ganwiki] Add 'suppressredirect' to transwiki usergroup and change assignment and revocation methods (T354850)]], [[gerrit:994220|[ganwiki] Add new namespace aliases (T355854)]] (duration: 09m 32s) [production]
21:53 <cjming@deploy2002> superpes and cjming: Continuing with sync [production]
21:51 <cjming@deploy2002> superpes and cjming: Backport for [[gerrit:994211|[ukwiki] Change autoconfirmed setting (T355972)]], [[gerrit:994214|[ganwiki] Add 'suppressredirect' to transwiki usergroup and change assignment and revocation methods (T354850)]], [[gerrit:994220|[ganwiki] Add new namespace aliases (T355854)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:50 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on sessionstore1005.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
21:50 <eevans@cumin1002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on sessionstore1005.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
21:49 <cjming@deploy2002> Started scap: Backport for [[gerrit:994211|[ukwiki] Change autoconfirmed setting (T355972)]], [[gerrit:994214|[ganwiki] Add 'suppressredirect' to transwiki usergroup and change assignment and revocation methods (T354850)]], [[gerrit:994220|[ganwiki] Add new namespace aliases (T355854)]] [production]
21:44 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994143|Run CheckerJob against read-only clusters (T354793)]] (duration: 07m 41s) [production]
21:42 <mutante> LDAP - added jnuche to group releng (T356043) - already done/approved in the past in T301149 [production]
21:41 <mutante> LDAP - added jhuneidi to group releng (T356043) - already done/approved in the past in T210028 [production]
21:40 <mutante> LDAP - added brennen to group releng (T356043) - already done/approved in the past in T215365 [production]
21:38 <cjming@deploy2002> cjming and ebernhardson: Continuing with sync [production]
21:38 <cjming@deploy2002> cjming and ebernhardson: Backport for [[gerrit:994143|Run CheckerJob against read-only clusters (T354793)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:37 <cjming@deploy2002> Started scap: Backport for [[gerrit:994143|Run CheckerJob against read-only clusters (T354793)]] [production]
21:36 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994142|Run CheckerJob against read-only clusters (T354793)]] (duration: 07m 49s) [production]
21:34 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
21:30 <cjming@deploy2002> ebernhardson and cjming: Continuing with sync [production]
21:30 <cjming@deploy2002> ebernhardson and cjming: Backport for [[gerrit:994142|Run CheckerJob against read-only clusters (T354793)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:28 <cjming@deploy2002> Started scap: Backport for [[gerrit:994142|Run CheckerJob against read-only clusters (T354793)]] [production]
21:01 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for sessionstore1004.eqiad.wmnet [production]
21:01 <eevans@cumin1002> START - Cookbook sre.hosts.remove-downtime for sessionstore1004.eqiad.wmnet [production]
20:52 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
20:51 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
20:38 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on sessionstore1004.eqiad.wmnet with reason: Commissioning — T353402 [production]
20:38 <eevans@cumin1002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on sessionstore1004.eqiad.wmnet with reason: Commissioning — T353402 [production]
20:35 <urandom> bootstrapping sessionstore1004/cassandra-a — T353402 [production]
20:01 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: wdqs::public [production]
19:45 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: wdqs::public [production]
19:36 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in cloudelastic [production]
19:36 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in cloudelastic [production]
19:36 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.ban (exit_code=99) Banning hosts: cloudelastic1010.eqiad.wmnet for use cloudelastic1010 as migration canary - bking@cumin2002 - T355617 [production]
19:36 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: cloudelastic1010.eqiad.wmnet for use cloudelastic1010 as migration canary - bking@cumin2002 - T355617 [production]
19:27 <Lucas_WMDE> FINISHED lucaswerkmeister-wmde@mwmaint2002:~$ mwscript CheckSignatures enwiki | tee T356168 # -- 268378 invalid signatures -- [production]
19:10 <dancy@deploy2002> rebuilt and synchronized wikiversions files: group0 wikis to 1.42.0-wmf.16 refs T354434 [production]
19:09 <ayounsi@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2005.codfw.wmnet with OS bookworm [production]