451-500 of 10000 results (76ms)
2024-01-31 §
05:53 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2097.codfw.wmnet with reason: Maintenance [production]
05:53 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db2097.codfw.wmnet with reason: Maintenance [production]
05:50 <marostegui@cumin1002> dbctl commit (dc=all): 'db1224 (re)pooling @ 1%: After onsite maintenance', diff saved to https://phabricator.wikimedia.org/P55904 and previous config saved to /var/cache/conftool/dbconfig/20240131-055057-root.json [production]
05:41 <eileen> civicrm upgraded from 6de61520 to 520337a0 [production]
05:30 <fab@deploy2002> Finished deploy [airflow-dags/research@97c6a4e]: (no justification provided) (duration: 00m 14s) [production]
05:30 <fab@deploy2002> Started deploy [airflow-dags/research@97c6a4e]: (no justification provided) [production]
03:29 <eileen> tools upgraded from 02281338 to c823e692 [production]
03:05 <fab@deploy2002> Finished deploy [airflow-dags/research@6a97a34]: (no justification provided) (duration: 00m 23s) [production]
03:05 <fab@deploy2002> Started deploy [airflow-dags/research@6a97a34]: (no justification provided) [production]
2024-01-30 §
23:54 <mutante> LDAP - added aklapper to group releng T356043 [production]
23:07 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for sessionstore1006.eqiad.wmnet [production]
23:07 <eevans@cumin1002> START - Cookbook sre.hosts.remove-downtime for sessionstore1006.eqiad.wmnet [production]
22:49 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on sessionstore1006.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
22:48 <eevans@cumin1002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on sessionstore1006.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
22:40 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
22:20 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for sessionstore1005.eqiad.wmnet [production]
22:20 <eevans@cumin1002> START - Cookbook sre.hosts.remove-downtime for sessionstore1005.eqiad.wmnet [production]
22:10 <cjming> end of UTC late backport window [production]
22:09 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994254|[eswiki] Add 13 namespaces to $wgExemptFromUserRobotsControl (T355033)]] (duration: 08m 24s) [production]
22:02 <cjming@deploy2002> cjming and superpes: Continuing with sync [production]
22:02 <cjming@deploy2002> cjming and superpes: Backport for [[gerrit:994254|[eswiki] Add 13 namespaces to $wgExemptFromUserRobotsControl (T355033)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:00 <cjming@deploy2002> Started scap: Backport for [[gerrit:994254|[eswiki] Add 13 namespaces to $wgExemptFromUserRobotsControl (T355033)]] [production]
21:59 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994211|[ukwiki] Change autoconfirmed setting (T355972)]], [[gerrit:994214|[ganwiki] Add 'suppressredirect' to transwiki usergroup and change assignment and revocation methods (T354850)]], [[gerrit:994220|[ganwiki] Add new namespace aliases (T355854)]] (duration: 09m 32s) [production]
21:53 <cjming@deploy2002> superpes and cjming: Continuing with sync [production]
21:51 <cjming@deploy2002> superpes and cjming: Backport for [[gerrit:994211|[ukwiki] Change autoconfirmed setting (T355972)]], [[gerrit:994214|[ganwiki] Add 'suppressredirect' to transwiki usergroup and change assignment and revocation methods (T354850)]], [[gerrit:994220|[ganwiki] Add new namespace aliases (T355854)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:50 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on sessionstore1005.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
21:50 <eevans@cumin1002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on sessionstore1005.eqiad.wmnet with reason: Bootstrapping — T353402 [production]
21:49 <cjming@deploy2002> Started scap: Backport for [[gerrit:994211|[ukwiki] Change autoconfirmed setting (T355972)]], [[gerrit:994214|[ganwiki] Add 'suppressredirect' to transwiki usergroup and change assignment and revocation methods (T354850)]], [[gerrit:994220|[ganwiki] Add new namespace aliases (T355854)]] [production]
21:44 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994143|Run CheckerJob against read-only clusters (T354793)]] (duration: 07m 41s) [production]
21:42 <mutante> LDAP - added jnuche to group releng (T356043) - already done/approved in the past in T301149 [production]
21:41 <mutante> LDAP - added jhuneidi to group releng (T356043) - already done/approved in the past in T210028 [production]
21:40 <mutante> LDAP - added brennen to group releng (T356043) - already done/approved in the past in T215365 [production]
21:38 <cjming@deploy2002> cjming and ebernhardson: Continuing with sync [production]
21:38 <cjming@deploy2002> cjming and ebernhardson: Backport for [[gerrit:994143|Run CheckerJob against read-only clusters (T354793)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:37 <cjming@deploy2002> Started scap: Backport for [[gerrit:994143|Run CheckerJob against read-only clusters (T354793)]] [production]
21:36 <cjming@deploy2002> Finished scap: Backport for [[gerrit:994142|Run CheckerJob against read-only clusters (T354793)]] (duration: 07m 49s) [production]
21:34 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
21:30 <cjming@deploy2002> ebernhardson and cjming: Continuing with sync [production]
21:30 <cjming@deploy2002> ebernhardson and cjming: Backport for [[gerrit:994142|Run CheckerJob against read-only clusters (T354793)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:28 <cjming@deploy2002> Started scap: Backport for [[gerrit:994142|Run CheckerJob against read-only clusters (T354793)]] [production]
21:01 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for sessionstore1004.eqiad.wmnet [production]
21:01 <eevans@cumin1002> START - Cookbook sre.hosts.remove-downtime for sessionstore1004.eqiad.wmnet [production]
20:52 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
20:51 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: activate first private IP host config - bking@cumin2002 - T355617 [production]
20:38 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on sessionstore1004.eqiad.wmnet with reason: Commissioning — T353402 [production]
20:38 <eevans@cumin1002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on sessionstore1004.eqiad.wmnet with reason: Commissioning — T353402 [production]
20:35 <urandom> bootstrapping sessionstore1004/cassandra-a — T353402 [production]
20:01 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: wdqs::public [production]
19:45 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: wdqs::public [production]
19:36 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in cloudelastic [production]