6751-6800 of 10000 results (102ms)
2024-02-10 §
11:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1198.eqiad.wmnet with reason: Maintenance [production]
11:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1198.eqiad.wmnet with reason: Maintenance [production]
11:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T352010)', diff saved to https://phabricator.wikimedia.org/P56624 and previous config saved to /var/cache/conftool/dbconfig/20240210-112129-ladsgroup.json [production]
11:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P56623 and previous config saved to /var/cache/conftool/dbconfig/20240210-110622-ladsgroup.json [production]
10:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P56622 and previous config saved to /var/cache/conftool/dbconfig/20240210-105116-ladsgroup.json [production]
10:44 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2101.codfw.wmnet with reason: Maintenance [production]
10:43 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2101.codfw.wmnet with reason: Maintenance [production]
10:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T352010)', diff saved to https://phabricator.wikimedia.org/P56621 and previous config saved to /var/cache/conftool/dbconfig/20240210-103609-ladsgroup.json [production]
08:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
08:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
05:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1245.eqiad.wmnet with reason: Maintenance [production]
05:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1245.eqiad.wmnet with reason: Maintenance [production]
05:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1244:3315 (T352010)', diff saved to https://phabricator.wikimedia.org/P56620 and previous config saved to /var/cache/conftool/dbconfig/20240210-054721-ladsgroup.json [production]
05:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1244:3315', diff saved to https://phabricator.wikimedia.org/P56619 and previous config saved to /var/cache/conftool/dbconfig/20240210-053215-ladsgroup.json [production]
05:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1244:3315', diff saved to https://phabricator.wikimedia.org/P56618 and previous config saved to /var/cache/conftool/dbconfig/20240210-051708-ladsgroup.json [production]
05:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1244:3315 (T352010)', diff saved to https://phabricator.wikimedia.org/P56617 and previous config saved to /var/cache/conftool/dbconfig/20240210-050201-ladsgroup.json [production]
03:28 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1189.eqiad.wmnet with reason: Maintenance [production]
03:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1189.eqiad.wmnet with reason: Maintenance [production]
03:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T352010)', diff saved to https://phabricator.wikimedia.org/P56615 and previous config saved to /var/cache/conftool/dbconfig/20240210-032801-ladsgroup.json [production]
03:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P56614 and previous config saved to /var/cache/conftool/dbconfig/20240210-031255-ladsgroup.json [production]
02:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P56613 and previous config saved to /var/cache/conftool/dbconfig/20240210-025748-ladsgroup.json [production]
02:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T352010)', diff saved to https://phabricator.wikimedia.org/P56612 and previous config saved to /var/cache/conftool/dbconfig/20240210-024242-ladsgroup.json [production]
02:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1244:3315 (T352010)', diff saved to https://phabricator.wikimedia.org/P56611 and previous config saved to /var/cache/conftool/dbconfig/20240210-021141-ladsgroup.json [production]
02:11 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1244.eqiad.wmnet with reason: Maintenance [production]
02:11 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1244.eqiad.wmnet with reason: Maintenance [production]
02:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1230 (T352010)', diff saved to https://phabricator.wikimedia.org/P56610 and previous config saved to /var/cache/conftool/dbconfig/20240210-021119-ladsgroup.json [production]
01:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P56609 and previous config saved to /var/cache/conftool/dbconfig/20240210-015612-ladsgroup.json [production]
01:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P56608 and previous config saved to /var/cache/conftool/dbconfig/20240210-014106-ladsgroup.json [production]
01:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1230 (T352010)', diff saved to https://phabricator.wikimedia.org/P56607 and previous config saved to /var/cache/conftool/dbconfig/20240210-012559-ladsgroup.json [production]
2024-02-09 §
23:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1230 (T352010)', diff saved to https://phabricator.wikimedia.org/P56606 and previous config saved to /var/cache/conftool/dbconfig/20240209-230425-ladsgroup.json [production]
23:04 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1230.eqiad.wmnet with reason: Maintenance [production]
23:04 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1230.eqiad.wmnet with reason: Maintenance [production]
21:38 <inflatador> bking@deploy2002 install 'python3-boto3' pkg T348685 [production]
21:36 <inflatador> bking@deploy2002 install 'python3-plac' pkg T348685 [production]
21:09 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: apply new systemd settings - bking@cumin2002 - T355617 [production]
21:06 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: apply new systemd settings - bking@cumin2002 - T355617 [production]
20:55 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: apply new systemd settings - bking@cumin2002 - T355617 [production]
20:46 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.UPGRADE (1 nodes at a time) for ElasticSearch cluster relforge: apply new systemd settings - bking@cumin2002 - T355617 [production]
20:28 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
20:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
20:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3315 (T352010)', diff saved to https://phabricator.wikimedia.org/P56605 and previous config saved to /var/cache/conftool/dbconfig/20240209-202830-ladsgroup.json [production]
20:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3315', diff saved to https://phabricator.wikimedia.org/P56604 and previous config saved to /var/cache/conftool/dbconfig/20240209-201324-ladsgroup.json [production]
19:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3315', diff saved to https://phabricator.wikimedia.org/P56603 and previous config saved to /var/cache/conftool/dbconfig/20240209-195817-ladsgroup.json [production]
19:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1213:3315 (T352010)', diff saved to https://phabricator.wikimedia.org/P56602 and previous config saved to /var/cache/conftool/dbconfig/20240209-194310-ladsgroup.json [production]
19:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1175 (T352010)', diff saved to https://phabricator.wikimedia.org/P56601 and previous config saved to /var/cache/conftool/dbconfig/20240209-193452-ladsgroup.json [production]
19:34 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
19:34 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
19:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T352010)', diff saved to https://phabricator.wikimedia.org/P56600 and previous config saved to /var/cache/conftool/dbconfig/20240209-193430-ladsgroup.json [production]
19:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P56599 and previous config saved to /var/cache/conftool/dbconfig/20240209-191923-ladsgroup.json [production]
19:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P56598 and previous config saved to /var/cache/conftool/dbconfig/20240209-190416-ladsgroup.json [production]