801-850 of 10000 results (32ms)
2022-03-10 §
06:29 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance [production]
06:29 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance [production]
06:28 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1116.eqiad.wmnet with reason: Maintenance [production]
06:28 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1116.eqiad.wmnet with reason: Maintenance [production]
06:28 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
06:28 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
06:25 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
06:25 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
06:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P22236 and previous config saved to /var/cache/conftool/dbconfig/20220310-062345-marostegui.json [production]
06:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P22235 and previous config saved to /var/cache/conftool/dbconfig/20220310-060840-marostegui.json [production]
06:07 <marostegui> dbmaint on s3@eqiad T272512 [production]
06:05 <marostegui> dbmaint on s7@eqiad T272512 [production]
05:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T298294)', diff saved to https://phabricator.wikimedia.org/P22234 and previous config saved to /var/cache/conftool/dbconfig/20220310-055335-marostegui.json [production]
05:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1099:3311 (T298294)', diff saved to https://phabricator.wikimedia.org/P22233 and previous config saved to /var/cache/conftool/dbconfig/20220310-054701-marostegui.json [production]
05:46 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
05:46 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
05:46 <marostegui> dbmaint on s5@eqiad T272512 [production]
05:46 <marostegui> dbmaint on s4@eqiad T272512 [production]
05:46 <marostegui> dbmaint on pc3@eqiad T272512 [production]
05:45 <marostegui> dbmaint on pc2@eqiad T272512 [production]
05:45 <marostegui> dbmaint on pc1@eqiad T272512 [production]
05:45 <marostegui> dbmaint on s2@eqiad T272512 [production]
05:42 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
05:42 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
05:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1170:3317 (T300775)', diff saved to https://phabricator.wikimedia.org/P22232 and previous config saved to /var/cache/conftool/dbconfig/20220310-053950-marostegui.json [production]
05:39 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
05:39 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
05:37 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
05:37 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
00:26 <ebysans@deploy1002> Finished deploy [airflow-dags/analytics@7975c27]: (no justification provided) (duration: 00m 08s) [production]
00:26 <ebysans@deploy1002> Started deploy [airflow-dags/analytics@7975c27]: (no justification provided) [production]
00:09 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
00:08 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
00:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
00:07 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
2022-03-09 §
23:17 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
23:16 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
23:16 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
23:14 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
23:09 <dancy@deploy1002> Synchronized php: group1 wikis to 1.38.0-wmf.25 refs T300201 (duration: 00m 49s) [production]
23:08 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
23:08 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
23:08 <dancy@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.38.0-wmf.25 refs T300201 [production]
23:00 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host cloudvirt1047.eqiad.wmnet [production]
22:59 <pt1979@cumin2002> START - Cookbook sre.hosts.dhcp for host cloudvirt1047.eqiad.wmnet [production]
22:54 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.dhcp (exit_code=99) for host cloudvirt1047.eqiad.wmnet [production]
22:54 <pt1979@cumin2002> START - Cookbook sre.hosts.dhcp for host cloudvirt1047.eqiad.wmnet [production]
22:35 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
22:35 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
22:31 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1127 (T300775)', diff saved to https://phabricator.wikimedia.org/P22229 and previous config saved to /var/cache/conftool/dbconfig/20220309-223130-marostegui.json [production]