9551-9600 of 10000 results (58ms)
2022-05-24 §
04:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P28394 and previous config saved to /var/cache/conftool/dbconfig/20220524-042539-ladsgroup.json [production]
04:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T298555)', diff saved to https://phabricator.wikimedia.org/P28393 and previous config saved to /var/cache/conftool/dbconfig/20220524-041034-ladsgroup.json [production]
03:28 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
03:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
03:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T298555)', diff saved to https://phabricator.wikimedia.org/P28392 and previous config saved to /var/cache/conftool/dbconfig/20220524-032848-ladsgroup.json [production]
03:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P28391 and previous config saved to /var/cache/conftool/dbconfig/20220524-031343-ladsgroup.json [production]
02:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P28390 and previous config saved to /var/cache/conftool/dbconfig/20220524-025838-ladsgroup.json [production]
02:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T298555)', diff saved to https://phabricator.wikimedia.org/P28389 and previous config saved to /var/cache/conftool/dbconfig/20220524-024333-ladsgroup.json [production]
02:36 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
02:36 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
02:36 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
02:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
02:11 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
02:09 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
02:09 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
02:06 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
01:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1122 (T298555)', diff saved to https://phabricator.wikimedia.org/P28388 and previous config saved to /var/cache/conftool/dbconfig/20220524-015145-ladsgroup.json [production]
01:51 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
01:51 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
01:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298555)', diff saved to https://phabricator.wikimedia.org/P28387 and previous config saved to /var/cache/conftool/dbconfig/20220524-015137-ladsgroup.json [production]
01:37 <ryankemper@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host relforge1004.eqiad.wmnet with OS bullseye [production]
01:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P28386 and previous config saved to /var/cache/conftool/dbconfig/20220524-013632-ladsgroup.json [production]
01:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P28385 and previous config saved to /var/cache/conftool/dbconfig/20220524-012127-ladsgroup.json [production]
01:19 <ryankemper@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on relforge1004.eqiad.wmnet with reason: host reimage [production]
01:16 <ryankemper@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on relforge1004.eqiad.wmnet with reason: host reimage [production]
01:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1110 (T298560)', diff saved to https://phabricator.wikimedia.org/P28384 and previous config saved to /var/cache/conftool/dbconfig/20220524-010810-ladsgroup.json [production]
01:08 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
01:08 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
01:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T298560)', diff saved to https://phabricator.wikimedia.org/P28383 and previous config saved to /var/cache/conftool/dbconfig/20220524-010802-ladsgroup.json [production]
01:06 <ryankemper@cumin1001> START - Cookbook sre.hosts.reimage for host relforge1004.eqiad.wmnet with OS bullseye [production]
01:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298555)', diff saved to https://phabricator.wikimedia.org/P28382 and previous config saved to /var/cache/conftool/dbconfig/20220524-010622-ladsgroup.json [production]
01:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1112 (T298555)', diff saved to https://phabricator.wikimedia.org/P28381 and previous config saved to /var/cache/conftool/dbconfig/20220524-010534-ladsgroup.json [production]
01:05 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
01:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 20:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
01:05 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
01:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
01:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T298555)', diff saved to https://phabricator.wikimedia.org/P28380 and previous config saved to /var/cache/conftool/dbconfig/20220524-010521-ladsgroup.json [production]
00:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315', diff saved to https://phabricator.wikimedia.org/P28379 and previous config saved to /var/cache/conftool/dbconfig/20220524-005257-ladsgroup.json [production]
00:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P28378 and previous config saved to /var/cache/conftool/dbconfig/20220524-005016-ladsgroup.json [production]
00:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315', diff saved to https://phabricator.wikimedia.org/P28377 and previous config saved to /var/cache/conftool/dbconfig/20220524-003752-ladsgroup.json [production]
00:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P28376 and previous config saved to /var/cache/conftool/dbconfig/20220524-003511-ladsgroup.json [production]
00:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T298560)', diff saved to https://phabricator.wikimedia.org/P28375 and previous config saved to /var/cache/conftool/dbconfig/20220524-002246-ladsgroup.json [production]
00:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T298555)', diff saved to https://phabricator.wikimedia.org/P28374 and previous config saved to /var/cache/conftool/dbconfig/20220524-002006-ladsgroup.json [production]
2022-05-23 §
23:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T298555)', diff saved to https://phabricator.wikimedia.org/P28373 and previous config saved to /var/cache/conftool/dbconfig/20220523-235415-ladsgroup.json [production]
23:54 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
23:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
23:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1182 (T298555)', diff saved to https://phabricator.wikimedia.org/P28372 and previous config saved to /var/cache/conftool/dbconfig/20220523-235407-ladsgroup.json [production]
23:49 <ebernhardson@deploy1002> Finished deploy [wikimedia/discovery/analytics@02f2375]: increase driver jvm heap for convert_to_esbulk (duration: 02m 18s) [production]
23:47 <ebernhardson@deploy1002> Started deploy [wikimedia/discovery/analytics@02f2375]: increase driver jvm heap for convert_to_esbulk [production]
23:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1182', diff saved to https://phabricator.wikimedia.org/P28371 and previous config saved to /var/cache/conftool/dbconfig/20220523-233902-ladsgroup.json [production]