3851-3900 of 10000 results (46ms)
2022-03-02 ยง
19:36 <robh@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:33 <robh@cumin1001> START - Cookbook sre.dns.netbox [production]
19:30 <mutante> stopped icinga-wm [production]
19:14 <brennen@deploy1002> Synchronized php: group1 wikis to 1.38.0-wmf.24 refs T300200 (duration: 00m 50s) [production]
19:13 <brennen@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.38.0-wmf.24 refs T300200 [production]
19:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1148 (T300992)', diff saved to https://phabricator.wikimedia.org/P21729 and previous config saved to /var/cache/conftool/dbconfig/20220302-191323-ladsgroup.json [production]
19:10 <brennen> 1.38.0-wmf.24 train (T300200): no current blockers; proceeding to group1 [production]
18:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1148', diff saved to https://phabricator.wikimedia.org/P21728 and previous config saved to /var/cache/conftool/dbconfig/20220302-185819-ladsgroup.json [production]
18:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1148', diff saved to https://phabricator.wikimedia.org/P21727 and previous config saved to /var/cache/conftool/dbconfig/20220302-184314-ladsgroup.json [production]
18:30 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1148 (T300992)', diff saved to https://phabricator.wikimedia.org/P21726 and previous config saved to /var/cache/conftool/dbconfig/20220302-182809-ladsgroup.json [production]
18:26 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
18:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1148 (T300992)', diff saved to https://phabricator.wikimedia.org/P21725 and previous config saved to /var/cache/conftool/dbconfig/20220302-182153-ladsgroup.json [production]
18:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1148.eqiad.wmnet with reason: Maintenance [production]
18:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1148.eqiad.wmnet with reason: Maintenance [production]
18:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1149 (T300992)', diff saved to https://phabricator.wikimedia.org/P21724 and previous config saved to /var/cache/conftool/dbconfig/20220302-182145-ladsgroup.json [production]
18:14 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-media: apply [production]
18:14 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-media: apply [production]
18:14 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
18:13 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
18:13 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/proton: apply [production]
18:13 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/proton: apply [production]
18:13 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/mathoid: apply [production]
18:12 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/mathoid: apply [production]
18:12 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/linkrecommendation: apply [production]
18:12 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/linkrecommendation: apply [production]
18:12 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/eventstreams-internal: apply [production]
18:12 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/eventstreams-internal: apply [production]
18:12 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/eventstreams: apply [production]
18:11 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/eventstreams: apply [production]
18:11 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-main: apply [production]
18:11 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/eventgate-main: apply [production]
18:11 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-logging-external: apply [production]
18:10 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/eventgate-logging-external: apply [production]
18:10 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-analytics: apply [production]
18:10 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/eventgate-analytics: apply [production]
18:10 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
18:10 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/eventgate-analytics-external: apply [production]
18:10 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/cxserver: apply [production]
18:09 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
18:09 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/blubberoid: apply [production]
18:09 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/blubberoid: apply [production]
18:09 <rzl@deploy1002> helmfile [staging] DONE helmfile.d/services/apertium: apply [production]
18:09 <rzl@deploy1002> helmfile [staging] START helmfile.d/services/apertium: apply [production]
18:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1149', diff saved to https://phabricator.wikimedia.org/P21723 and previous config saved to /var/cache/conftool/dbconfig/20220302-180640-ladsgroup.json [production]
17:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1149', diff saved to https://phabricator.wikimedia.org/P21722 and previous config saved to /var/cache/conftool/dbconfig/20220302-175136-ladsgroup.json [production]
17:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1149 (T300992)', diff saved to https://phabricator.wikimedia.org/P21721 and previous config saved to /var/cache/conftool/dbconfig/20220302-173631-ladsgroup.json [production]
17:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1149 (T300992)', diff saved to https://phabricator.wikimedia.org/P21720 and previous config saved to /var/cache/conftool/dbconfig/20220302-173112-ladsgroup.json [production]
17:31 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1149.eqiad.wmnet with reason: Maintenance [production]
17:31 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1149.eqiad.wmnet with reason: Maintenance [production]