2801-2850 of 10000 results (83ms)
2023-08-07 §
06:22 <kartik@deploy1002> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
06:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1224 upgrade to mariadb 10.6', diff saved to https://phabricator.wikimedia.org/P50149 and previous config saved to /var/cache/conftool/dbconfig/20230807-061653-root.json [production]
06:10 <ayounsi@cumin1001> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Update wheels for Aerleon 1.6.0 upgrade - ayounsi@cumin1001 [production]
06:09 <ayounsi@cumin1001> START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Update wheels for Aerleon 1.6.0 upgrade - ayounsi@cumin1001 [production]
2023-08-05 §
05:57 <_joe_> mounting the volume under /srv/dataimport on both puppetmaster frontends [production]
05:53 <_joe_> creating logical volume "dataimport" on the puppetmaster frontends [production]
02:40 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
02:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
01:38 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1225.eqiad.wmnet with reason: Maintenance [production]
01:38 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1225.eqiad.wmnet with reason: Maintenance [production]
01:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1222 (T342617)', diff saved to https://phabricator.wikimedia.org/P50148 and previous config saved to /var/cache/conftool/dbconfig/20230805-013831-ladsgroup.json [production]
01:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1222', diff saved to https://phabricator.wikimedia.org/P50147 and previous config saved to /var/cache/conftool/dbconfig/20230805-012325-ladsgroup.json [production]
01:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1222', diff saved to https://phabricator.wikimedia.org/P50146 and previous config saved to /var/cache/conftool/dbconfig/20230805-010819-ladsgroup.json [production]
00:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1222 (T342617)', diff saved to https://phabricator.wikimedia.org/P50145 and previous config saved to /var/cache/conftool/dbconfig/20230805-005312-ladsgroup.json [production]
00:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T342617)', diff saved to https://phabricator.wikimedia.org/P50144 and previous config saved to /var/cache/conftool/dbconfig/20230805-003155-ladsgroup.json [production]
00:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P50143 and previous config saved to /var/cache/conftool/dbconfig/20230805-001649-ladsgroup.json [production]
00:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P50142 and previous config saved to /var/cache/conftool/dbconfig/20230805-000143-ladsgroup.json [production]
2023-08-04 §
23:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T342617)', diff saved to https://phabricator.wikimedia.org/P50141 and previous config saved to /var/cache/conftool/dbconfig/20230804-234637-ladsgroup.json [production]
23:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1222 (T342617)', diff saved to https://phabricator.wikimedia.org/P50140 and previous config saved to /var/cache/conftool/dbconfig/20230804-234121-ladsgroup.json [production]
23:41 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1222.eqiad.wmnet with reason: Maintenance [production]
23:41 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1222.eqiad.wmnet with reason: Maintenance [production]
23:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T342617)', diff saved to https://phabricator.wikimedia.org/P50139 and previous config saved to /var/cache/conftool/dbconfig/20230804-234101-ladsgroup.json [production]
23:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P50138 and previous config saved to /var/cache/conftool/dbconfig/20230804-232555-ladsgroup.json [production]
23:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P50137 and previous config saved to /var/cache/conftool/dbconfig/20230804-231048-ladsgroup.json [production]
23:00 <tzatziki> removing 1 file for legal compliance [production]
22:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T342617)', diff saved to https://phabricator.wikimedia.org/P50136 and previous config saved to /var/cache/conftool/dbconfig/20230804-225542-ladsgroup.json [production]
22:33 <bking@deploy1002> Finished deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 (duration: 00m 54s) [production]
22:32 <bking@deploy1002> Started deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 [production]
22:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2175 (T342617)', diff saved to https://phabricator.wikimedia.org/P50135 and previous config saved to /var/cache/conftool/dbconfig/20230804-222905-ladsgroup.json [production]
22:29 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2175.codfw.wmnet with reason: Maintenance [production]
22:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2175.codfw.wmnet with reason: Maintenance [production]
22:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T342617)', diff saved to https://phabricator.wikimedia.org/P50134 and previous config saved to /var/cache/conftool/dbconfig/20230804-222845-ladsgroup.json [production]
22:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1197 (T342617)', diff saved to https://phabricator.wikimedia.org/P50133 and previous config saved to /var/cache/conftool/dbconfig/20230804-221915-ladsgroup.json [production]
22:19 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
22:18 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
22:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T342617)', diff saved to https://phabricator.wikimedia.org/P50132 and previous config saved to /var/cache/conftool/dbconfig/20230804-221855-ladsgroup.json [production]
22:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2170:3312', diff saved to https://phabricator.wikimedia.org/P50131 and previous config saved to /var/cache/conftool/dbconfig/20230804-221338-ladsgroup.json [production]
22:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P50130 and previous config saved to /var/cache/conftool/dbconfig/20230804-220348-ladsgroup.json [production]
21:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2170:3312', diff saved to https://phabricator.wikimedia.org/P50129 and previous config saved to /var/cache/conftool/dbconfig/20230804-215832-ladsgroup.json [production]
21:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P50128 and previous config saved to /var/cache/conftool/dbconfig/20230804-214842-ladsgroup.json [production]
21:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T342617)', diff saved to https://phabricator.wikimedia.org/P50127 and previous config saved to /var/cache/conftool/dbconfig/20230804-214326-ladsgroup.json [production]
21:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T342617)', diff saved to https://phabricator.wikimedia.org/P50126 and previous config saved to /var/cache/conftool/dbconfig/20230804-213336-ladsgroup.json [production]
21:20 <bking@deploy1002> Finished deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 (duration: 00m 44s) [production]
21:19 <bking@deploy1002> Started deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 [production]
21:16 <bking@deploy1002> Finished deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 (duration: 00m 09s) [production]
21:16 <bking@deploy1002> Started deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 [production]
21:16 <bking@deploy1002> Finished deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 (duration: 00m 15s) [production]
21:15 <bking@deploy1002> Started deploy [wdqs/wdqs@dff41b7] (wcqs): 0.3.124 [production]
20:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1188 (T342617)', diff saved to https://phabricator.wikimedia.org/P50125 and previous config saved to /var/cache/conftool/dbconfig/20230804-205647-ladsgroup.json [production]
20:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1188.eqiad.wmnet with reason: Maintenance [production]