3001-3050 of 10000 results (50ms)
2022-05-04 ยง
23:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P27554 and previous config saved to /var/cache/conftool/dbconfig/20220504-230553-ladsgroup.json [production]
23:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T307525)', diff saved to https://phabricator.wikimedia.org/P27553 and previous config saved to /var/cache/conftool/dbconfig/20220504-230432-ladsgroup.json [production]
22:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1140.eqiad.wmnet with reason: Maintenance [production]
22:57 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1140.eqiad.wmnet with reason: Maintenance [production]
22:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136 (T307525)', diff saved to https://phabricator.wikimedia.org/P27552 and previous config saved to /var/cache/conftool/dbconfig/20220504-225048-ladsgroup.json [production]
22:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 8 hosts with reason: Maintenance [production]
22:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 8 hosts with reason: Maintenance [production]
22:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
22:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
22:44 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
22:43 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
22:43 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
22:42 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
22:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance [production]
22:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1005.eqiad.wmnet with reason: Maintenance [production]
22:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316 (T307525)', diff saved to https://phabricator.wikimedia.org/P27551 and previous config saved to /var/cache/conftool/dbconfig/20220504-223732-ladsgroup.json [production]
22:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1136 (T307525)', diff saved to https://phabricator.wikimedia.org/P27550 and previous config saved to /var/cache/conftool/dbconfig/20220504-223601-ladsgroup.json [production]
22:35 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
22:35 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
22:30 <razzi@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host matomo1002.eqiad.wmnet [production]
22:26 <razzi@cumin1001> START - Cookbook sre.hosts.reboot-single for host matomo1002.eqiad.wmnet [production]
22:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P27548 and previous config saved to /var/cache/conftool/dbconfig/20220504-222227-ladsgroup.json [production]
22:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P27547 and previous config saved to /var/cache/conftool/dbconfig/20220504-220722-ladsgroup.json [production]
22:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T307525)', diff saved to https://phabricator.wikimedia.org/P27546 and previous config saved to /var/cache/conftool/dbconfig/20220504-220417-ladsgroup.json [production]
22:04 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
22:04 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
22:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T307525)', diff saved to https://phabricator.wikimedia.org/P27545 and previous config saved to /var/cache/conftool/dbconfig/20220504-220409-ladsgroup.json [production]
21:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 10 hosts with reason: Maintenance [production]
21:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 10 hosts with reason: Maintenance [production]
21:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance [production]
21:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance [production]
21:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316 (T307525)', diff saved to https://phabricator.wikimedia.org/P27544 and previous config saved to /var/cache/conftool/dbconfig/20220504-215217-ladsgroup.json [production]
21:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P27543 and previous config saved to /var/cache/conftool/dbconfig/20220504-214903-ladsgroup.json [production]
21:45 <urbanecm> Start server-side upload of 1 TIFF file (~2.1G; T300857) [production]
21:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1096:3316 (T307525)', diff saved to https://phabricator.wikimedia.org/P27542 and previous config saved to /var/cache/conftool/dbconfig/20220504-213916-ladsgroup.json [production]
21:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
21:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
21:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1131 (T307525)', diff saved to https://phabricator.wikimedia.org/P27541 and previous config saved to /var/cache/conftool/dbconfig/20220504-213908-ladsgroup.json [production]
21:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P27540 and previous config saved to /var/cache/conftool/dbconfig/20220504-213358-ladsgroup.json [production]
21:27 <cjming> end of UTC late backport & config window [production]
21:25 <cjming@deploy1002> Synchronized php-1.39.0-wmf.10/skins/Vector/resources/skins.vector.styles/components: Backport: [[gerrit:789263|Fix TOC fadeout placement (T306893)]] (duration: 00m 51s) [production]
21:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1131', diff saved to https://phabricator.wikimedia.org/P27539 and previous config saved to /var/cache/conftool/dbconfig/20220504-212401-ladsgroup.json [production]
21:21 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:20 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:20 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T307525)', diff saved to https://phabricator.wikimedia.org/P27538 and previous config saved to /var/cache/conftool/dbconfig/20220504-211853-ladsgroup.json [production]
21:16 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
21:16 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
21:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3317 (T307525)', diff saved to https://phabricator.wikimedia.org/P27537 and previous config saved to /var/cache/conftool/dbconfig/20220504-211607-ladsgroup.json [production]