3001-3050 of 10000 results (65ms)
2022-06-27 ยง
07:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30295 and previous config saved to /var/cache/conftool/dbconfig/20220627-074032-root.json [production]
07:39 <mlitn@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:808124|Update tagline for jvwiki (T311104)]] (duration: 03m 42s) [production]
07:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30294 and previous config saved to /var/cache/conftool/dbconfig/20220627-073938-ladsgroup.json [production]
07:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30293 and previous config saved to /var/cache/conftool/dbconfig/20220627-073929-ladsgroup.json [production]
07:35 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
07:33 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
07:33 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
07:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
07:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30292 and previous config saved to /var/cache/conftool/dbconfig/20220627-072424-ladsgroup.json [production]
07:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1157 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30291 and previous config saved to /var/cache/conftool/dbconfig/20220627-071539-root.json [production]
07:15 <mlitn@deploy1002> Synchronized php-1.39.0-wmf.17/extensions/ImageSuggestions/maintenance/SendNotificationsForUnillustratedWatchedTitles.php: Backport: [[gerrit:808120|Echo tables can live in a different db]] (duration: 03m 45s) [production]
07:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1144 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30290 and previous config saved to /var/cache/conftool/dbconfig/20220627-071506-root.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1109 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30289 and previous config saved to /var/cache/conftool/dbconfig/20220627-071434-root.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1105 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30288 and previous config saved to /var/cache/conftool/dbconfig/20220627-071414-root.json [production]
07:13 <marostegui@cumin1001> dbctl commit (dc=all): 'db1121 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30287 and previous config saved to /var/cache/conftool/dbconfig/20220627-071304-root.json [production]
07:12 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30286 and previous config saved to /var/cache/conftool/dbconfig/20220627-071255-root.json [production]
07:12 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30285 and previous config saved to /var/cache/conftool/dbconfig/20220627-071226-root.json [production]
07:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30284 and previous config saved to /var/cache/conftool/dbconfig/20220627-070919-ladsgroup.json [production]
07:00 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
06:58 <marostegui@cumin1001> dbctl commit (dc=all): 'db1121 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30283 and previous config saved to /var/cache/conftool/dbconfig/20220627-065800-root.json [production]
06:57 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30282 and previous config saved to /var/cache/conftool/dbconfig/20220627-065751-root.json [production]
06:57 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30281 and previous config saved to /var/cache/conftool/dbconfig/20220627-065729-root.json [production]
06:57 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30280 and previous config saved to /var/cache/conftool/dbconfig/20220627-065722-root.json [production]
06:57 <marostegui@cumin1001> dbctl commit (dc=all): 'db1110 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30279 and previous config saved to /var/cache/conftool/dbconfig/20220627-065716-root.json [production]
06:56 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
06:56 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
06:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30278 and previous config saved to /var/cache/conftool/dbconfig/20220627-065414-ladsgroup.json [production]
06:53 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
06:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 8:00:00 on 14 hosts with reason: Maintenance [production]
06:52 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 3 days, 8:00:00 on 14 hosts with reason: Maintenance [production]
06:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 6:00:00 on db2103.codfw.wmnet with reason: Maintenance [production]
06:52 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 6:00:00 on db2103.codfw.wmnet with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on 14 hosts with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on 14 hosts with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2103.codfw.wmnet with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2103.codfw.wmnet with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30277 and previous config saved to /var/cache/conftool/dbconfig/20220627-065009-ladsgroup.json [production]
06:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
06:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1121 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30276 and previous config saved to /var/cache/conftool/dbconfig/20220627-064256-root.json [production]
06:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30275 and previous config saved to /var/cache/conftool/dbconfig/20220627-064247-root.json [production]
06:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30274 and previous config saved to /var/cache/conftool/dbconfig/20220627-064225-root.json [production]
06:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30273 and previous config saved to /var/cache/conftool/dbconfig/20220627-064219-root.json [production]
06:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1110 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30272 and previous config saved to /var/cache/conftool/dbconfig/20220627-064212-root.json [production]
06:27 <marostegui@cumin1001> dbctl commit (dc=all): 'db1121 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30271 and previous config saved to /var/cache/conftool/dbconfig/20220627-062752-root.json [production]
06:27 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30270 and previous config saved to /var/cache/conftool/dbconfig/20220627-062743-root.json [production]
06:27 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3311 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30269 and previous config saved to /var/cache/conftool/dbconfig/20220627-062721-root.json [production]
06:27 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30268 and previous config saved to /var/cache/conftool/dbconfig/20220627-062715-root.json [production]