6701-6750 of 10000 results (73ms)
2022-06-27 ยง
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30320 and previous config saved to /var/cache/conftool/dbconfig/20220627-082627-root.json [production]
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30319 and previous config saved to /var/cache/conftool/dbconfig/20220627-082611-root.json [production]
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30318 and previous config saved to /var/cache/conftool/dbconfig/20220627-082605-root.json [production]
08:25 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30317 and previous config saved to /var/cache/conftool/dbconfig/20220627-082556-root.json [production]
08:25 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30316 and previous config saved to /var/cache/conftool/dbconfig/20220627-082544-root.json [production]
08:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30315 and previous config saved to /var/cache/conftool/dbconfig/20220627-081353-ladsgroup.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30314 and previous config saved to /var/cache/conftool/dbconfig/20220627-081130-root.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30313 and previous config saved to /var/cache/conftool/dbconfig/20220627-081123-root.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30312 and previous config saved to /var/cache/conftool/dbconfig/20220627-081107-root.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30311 and previous config saved to /var/cache/conftool/dbconfig/20220627-081101-root.json [production]
08:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30310 and previous config saved to /var/cache/conftool/dbconfig/20220627-081052-root.json [production]
08:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30309 and previous config saved to /var/cache/conftool/dbconfig/20220627-081040-root.json [production]
07:59 <moritzm> installing openssl security updates [production]
07:59 <matthiasmullie> UTC morning backport done [production]
07:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30308 and previous config saved to /var/cache/conftool/dbconfig/20220627-075848-ladsgroup.json [production]
07:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30307 and previous config saved to /var/cache/conftool/dbconfig/20220627-075626-root.json [production]
07:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30306 and previous config saved to /var/cache/conftool/dbconfig/20220627-075619-root.json [production]
07:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30305 and previous config saved to /var/cache/conftool/dbconfig/20220627-075604-root.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30304 and previous config saved to /var/cache/conftool/dbconfig/20220627-075557-root.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30303 and previous config saved to /var/cache/conftool/dbconfig/20220627-075548-root.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30302 and previous config saved to /var/cache/conftool/dbconfig/20220627-075536-root.json [production]
07:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30301 and previous config saved to /var/cache/conftool/dbconfig/20220627-074343-ladsgroup.json [production]
07:43 <mlitn@deploy1002> Synchronized static/images/mobile/copyright/wikipedia-tagline-jv.svg: Config: [[gerrit:808124|Update tagline for jvwiki (T311104)]] (duration: 03m 33s) [production]
07:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30300 and previous config saved to /var/cache/conftool/dbconfig/20220627-074122-root.json [production]
07:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30299 and previous config saved to /var/cache/conftool/dbconfig/20220627-074116-root.json [production]
07:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30298 and previous config saved to /var/cache/conftool/dbconfig/20220627-074100-root.json [production]
07:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30297 and previous config saved to /var/cache/conftool/dbconfig/20220627-074053-root.json [production]
07:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30296 and previous config saved to /var/cache/conftool/dbconfig/20220627-074044-root.json [production]
07:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30295 and previous config saved to /var/cache/conftool/dbconfig/20220627-074032-root.json [production]
07:39 <mlitn@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:808124|Update tagline for jvwiki (T311104)]] (duration: 03m 42s) [production]
07:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30294 and previous config saved to /var/cache/conftool/dbconfig/20220627-073938-ladsgroup.json [production]
07:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30293 and previous config saved to /var/cache/conftool/dbconfig/20220627-073929-ladsgroup.json [production]
07:35 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
07:33 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
07:33 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
07:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
07:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30292 and previous config saved to /var/cache/conftool/dbconfig/20220627-072424-ladsgroup.json [production]
07:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1157 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30291 and previous config saved to /var/cache/conftool/dbconfig/20220627-071539-root.json [production]
07:15 <mlitn@deploy1002> Synchronized php-1.39.0-wmf.17/extensions/ImageSuggestions/maintenance/SendNotificationsForUnillustratedWatchedTitles.php: Backport: [[gerrit:808120|Echo tables can live in a different db]] (duration: 03m 45s) [production]
07:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1144 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30290 and previous config saved to /var/cache/conftool/dbconfig/20220627-071506-root.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1109 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30289 and previous config saved to /var/cache/conftool/dbconfig/20220627-071434-root.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1105 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30288 and previous config saved to /var/cache/conftool/dbconfig/20220627-071414-root.json [production]
07:13 <marostegui@cumin1001> dbctl commit (dc=all): 'db1121 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30287 and previous config saved to /var/cache/conftool/dbconfig/20220627-071304-root.json [production]
07:12 <marostegui@cumin1001> dbctl commit (dc=all): 'db1112 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30286 and previous config saved to /var/cache/conftool/dbconfig/20220627-071255-root.json [production]
07:12 <marostegui@cumin1001> dbctl commit (dc=all): 'db1099:3318 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30285 and previous config saved to /var/cache/conftool/dbconfig/20220627-071226-root.json [production]
07:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30284 and previous config saved to /var/cache/conftool/dbconfig/20220627-070919-ladsgroup.json [production]
07:00 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
06:58 <marostegui@cumin1001> dbctl commit (dc=all): 'db1121 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30283 and previous config saved to /var/cache/conftool/dbconfig/20220627-065800-root.json [production]