3051-3100 of 10000 results (53ms)
2022-06-27 ยง
08:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30330 and previous config saved to /var/cache/conftool/dbconfig/20220627-084138-root.json [production]
08:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30329 and previous config saved to /var/cache/conftool/dbconfig/20220627-084131-root.json [production]
08:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30328 and previous config saved to /var/cache/conftool/dbconfig/20220627-084115-root.json [production]
08:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30327 and previous config saved to /var/cache/conftool/dbconfig/20220627-084109-root.json [production]
08:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30326 and previous config saved to /var/cache/conftool/dbconfig/20220627-084100-root.json [production]
08:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30325 and previous config saved to /var/cache/conftool/dbconfig/20220627-084048-root.json [production]
08:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30324 and previous config saved to /var/cache/conftool/dbconfig/20220627-083314-ladsgroup.json [production]
08:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30323 and previous config saved to /var/cache/conftool/dbconfig/20220627-082907-ladsgroup.json [production]
08:29 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
08:29 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
08:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30322 and previous config saved to /var/cache/conftool/dbconfig/20220627-082859-ladsgroup.json [production]
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30321 and previous config saved to /var/cache/conftool/dbconfig/20220627-082634-root.json [production]
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30320 and previous config saved to /var/cache/conftool/dbconfig/20220627-082627-root.json [production]
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30319 and previous config saved to /var/cache/conftool/dbconfig/20220627-082611-root.json [production]
08:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30318 and previous config saved to /var/cache/conftool/dbconfig/20220627-082605-root.json [production]
08:25 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30317 and previous config saved to /var/cache/conftool/dbconfig/20220627-082556-root.json [production]
08:25 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30316 and previous config saved to /var/cache/conftool/dbconfig/20220627-082544-root.json [production]
08:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30315 and previous config saved to /var/cache/conftool/dbconfig/20220627-081353-ladsgroup.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30314 and previous config saved to /var/cache/conftool/dbconfig/20220627-081130-root.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30313 and previous config saved to /var/cache/conftool/dbconfig/20220627-081123-root.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30312 and previous config saved to /var/cache/conftool/dbconfig/20220627-081107-root.json [production]
08:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30311 and previous config saved to /var/cache/conftool/dbconfig/20220627-081101-root.json [production]
08:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30310 and previous config saved to /var/cache/conftool/dbconfig/20220627-081052-root.json [production]
08:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30309 and previous config saved to /var/cache/conftool/dbconfig/20220627-081040-root.json [production]
07:59 <moritzm> installing openssl security updates [production]
07:59 <matthiasmullie> UTC morning backport done [production]
07:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30308 and previous config saved to /var/cache/conftool/dbconfig/20220627-075848-ladsgroup.json [production]
07:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30307 and previous config saved to /var/cache/conftool/dbconfig/20220627-075626-root.json [production]
07:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30306 and previous config saved to /var/cache/conftool/dbconfig/20220627-075619-root.json [production]
07:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30305 and previous config saved to /var/cache/conftool/dbconfig/20220627-075604-root.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30304 and previous config saved to /var/cache/conftool/dbconfig/20220627-075557-root.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30303 and previous config saved to /var/cache/conftool/dbconfig/20220627-075548-root.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30302 and previous config saved to /var/cache/conftool/dbconfig/20220627-075536-root.json [production]
07:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30301 and previous config saved to /var/cache/conftool/dbconfig/20220627-074343-ladsgroup.json [production]
07:43 <mlitn@deploy1002> Synchronized static/images/mobile/copyright/wikipedia-tagline-jv.svg: Config: [[gerrit:808124|Update tagline for jvwiki (T311104)]] (duration: 03m 33s) [production]
07:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3315 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30300 and previous config saved to /var/cache/conftool/dbconfig/20220627-074122-root.json [production]
07:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1144:3314 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30299 and previous config saved to /var/cache/conftool/dbconfig/20220627-074116-root.json [production]
07:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30298 and previous config saved to /var/cache/conftool/dbconfig/20220627-074100-root.json [production]
07:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30297 and previous config saved to /var/cache/conftool/dbconfig/20220627-074053-root.json [production]
07:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30296 and previous config saved to /var/cache/conftool/dbconfig/20220627-074044-root.json [production]
07:40 <marostegui@cumin1001> dbctl commit (dc=all): 'db1109 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30295 and previous config saved to /var/cache/conftool/dbconfig/20220627-074032-root.json [production]
07:39 <mlitn@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:808124|Update tagline for jvwiki (T311104)]] (duration: 03m 42s) [production]
07:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30294 and previous config saved to /var/cache/conftool/dbconfig/20220627-073938-ladsgroup.json [production]
07:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
07:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T307525)', diff saved to https://phabricator.wikimedia.org/P30293 and previous config saved to /var/cache/conftool/dbconfig/20220627-073929-ladsgroup.json [production]
07:35 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
07:33 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
07:33 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
07:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]