3101-3150 of 10000 results (68ms)
2022-06-27 ยง
10:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30380 and previous config saved to /var/cache/conftool/dbconfig/20220627-104653-ladsgroup.json [production]
10:45 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30379 and previous config saved to /var/cache/conftool/dbconfig/20220627-104555-root.json [production]
10:45 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30378 and previous config saved to /var/cache/conftool/dbconfig/20220627-104543-root.json [production]
10:39 <hashar> Restarting CI Jenkins [production]
10:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30377 and previous config saved to /var/cache/conftool/dbconfig/20220627-103229-root.json [production]
10:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30376 and previous config saved to /var/cache/conftool/dbconfig/20220627-103224-root.json [production]
10:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30375 and previous config saved to /var/cache/conftool/dbconfig/20220627-103218-root.json [production]
10:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30374 and previous config saved to /var/cache/conftool/dbconfig/20220627-103148-ladsgroup.json [production]
10:30 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30373 and previous config saved to /var/cache/conftool/dbconfig/20220627-103051-root.json [production]
10:30 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30372 and previous config saved to /var/cache/conftool/dbconfig/20220627-103039-root.json [production]
10:17 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30371 and previous config saved to /var/cache/conftool/dbconfig/20220627-101725-root.json [production]
10:17 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30370 and previous config saved to /var/cache/conftool/dbconfig/20220627-101720-root.json [production]
10:17 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30369 and previous config saved to /var/cache/conftool/dbconfig/20220627-101714-root.json [production]
10:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30368 and previous config saved to /var/cache/conftool/dbconfig/20220627-101643-ladsgroup.json [production]
10:15 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30367 and previous config saved to /var/cache/conftool/dbconfig/20220627-101547-root.json [production]
10:15 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30366 and previous config saved to /var/cache/conftool/dbconfig/20220627-101535-root.json [production]
10:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30365 and previous config saved to /var/cache/conftool/dbconfig/20220627-101235-ladsgroup.json [production]
10:12 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
10:12 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
10:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30364 and previous config saved to /var/cache/conftool/dbconfig/20220627-101226-ladsgroup.json [production]
10:02 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30363 and previous config saved to /var/cache/conftool/dbconfig/20220627-100221-root.json [production]
10:02 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30362 and previous config saved to /var/cache/conftool/dbconfig/20220627-100216-root.json [production]
10:02 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30361 and previous config saved to /var/cache/conftool/dbconfig/20220627-100211-root.json [production]
10:00 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30360 and previous config saved to /var/cache/conftool/dbconfig/20220627-100043-root.json [production]
10:00 <jgiannelos@deploy1002> helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: sync [production]
10:00 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30359 and previous config saved to /var/cache/conftool/dbconfig/20220627-100031-root.json [production]
09:57 <elukey> copy cassandra and cassandra-tools packages in component/cassandra{311,dev} from wikimedia buster to bullseye - T310980 [production]
09:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30358 and previous config saved to /var/cache/conftool/dbconfig/20220627-095721-ladsgroup.json [production]
09:54 <jgiannelos@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: sync [production]
09:54 <jgiannelos@deploy1002> helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: apply [production]
09:54 <jgiannelos@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: apply [production]
09:53 <jgiannelos@deploy1002> helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: apply [production]
09:53 <jgiannelos@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: apply [production]
09:47 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30357 and previous config saved to /var/cache/conftool/dbconfig/20220627-094718-root.json [production]
09:47 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30356 and previous config saved to /var/cache/conftool/dbconfig/20220627-094712-root.json [production]
09:47 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30355 and previous config saved to /var/cache/conftool/dbconfig/20220627-094707-root.json [production]
09:45 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 5%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30354 and previous config saved to /var/cache/conftool/dbconfig/20220627-094539-root.json [production]
09:45 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30353 and previous config saved to /var/cache/conftool/dbconfig/20220627-094527-root.json [production]
09:43 <jgiannelos@deploy1002> helmfile [staging] START helmfile.d/services/tegola-vector-tiles: apply [production]
09:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173', diff saved to https://phabricator.wikimedia.org/P30352 and previous config saved to /var/cache/conftool/dbconfig/20220627-094216-ladsgroup.json [production]
09:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1157 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30351 and previous config saved to /var/cache/conftool/dbconfig/20220627-093214-root.json [production]
09:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30350 and previous config saved to /var/cache/conftool/dbconfig/20220627-093208-root.json [production]
09:32 <marostegui@cumin1001> dbctl commit (dc=all): 'db1106 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30349 and previous config saved to /var/cache/conftool/dbconfig/20220627-093203-root.json [production]
09:30 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3314 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30348 and previous config saved to /var/cache/conftool/dbconfig/20220627-093035-root.json [production]
09:30 <marostegui@cumin1001> dbctl commit (dc=all): 'db1146:3312 (re)pooling @ 2%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30347 and previous config saved to /var/cache/conftool/dbconfig/20220627-093023-root.json [production]
09:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30346 and previous config saved to /var/cache/conftool/dbconfig/20220627-092710-ladsgroup.json [production]
09:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1173 (T298565)', diff saved to https://phabricator.wikimedia.org/P30345 and previous config saved to /var/cache/conftool/dbconfig/20220627-092256-ladsgroup.json [production]
09:22 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
09:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1173.eqiad.wmnet with reason: Maintenance [production]
09:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1106 db1146 db1156 db1157 db1161 for kernel reboots', diff saved to https://phabricator.wikimedia.org/P30344 and previous config saved to /var/cache/conftool/dbconfig/20220627-092154-root.json [production]