3951-4000 of 10000 results (63ms)
2019-10-18 §
08:02 <ema> depool cp4028 and reimage as text_ats T227432 [production]
07:58 <marostegui> Deploy schema change on db1076 [production]
07:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1076 for schema change', diff saved to https://phabricator.wikimedia.org/P9388 and previous config saved to /var/cache/conftool/dbconfig/20191018-075709-marostegui.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1129 after schema change', diff saved to https://phabricator.wikimedia.org/P9387 and previous config saved to /var/cache/conftool/dbconfig/20191018-075529-marostegui.json [production]
07:21 <moritzm> installing unbound security updates on buster [production]
07:20 <moritzm> installing libdatetime-timezone-perl updates (time zone updates)# [production]
05:53 <vgutierrez> switch cp10184 from nginx to ats-tls - T231433 [production]
05:34 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
05:34 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
05:32 <vgutierrez> switch cp2014 from nginx to ats-tls - T231433 [production]
05:19 <marostegui> Rename m5 labtestwiki database - T233236 [production]
05:15 <marostegui> Deploy schema change on db1129 T233135 T234066 [production]
05:15 <marostegui> Compress tables on db2091:3314 T235599 [production]
05:14 <vgutierrez> switch cp3039 from nginx to ats-tls - T231433 [production]
05:13 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1129 for schema change', diff saved to https://phabricator.wikimedia.org/P9386 and previous config saved to /var/cache/conftool/dbconfig/20191018-051355-marostegui.json [production]
05:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1099:3311 and db2086:3318 after table compression', diff saved to https://phabricator.wikimedia.org/P9385 and previous config saved to /var/cache/conftool/dbconfig/20191018-050831-marostegui.json [production]
04:57 <vgutierrez> switch cp4025 from nginx to ats-tls - T231433 [production]
04:34 <vgutierrez> switch cp5005 from nginx to ats-tls - T231433 [production]
04:31 <vgutierrez> restarting nagios-nrpe-server on stat1007 [production]
2019-10-17 §
21:42 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@d663006]: Update mobileapps to f345673 (duration: 05m 38s) [production]
21:37 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@d663006]: Update mobileapps to f345673 [production]
20:48 <hauskater> Arm eventlogging key via keyholder for beta cluster following the instructions on Wikitech for T235674 [releng]
19:31 <eileen> civicrm revision changed from 4eac801762 to ff69d64ad4, config revision is dc3a88889d [production]
18:26 <mutante> wtp1025 - cd /srv/deployment/parsoid/deploy/src ; sudo -u deploy-service ln -s ../vendor (for benchmarking test) [production]
18:01 <_joe_> depooled wtp1025 from parsoid, parsoid-php to allow running benchmarks there [production]
18:01 <elukey> update librdkafka on eventlog1002 and restart eventlogging [analytics]
18:01 <elukey> update librdkafka on eventlog1002 and restart eventlogging [production]
17:10 <James_F> Zuul: OOUI] Drop generic-node10-rundoc-docker and …-npmaudit-docker experiments [releng]
16:33 <James_F> Zuul: Moving OOUI from node 6 to node 10 job T235570 [releng]
16:29 <jeh> cleanup old fullstackd puppet certs [cloudinfra]
16:11 <James_F> Docker: Pushing node10-test-browser-php72-composer:0.1.1 [releng]
15:27 <jeh> update eqiad1's endpoint catalog with the new wikimediacloud.org domain T223907 [openstack]
15:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1090:3317 and remove db1136 from its temporary vslow,dump role', diff saved to https://phabricator.wikimedia.org/P9382 and previous config saved to /var/cache/conftool/dbconfig/20191017-151952-marostegui.json [production]
15:07 <dcausse> unbanning elastic1050:psi [production]
15:01 <dcausse> dumping jvm heap on elastic1050:psi to investigate gc issues [production]
14:56 <jeh> added icingia downtime for cloudcontrol100[34] and checker.tools.wmflabs.org for service restarts T223907 [openstack]
14:54 <jeh> update eqiad1's hiera keystone_host to new wikimediacloud.org domain T223907 [openstack]
14:46 <moritzm> installing 4.9.189 Linux update on jessie hosts (no reboots, deploying the package only at this point) [production]
14:41 <jeh> deleting failed stresstest VMs that have multiple designate records stresstest1024-16-[16,17,64] left over from newton upgrade T212302 [testlabs]
14:37 <dcausse> banning elastic1050:psi to investigate gc issues [production]
14:32 <moritzm> uploaded linux-meta 1.22 for jessie-wikimedia [production]
14:32 <bblack> disable puppet on cache fleet (cp*) ahead of cert deployment refactoring - T234803 [production]
14:30 <jeh> cleaning up failed nova fullstack vms related to puppet ca T234332 [admin-monitoring]
14:09 <cdanis> ✔️ cdanis@install1002.wikimedia.org ~ 🕙☕ sudo -E reprepro --restrict grafana update buster-wikimedia [production]
13:41 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1129 after PDU maintenance', diff saved to https://phabricator.wikimedia.org/P9381 and previous config saved to /var/cache/conftool/dbconfig/20191017-134112-marostegui.json [production]
13:30 <marostegui@cumin1001> dbctl commit (dc=all): 'More traffic to db1129 after PDU maintenance', diff saved to https://phabricator.wikimedia.org/P9380 and previous config saved to /var/cache/conftool/dbconfig/20191017-133047-marostegui.json [production]
13:06 <XioNoX> rollback failover vrrp from cr2-eqiad to cr1-eqiad - T227133 [production]
12:56 <XioNoX> restart mr1-eqiad [production]
12:54 <XioNoX> downtiming all mgmt host for 30min (mr1-eqiad needs to be rebooted) [production]
12:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2088:3312 for compression T235599', diff saved to https://phabricator.wikimedia.org/P9379 and previous config saved to /var/cache/conftool/dbconfig/20191017-125248-marostegui.json [production]