2801-2850 of 10000 results (63ms)
2019-10-21 ยง
10:38 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:544853| Bumping portals to master (T128546)]] (duration: 01m 00s) [production]
10:19 <hashar> contint1001 / contint2001 : marking integration/config zuul merger repo readonly: sudo chown -R root:root /srv/zuul/git/integration/config [production]
10:13 <hashar> CI in trouble due to a huge number of changes [production]
10:08 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:06 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:51 <Amir1> maintenance script is done [production]
09:35 <moritzm> removing PHP 7.0 from deployment servers [production]
09:20 <Amir1> start of foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https (T234774) [production]
09:18 <moritzm> installing php7.0 security updates [production]
09:11 <moritzm> installing subversion updates on Stretch (fixes compatibility with security fix for Apache update) [production]
09:07 <moritzm> installing jackson-databind security updates [production]
09:01 <moritzm> installing openjpeg2 security updates [production]
08:52 <godog> roll-restart logstash to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/544209 [production]
08:34 <Urbanecm> Deploy security patch (T234862) [production]
08:34 <vgutierrez> Switch from nginx to ats-tls on cp2022 - T231627 [production]
08:30 <ema> pool cp4029 with ATS backend T227432 [production]
08:20 <vgutierrez> Switch from nginx to ats-tls on cp2020 - T231627 [production]
08:09 <vgutierrez> Switch from nginx to ats-tls on cp2018 - T231627 [production]
08:08 <godog> swift eqiad-prod: add weight to ms-be105[1-6] - T232367 [production]
08:03 <godog> swift codfw-prod: final weight to ms-be205[1-6] - T233638 [production]
07:59 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
07:57 <vgutierrez> Switch from nginx to ats-tls on cp3046 - T231627 [production]
07:57 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:50 <ema@puppetmaster1001> conftool action : set/weight=100; selector: name=cp4029.ulsfo.wmnet,service=ats-be [production]
07:45 <moritzm> installing aspell security updates on jessie [production]
07:43 <vgutierrez> Switch from nginx to ats-tls on cp3045 - T231627 [production]
07:35 <moritzm> installing openjdk-11 security updates [production]
07:32 <ema> depool cp4029 and reimage as text_ats T227432 [production]
07:15 <vgutierrez> Switch from nginx to ats-tls on cp1075 - T231627 [production]
07:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool non partitioned db1089 into s1 special slaves to check for slow queries T223151', diff saved to https://phabricator.wikimedia.org/P9406 and previous config saved to /var/cache/conftool/dbconfig/20191021-070655-marostegui.json [production]
07:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Change weights from 1 to 100 on s1 eqiad - T231018', diff saved to https://phabricator.wikimedia.org/P9405 and previous config saved to /var/cache/conftool/dbconfig/20191021-070352-marostegui.json [production]
07:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Change weights from 1 to 100 on s1 codfw - T231018', diff saved to https://phabricator.wikimedia.org/P9404 and previous config saved to /var/cache/conftool/dbconfig/20191021-070119-marostegui.json [production]
06:59 <vgutierrez> Switch from nginx to ats-tls on cp2001 - T231627 [production]
06:46 <vgutierrez> Switch from nginx to ats-tls on cp3030 - T231627 [production]
06:28 <vgutierrez> Install python3-cryptography-2.6.1-3+deb10u2 on acme-chief hosts - T234131 [production]
06:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1105:3311', diff saved to https://phabricator.wikimedia.org/P9403 and previous config saved to /var/cache/conftool/dbconfig/20191021-061518-marostegui.json [production]
06:12 <vgutierrez> Switch cp1086 from nginx to ats-tls - T231433 [production]
06:10 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
06:10 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
05:58 <marostegui@cumin1001> dbctl commit (dc=all): 'Give weight 100 to db1130 on s5 to check for slow queries T223151', diff saved to https://phabricator.wikimedia.org/P9402 and previous config saved to /var/cache/conftool/dbconfig/20191021-055843-marostegui.json [production]
05:54 <vgutierrez> Switch cp2017 from nginx to ats-tls - T231433 [production]
05:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Give more traffic to db1105:3311', diff saved to https://phabricator.wikimedia.org/P9401 and previous config saved to /var/cache/conftool/dbconfig/20191021-055017-marostegui.json [production]
05:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db2048 and db2061, those hosts will be decommissioned T228258', diff saved to https://phabricator.wikimedia.org/P9400 and previous config saved to /var/cache/conftool/dbconfig/20191021-054340-marostegui.json [production]
05:42 <_joe_> slowly removing service objects from production etcd T233973 [production]
05:38 <vgutierrez> Switch cp3044 from nginx to ats-tls - T231433 [production]
05:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Give more traffic to db1105:3311', diff saved to https://phabricator.wikimedia.org/P9399 and previous config saved to /var/cache/conftool/dbconfig/20191021-053737-marostegui.json [production]
05:30 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
05:30 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
05:28 <marostegui> Compress tables on db2084:3314 db2091:3312 - T235599 [production]
05:26 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1105:3312', diff saved to https://phabricator.wikimedia.org/P9398 and previous config saved to /var/cache/conftool/dbconfig/20191021-052643-marostegui.json [production]