|
2020-01-17
§
|
| 20:07 |
<urandom> |
bootstrapping restbase2023-b — T243000 |
[production] |
| 20:01 |
<bblack> |
reset bgp peerings with gfiber on cr2-eqiad |
[production] |
| 19:14 |
<mutante> |
gerrit - switching operations/debs/hhvm to READONLY mode and adding ARCHIVED to description (T237038) |
[production] |
| 18:19 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) |
[production] |
| 18:18 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 17:14 |
<urandom> |
bootstrapping restbase2023-a — T243000 |
[production] |
| 16:33 |
<marostegui> |
Stop replication on db1107 |
[production] |
| 16:25 |
<ebernhardson@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@938d253]: Move weekly elasticsearch transfer to airflow (duration: 00m 21s) |
[production] |
| 16:25 |
<ebernhardson@deploy1001> |
Started deploy [wikimedia/discovery/analytics@938d253]: Move weekly elasticsearch transfer to airflow |
[production] |
| 14:31 |
<urandom> |
bootstrapping restbase2022-c — T243000 |
[production] |
| 14:09 |
<awight@deploy1001> |
Synchronized php-1.35.0-wmf.15/extensions/Cite: UBN backport: [[gerrit:565562|Fix for nested #tag:references and empty name (T242437)]] (duration: 00m 57s) |
[production] |
| 14:03 |
<awight> |
beginning Friday deployment for UBN, T242437 |
[production] |
| 13:38 |
<moritzm> |
masking squid3 on old URL downloaders T224551 |
[production] |
| 13:35 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:35 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:35 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:35 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:35 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:35 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:55 |
<effie> |
Updgrade netmon* to to php 7.2.26 and restart - T241222 |
[production] |
| 11:48 |
<moritzm> |
upgrading PHP 7.2 on netmon* (also apache restart for SSL update) |
[production] |
| 11:13 |
<elukey> |
restart nginx on analitycs tool hosts to pick up openssl updates |
[production] |
| 11:05 |
<moritzm> |
restarting apache on matomo1001 to pick up SSL updates |
[production] |
| 11:04 |
<XioNoX> |
Running homer to remove decom cloud vlans in eqiad/codfw - T240670 |
[production] |
| 11:01 |
<XioNoX> |
delete vlan cloud-instances1-b-eqiad from asw2-b-eqiad - T240670 |
[production] |
| 10:43 |
<moritzm> |
restarting apache on miscweb* to pick up SSL updates |
[production] |
| 10:39 |
<moritzm> |
restarting apache on puppetboard* to pick up SSL updates |
[production] |
| 10:32 |
<moritzm> |
installing remaining OpenSSL 1.0.2 updates |
[production] |
| 09:25 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:25 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:00 |
<Dog29> |
Finished installing ATS |
[production] |
| 09:00 |
<Dog29> |
Installing ATS |
[production] |
| 08:58 |
<Dog29> |
tools.stashbot Ended deploy of Parsoid |
[production] |
| 08:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db2103', diff saved to https://phabricator.wikimedia.org/P10202 and previous config saved to /var/cache/conftool/dbconfig/20200117-085808-marostegui.json |
[production] |
| 08:57 |
<Dog29> |
tools.stashbot Starting deploy of Parsoid |
[production] |
| 07:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1082', diff saved to https://phabricator.wikimedia.org/P10201 and previous config saved to /var/cache/conftool/dbconfig/20200117-075125-marostegui.json |
[production] |
| 07:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1082', diff saved to https://phabricator.wikimedia.org/P10200 and previous config saved to /var/cache/conftool/dbconfig/20200117-074626-marostegui.json |
[production] |
| 07:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1081', diff saved to https://phabricator.wikimedia.org/P10199 and previous config saved to /var/cache/conftool/dbconfig/20200117-073954-marostegui.json |
[production] |
| 07:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1082', diff saved to https://phabricator.wikimedia.org/P10198 and previous config saved to /var/cache/conftool/dbconfig/20200117-073917-marostegui.json |
[production] |
| 07:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1082', diff saved to https://phabricator.wikimedia.org/P10197 and previous config saved to /var/cache/conftool/dbconfig/20200117-072544-marostegui.json |
[production] |
| 07:10 |
<marostegui> |
Stop and upgrade db1082 |
[production] |
| 07:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es2019', diff saved to https://phabricator.wikimedia.org/P10193 and previous config saved to /var/cache/conftool/dbconfig/20200117-070636-marostegui.json |
[production] |
| 07:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool es2012', diff saved to https://phabricator.wikimedia.org/P10192 and previous config saved to /var/cache/conftool/dbconfig/20200117-070516-marostegui.json |
[production] |
| 07:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es2012', diff saved to https://phabricator.wikimedia.org/P10191 and previous config saved to /var/cache/conftool/dbconfig/20200117-070320-marostegui.json |
[production] |
| 06:35 |
<marostegui> |
Compress db1125:3314 tables - this will create lag on s4 labs hosts |
[production] |
| 06:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1081', diff saved to https://phabricator.wikimedia.org/P10190 and previous config saved to /var/cache/conftool/dbconfig/20200117-062838-marostegui.json |
[production] |
| 06:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1081', diff saved to https://phabricator.wikimedia.org/P10189 and previous config saved to /var/cache/conftool/dbconfig/20200117-061602-marostegui.json |
[production] |
| 06:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1081', diff saved to https://phabricator.wikimedia.org/P10188 and previous config saved to /var/cache/conftool/dbconfig/20200117-060259-marostegui.json |
[production] |
| 02:45 |
<urandom> |
bootstrapping restbase2022-b — T243000 |
[production] |
| 00:45 |
<mutante> |
urldownloaders - rm /etc/logrotate.d/squid3 ; systemctl start logrotate (this fixes failed logrotate because of squid3 vs squid file = duplicate entry, but puppet will recreate it) |
[production] |