|
2020-04-06
§
|
| 11:48 |
<elukey@cumin1001> |
START - Cookbook sre.aqs.roll-restart |
[production] |
| 11:48 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:586325|Create account creator and rollback groups on yowiki (T249487)]] (duration: 00m 59s) |
[production] |
| 11:32 |
<awight@deploy1001> |
Synchronized php-1.35.0-wmf.26/extensions/ContentTranslation: SWAT: [[gerrit:586311|Avoid failure on restoring draft with no categories (T249400)]] (duration: 01m 02s) |
[production] |
| 11:25 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: double-syncing (duration: 00m 58s) |
[production] |
| 11:24 |
<marostegui> |
Deploy schema change on db1105:3311 |
[production] |
| 11:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1105:3311 for schema change', diff saved to https://phabricator.wikimedia.org/P10900 and previous config saved to /var/cache/conftool/dbconfig/20200406-112417-marostegui.json |
[production] |
| 11:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P10899 and previous config saved to /var/cache/conftool/dbconfig/20200406-112123-marostegui.json |
[production] |
| 11:18 |
<elukey> |
import AMD ROCm 3.3 packages in buster-wikimedia (component thirdparty/rocm33) - T247082 |
[production] |
| 11:17 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:580394|cirrus: Increase commonswiki near match weight (T245642)]] (duration: 00m 59s) |
[production] |
| 11:11 |
<awight@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:585779| Whitelist X-Wikimedia-Debug header for cross-wiki API requests (T249107)]] (duration: 00m 59s) |
[production] |
| 10:51 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:586305| Bumping portals to master (563985)]] (duration: 00m 58s) |
[production] |
| 10:50 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:586305| Bumping portals to master (563985)]] (duration: 01m 12s) |
[production] |
| 09:50 |
<XioNoX> |
push pfw firewall policies - T249267 |
[production] |
| 09:40 |
<marostegui> |
Deploy schema change on db1099:3311 |
[production] |
| 09:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1099:3311 for schema change', diff saved to https://phabricator.wikimedia.org/P10898 and previous config saved to /var/cache/conftool/dbconfig/20200406-093944-marostegui.json |
[production] |
| 09:11 |
<ema> |
cp2027: upgrade varnish to 5.1.3-1wm13 and restart varnish-fe T249344 |
[production] |
| 09:08 |
<ema> |
upload varnish 5.1.3-1wm13 to buster-wikimedia on apt1001.wm.org T249344 |
[production] |
| 08:55 |
<ariel@deploy1001> |
Finished deploy [dumps/dumps@ae1e705]: add prefetch test, fix multistream index file download link (duration: 00m 09s) |
[production] |
| 08:55 |
<ariel@deploy1001> |
Started deploy [dumps/dumps@ae1e705]: add prefetch test, fix multistream index file download link |
[production] |
| 08:54 |
<elukey> |
bootstrap wdqs200[7,8] - T246343 |
[production] |
| 08:50 |
<marostegui> |
Deploy schema change on db1139:3311 |
[production] |
| 08:18 |
<_joe_> |
conversion of codfw api done |
[production] |
| 08:07 |
<marostegui> |
Deploy schema change on dbstore1003:3311 |
[production] |
| 07:54 |
<vgutierrez> |
rolling restart of ats-tls to disable wmf-analytics log - T249335 T237993 |
[production] |
| 07:50 |
<dcausse> |
search index: deleting stale index wikidatawiki_content_1585224806 on cloudelastic:9243 |
[production] |
| 07:49 |
<_joe_> |
eqiad API migrated to envoy for local TLS termination, now starting codfw |
[production] |
| 07:35 |
<elukey> |
restart elasticsearch_6@cloudelastic-chi-eqiad on cloudelastic1003 as attempt to fix heavy GC runs (old gen) - T231517 |
[production] |
| 07:35 |
<marostegui> |
Rename wb_terms on eqiad excluding labsdb1009, labdb1010, labsdb1011 - T248086 |
[production] |
| 07:06 |
<marostegui> |
Rename wb_terms on codfw - T248086 |
[production] |
| 06:45 |
<XioNoX> |
delete BGP to AS25074 in amsix |
[production] |
| 06:36 |
<_joe_> |
converting the api servers to envoy for TLS in eqiad |
[production] |
| 06:30 |
<marostegui> |
Upgrade dbproxy1019 - T231520 |
[production] |
| 06:18 |
<marostegui> |
Deploy schema change on s1 codfw master, this will generate lag on codfw |
[production] |
| 05:54 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 05:54 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 05:50 |
<vgutierrez> |
ats-tls restart in cp3056, cp3058 and cp3062 - T249335 |
[production] |
| 05:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1079 after schema change', diff saved to https://phabricator.wikimedia.org/P10897 and previous config saved to /var/cache/conftool/dbconfig/20200406-054559-marostegui.json |
[production] |
| 05:18 |
<marostegui> |
Deploy schema change on db1079 (this will generate lag on s7 labs) |
[production] |
| 05:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1079 for schema change', diff saved to https://phabricator.wikimedia.org/P10896 and previous config saved to /var/cache/conftool/dbconfig/20200406-051744-marostegui.json |
[production] |
| 05:16 |
<vgutierrez> |
Enable inbound TLSv1.3 in upload@eqiad - T170567 |
[production] |
| 05:16 |
<vgutierrez> |
Enable TLS Session Tickets on eqiad - T245616 |
[production] |
| 05:01 |
<vgutierrez> |
ats-tls restart in cp1075, cp1081 and cp1087 - T249335 |
[production] |