|
2020-04-06
ยง
|
| 12:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1089 after schema change', diff saved to https://phabricator.wikimedia.org/P10903 and previous config saved to /var/cache/conftool/dbconfig/20200406-125222-marostegui.json |
[production] |
| 12:46 |
<urbanecm@deploy1001> |
rebuilt and synchronized wikiversions files: 77b9ae9: Create grwikimedia |
[production] |
| 12:44 |
<urbanecm@deploy1001> |
Synchronized dblists/: 77b9ae9: Create grwikimedia (duration: 00m 59s) |
[production] |
| 12:37 |
<XioNoX> |
Update eqiad analytics filters with new APT IPs |
[production] |
| 12:27 |
<hnowlan@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . |
[production] |
| 12:21 |
<marostegui> |
Deploy schema change on db1089 |
[production] |
| 12:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1089 for schema change', diff saved to https://phabricator.wikimedia.org/P10902 and previous config saved to /var/cache/conftool/dbconfig/20200406-122123-marostegui.json |
[production] |
| 12:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1105:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P10901 and previous config saved to /var/cache/conftool/dbconfig/20200406-122058-marostegui.json |
[production] |
| 12:14 |
<hnowlan@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . |
[production] |
| 12:08 |
<hnowlan@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . |
[production] |
| 12:04 |
<godog> |
test grafana 6.7.2 upgrade on grafana2001 - T244208 |
[production] |
| 11:57 |
<awight> |
EU swat complete |
[production] |
| 11:53 |
<awight@deploy1001> |
Synchronized php-1.35.0-wmf.26/extensions/TwoColConflict: SWAT: [[gerrit:586309|Backport talk page and EventLogging changes (T248243, T249404) (duration: 00m 59s) |
[production] |
| 11:52 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) |
[production] |
| 11:48 |
<elukey@cumin1001> |
START - Cookbook sre.aqs.roll-restart |
[production] |
| 11:48 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:586325|Create account creator and rollback groups on yowiki (T249487)]] (duration: 00m 59s) |
[production] |
| 11:32 |
<awight@deploy1001> |
Synchronized php-1.35.0-wmf.26/extensions/ContentTranslation: SWAT: [[gerrit:586311|Avoid failure on restoring draft with no categories (T249400)]] (duration: 01m 02s) |
[production] |
| 11:25 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: double-syncing (duration: 00m 58s) |
[production] |
| 11:24 |
<marostegui> |
Deploy schema change on db1105:3311 |
[production] |
| 11:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1105:3311 for schema change', diff saved to https://phabricator.wikimedia.org/P10900 and previous config saved to /var/cache/conftool/dbconfig/20200406-112417-marostegui.json |
[production] |
| 11:21 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P10899 and previous config saved to /var/cache/conftool/dbconfig/20200406-112123-marostegui.json |
[production] |
| 11:18 |
<elukey> |
import AMD ROCm 3.3 packages in buster-wikimedia (component thirdparty/rocm33) - T247082 |
[production] |
| 11:17 |
<awight@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:580394|cirrus: Increase commonswiki near match weight (T245642)]] (duration: 00m 59s) |
[production] |
| 11:11 |
<awight@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:585779| Whitelist X-Wikimedia-Debug header for cross-wiki API requests (T249107)]] (duration: 00m 59s) |
[production] |
| 10:51 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:586305| Bumping portals to master (563985)]] (duration: 00m 58s) |
[production] |
| 10:50 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:586305| Bumping portals to master (563985)]] (duration: 01m 12s) |
[production] |
| 09:50 |
<XioNoX> |
push pfw firewall policies - T249267 |
[production] |
| 09:40 |
<marostegui> |
Deploy schema change on db1099:3311 |
[production] |
| 09:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1099:3311 for schema change', diff saved to https://phabricator.wikimedia.org/P10898 and previous config saved to /var/cache/conftool/dbconfig/20200406-093944-marostegui.json |
[production] |
| 09:11 |
<ema> |
cp2027: upgrade varnish to 5.1.3-1wm13 and restart varnish-fe T249344 |
[production] |
| 09:08 |
<ema> |
upload varnish 5.1.3-1wm13 to buster-wikimedia on apt1001.wm.org T249344 |
[production] |
| 08:55 |
<ariel@deploy1001> |
Finished deploy [dumps/dumps@ae1e705]: add prefetch test, fix multistream index file download link (duration: 00m 09s) |
[production] |
| 08:55 |
<ariel@deploy1001> |
Started deploy [dumps/dumps@ae1e705]: add prefetch test, fix multistream index file download link |
[production] |
| 08:54 |
<elukey> |
bootstrap wdqs200[7,8] - T246343 |
[production] |
| 08:50 |
<marostegui> |
Deploy schema change on db1139:3311 |
[production] |
| 08:18 |
<_joe_> |
conversion of codfw api done |
[production] |
| 08:07 |
<marostegui> |
Deploy schema change on dbstore1003:3311 |
[production] |
| 07:54 |
<vgutierrez> |
rolling restart of ats-tls to disable wmf-analytics log - T249335 T237993 |
[production] |
| 07:50 |
<dcausse> |
search index: deleting stale index wikidatawiki_content_1585224806 on cloudelastic:9243 |
[production] |
| 07:49 |
<_joe_> |
eqiad API migrated to envoy for local TLS termination, now starting codfw |
[production] |
| 07:35 |
<elukey> |
restart elasticsearch_6@cloudelastic-chi-eqiad on cloudelastic1003 as attempt to fix heavy GC runs (old gen) - T231517 |
[production] |
| 07:35 |
<marostegui> |
Rename wb_terms on eqiad excluding labsdb1009, labdb1010, labsdb1011 - T248086 |
[production] |
| 07:06 |
<marostegui> |
Rename wb_terms on codfw - T248086 |
[production] |
| 06:45 |
<XioNoX> |
delete BGP to AS25074 in amsix |
[production] |
| 06:36 |
<_joe_> |
converting the api servers to envoy for TLS in eqiad |
[production] |
| 06:30 |
<marostegui> |
Upgrade dbproxy1019 - T231520 |
[production] |
| 06:18 |
<marostegui> |
Deploy schema change on s1 codfw master, this will generate lag on codfw |
[production] |
| 05:54 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 05:54 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 05:50 |
<vgutierrez> |
ats-tls restart in cp3056, cp3058 and cp3062 - T249335 |
[production] |