|
2020-08-26
§
|
| 08:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1110 replication broken', diff saved to https://phabricator.wikimedia.org/P12360 and previous config saved to /var/cache/conftool/dbconfig/20200826-084044-marostegui.json |
[production] |
| 08:16 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 08:14 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 05:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1091 for MCR change', diff saved to https://phabricator.wikimedia.org/P12358 and previous config saved to /var/cache/conftool/dbconfig/20200826-054557-marostegui.json |
[production] |
| 05:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1114, db1135 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12357 and previous config saved to /var/cache/conftool/dbconfig/20200826-054409-marostegui.json |
[production] |
| 05:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1114, db1135 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12356 and previous config saved to /var/cache/conftool/dbconfig/20200826-053345-marostegui.json |
[production] |
| 05:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1114, db1135 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12355 and previous config saved to /var/cache/conftool/dbconfig/20200826-052355-marostegui.json |
[production] |
| 05:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1114, db1135 after MCR changes', diff saved to https://phabricator.wikimedia.org/P12354 and previous config saved to /var/cache/conftool/dbconfig/20200826-050849-marostegui.json |
[production] |
| 05:03 |
<marostegui> |
Update db1135 and db1114 after MCR changes |
[production] |
|
2020-08-25
§
|
| 21:51 |
<mutante> |
xhgui1001/xhgui2001 - Unpacking xhgui (0.12.0-2-wmf1) over (0.9.0-1-wmf1) (T260397) |
[production] |
| 21:50 |
<mutante> |
xhgui1001 - Unpacking xhgui (0.12.0-2-wmf1) over (0.9.0-1-wmf1) ... |
[production] |
| 21:46 |
<mutante> |
importing xhgui 0.12.0-2-wmf1 to buster-wikimedia APT repo (T260397) |
[production] |
| 19:40 |
<dcausse@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@125cb6d]: test: Add wikidata ttl import (duration: 00m 54s) |
[production] |
| 19:39 |
<dcausse@deploy1001> |
Started deploy [wikimedia/discovery/analytics@125cb6d]: test: Add wikidata ttl import |
[production] |
| 19:15 |
<marxarelli> |
1.36.0-wmf.6 promoted to group0 (T257974). no new errors |
[production] |
| 19:09 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.6 |
[production] |
| 19:05 |
<moritzm> |
installing Java security updates on cloudelastic* hosts |
[production] |
| 19:02 |
<moritzm> |
installing Java security updates on elastic* hosts |
[production] |
| 18:19 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-reload |
[production] |
| 17:58 |
<dduvall@deploy1001> |
Finished scap: testwikis wikis to 1.36.0-wmf.6 (duration: 41m 58s) |
[production] |
| 17:30 |
<dcausse@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@bc2f7f1]: test: Add wikidata ttl import (duration: 01m 52s) |
[production] |
| 17:28 |
<dcausse@deploy1001> |
Started deploy [wikimedia/discovery/analytics@bc2f7f1]: test: Add wikidata ttl import |
[production] |
| 17:17 |
<dduvall@deploy1001> |
Started scap: testwikis wikis to 1.36.0-wmf.6 |
[production] |
| 17:08 |
<dduvall@deploy1001> |
Pruned MediaWiki: 1.36.0-wmf.4 (duration: 01m 40s) |
[production] |
| 17:01 |
<dduvall@deploy1001> |
Pruned MediaWiki: 1.36.0-wmf.3 (duration: 19m 12s) |
[production] |
| 17:01 |
<herron> |
imported logstash, elasticsearch, and kibana 7.9.0 -oss packages into buster-wikimedia thirdparty/elastic79 |
[production] |
| 16:42 |
<dcausse@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@89b4f74]: test: Add wikidata ttl import (duration: 00m 49s) |
[production] |
| 16:41 |
<dcausse@deploy1001> |
Started deploy [wikimedia/discovery/analytics@89b4f74]: test: Add wikidata ttl import |
[production] |
| 16:21 |
<shdubsh> |
restart logstash on logstash1007 -- gc duration outlier |
[production] |
| 16:08 |
<dcausse@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@ae6dd8d]: test: Add wikidata ttl import (duration: 00m 54s) |
[production] |
| 16:07 |
<dcausse@deploy1001> |
Started deploy [wikimedia/discovery/analytics@ae6dd8d]: test: Add wikidata ttl import |
[production] |
| 16:00 |
<gehel> |
repool wdqs1005 - catched up on lag |
[production] |
| 15:47 |
<elukey> |
restart mariadb@analytics_meta on db1108 to apply a replication filter (exclude superset_staging database from replication) |
[production] |
| 15:44 |
<jgleeson> |
fundraising-tools updated from dcad0bfe75 to 3fe3a23114 |
[production] |
| 15:41 |
<dcausse@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@cbf2f9d]: Add wikidata ttl import (duration: 01m 38s) |
[production] |
| 15:39 |
<dcausse@deploy1001> |
Started deploy [wikimedia/discovery/analytics@cbf2f9d]: Add wikidata ttl import |
[production] |
| 15:22 |
<liw> |
testing upcoming Scap release on beta |
[production] |
| 14:56 |
<moritzm> |
installing rake security updates on stretch |
[production] |
| 14:56 |
<moritzm> |
installing take security updates on stretch |
[production] |
| 14:36 |
<oblivian@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'staging' . |
[production] |
| 14:32 |
<volker-e@deploy1001> |
Finished deploy [design/style-guide@e3fda83]: Deploy design/style-guide: (duration: 00m 05s) |
[production] |
| 14:32 |
<volker-e@deploy1001> |
Started deploy [design/style-guide@e3fda83]: Deploy design/style-guide: |
[production] |
| 14:26 |
<XioNoX> |
disable IPv6 BGP to Init7 in knams |
[production] |
| 14:10 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@7a3221d]: add hostname checking --bug T207538 (duration: 03m 50s) |
[production] |
| 14:06 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@7a3221d]: add hostname checking --bug T207538 |
[production] |
| 13:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1114 for MCR change', diff saved to https://phabricator.wikimedia.org/P12347 and previous config saved to /var/cache/conftool/dbconfig/20200825-135248-marostegui.json |
[production] |
| 13:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'fully repool db1111 MCR changes', diff saved to https://phabricator.wikimedia.org/P12346 and previous config saved to /var/cache/conftool/dbconfig/20200825-134736-marostegui.json |
[production] |
| 13:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1111 MCR changes', diff saved to https://phabricator.wikimedia.org/P12345 and previous config saved to /var/cache/conftool/dbconfig/20200825-133734-marostegui.json |
[production] |
| 13:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1111 MCR changes', diff saved to https://phabricator.wikimedia.org/P12344 and previous config saved to /var/cache/conftool/dbconfig/20200825-132027-marostegui.json |
[production] |
| 13:17 |
<moritzm> |
installing firejail security updates on remaining mw* servers in eqiad |
[production] |