|
2017-10-26
ยง
|
| 17:16 |
<awight@tin> |
Started deploy [ores/deploy@971be22]: Rolling back scb1002, T175180 T178441 |
[production] |
| 17:14 |
<awight@tin> |
Finished deploy [ores/deploy@971be22]: ORES w/ revscoring 2 and Celery 4, T175180 T178441 (duration: 03m 20s) |
[production] |
| 17:11 |
<awight@tin> |
Started deploy [ores/deploy@971be22]: ORES w/ revscoring 2 and Celery 4, T175180 T178441 |
[production] |
| 17:09 |
<marostegui> |
Optimize ruwiki.recentchanges on dbstore1001 - T162789 |
[production] |
| 17:08 |
<dcausse> |
[logstash] reimporting today logs |
[production] |
| 17:08 |
<dcausse> |
[logstash] deleting today logs to reapply mapping template |
[production] |
| 16:58 |
<ottomata> |
now mirroring main Kafka cluster topics to jumbo Kafka cluster, with MirrorMaker instances running on analytics-eqiad broker nodes. https://phabricator.wikimedia.org/T177216 |
[production] |
| 16:44 |
<awight@tin> |
Finished deploy [ores/deploy@971be22]: Push ORES w/ Celery 4 support to new cluster (try to rebuild venv), T178441 (duration: 01m 02s) |
[production] |
| 16:43 |
<awight@tin> |
Started deploy [ores/deploy@971be22]: Push ORES w/ Celery 4 support to new cluster (try to rebuild venv), T178441 |
[production] |
| 16:33 |
<awight@tin> |
Finished deploy [ores/deploy@971be22]: Push ORES w/ Celery 4 support to new cluster (take 3), T178441 (duration: 15m 33s) |
[production] |
| 16:21 |
<urandom> |
restarted cassandra, restbase2001-b |
[production] |
| 16:18 |
<awight@tin> |
Started deploy [ores/deploy@971be22]: Push ORES w/ Celery 4 support to new cluster (take 3), T178441 |
[production] |
| 16:15 |
<urandom> |
truncating cassandra hints in restbase-ng cluster |
[production] |
| 16:12 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@860cbfe]: Revert all summaries, back to all but WP (duration: 07m 57s) |
[production] |
| 16:04 |
<mobrovac@tin> |
Started deploy [restbase/deploy@860cbfe]: Revert all summaries, back to all but WP |
[production] |
| 16:04 |
<mobrovac@tin> |
Started deploy [restbase/deploy@860cbfe]: Revert all summaries, back to all but WP |
[production] |
| 15:50 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@522321a]: Double-process all summaries (duration: 09m 09s) |
[production] |
| 15:45 |
<urandom> |
restarting cassandra, restbase2001-b |
[production] |
| 15:41 |
<andrewbogott> |
upgrading dnsmasq on labnet1001 and labnet1002 |
[production] |
| 15:41 |
<mobrovac@tin> |
Started deploy [restbase/deploy@522321a]: Double-process all summaries |
[production] |
| 15:36 |
<dcausse> |
reindexing logs in logstash |
[production] |
| 15:34 |
<dcausse> |
deleting and reimporting logs in logstash (today data, might lose some logs in the process) |
[production] |
| 14:10 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1101 - T178383 (duration: 00m 49s) |
[production] |
| 14:04 |
<zeljkof> |
EU SWAT finished |
[production] |
| 14:03 |
<zfilipin@tin> |
Synchronized php-1.31.0-wmf.5/extensions/Wikibase: SWAT: [[gerrit:386548|Make search for titles be always uppercase (T179045)]] (duration: 01m 36s) |
[production] |
| 13:52 |
<zfilipin@tin> |
Synchronized php-1.31.0-wmf.5/extensions/Wikidata: SWAT: [[gerrit:386598|Make search for titles be always uppercase (T179045)]] (duration: 02m 10s) |
[production] |
| 13:46 |
<Amir1> |
ladsgroup@terbium:~$ mwscript extensions/Wikidata/extensions/Wikibase/repo/maintenance/rebuildPropertyInfo.php --wiki=wikidatawiki --rebuild-all --force (T178180) |
[production] |
| 13:41 |
<godog> |
delete old CF cassandra metrics - T173436 |
[production] |
| 13:32 |
<godog> |
force delete old graphite cassandra metrics - T179057 |
[production] |
| 13:28 |
<moritzm> |
installing icu security update on trusty |
[production] |
| 13:28 |
<zfilipin@tin> |
Synchronized wmf-config/Wikibase.php: SWAT: [[gerrit:386594|Add property for RDF mapping of external identifiers for Wikidata (T178180)]] (duration: 00m 50s) |
[production] |
| 13:21 |
<marostegui> |
Compress InnoDB on db2040 - T178359 |
[production] |
| 13:21 |
<zfilipin@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:386567|Enable signature button in Projekt namespace on sewikimedia (T175363)]] (duration: 00m 50s) |
[production] |
| 13:13 |
<zfilipin@tin> |
Synchronized wmf-config/InterwikiSortOrders.php: SWAT: [[gerrit:386324|Fix interwiki sort order for Northern Sami (T178965)]] (duration: 00m 50s) |
[production] |
| 12:57 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Reorganize future rc multi-instance hosts - T178359 (duration: 00m 50s) |
[production] |
| 12:30 |
<bblack> |
powercycle cp4024 (depooled) |
[production] |
| 12:29 |
<bblack@neodymium> |
conftool action : set/pooled=no; selector: name=cp4024.ulsfo.wmnet |
[production] |
| 10:36 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Clean up some comments in s3 (duration: 00m 50s) |
[production] |
| 09:01 |
<moritzm> |
upgrade script runners to wikidiff2 1.5.1 |
[production] |
| 08:58 |
<ema> |
cache_text/upload varnish-be: set rutime parameter timeout_idle to 120s |
[production] |
| 08:34 |
<moritzm> |
uploaded php-wikidiff2/hhvm-wikidiff2 1.5.1 for stretch-wikimedia |
[production] |
| 08:22 |
<godog> |
add 100G to graphite1003's /var/lib/carbon |
[production] |
| 08:14 |
<moritzm> |
upgrade deployment servers to wikidiff2 1.5.1 |
[production] |
| 07:57 |
<marostegui> |
Drop databases in s1 and s2 from db1047 and unconfigure replication - T177405 |
[production] |
| 07:43 |
<marostegui> |
Stop MySQL on db1047 to copy data over db1108 - T177405 T156844 |
[production] |
| 07:28 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1077 - T164488 (duration: 00m 50s) |
[production] |
| 07:13 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1077 - T164488 (duration: 00m 50s) |
[production] |
| 07:12 |
<marostegui> |
Stop replication in sync on db1103 and db1077 to fix data drifts - T164488 |
[production] |
| 07:10 |
<moritzm> |
upgrade mw1319-mw1328 to wikidiff2 1.5.1 |
[production] |
| 06:57 |
<marostegui> |
Stop MySQL on db2088 and db2084 to copy s2 and s4 to db2091 - T178359 |
[production] |