|
2016-08-04
ยง
|
| 20:13 |
<urandom> |
T140825,T140869: Performing rolling restart of codfw Cassandra instances |
[production] |
| 20:10 |
<urandom> |
T140825,T140869: Cassandra instance restarts complete: restbase1011.eqiad.wmnet |
[production] |
| 20:07 |
<marxarelli> |
Running jenkins-jobs update config/ 'selenium-*' to deploy https://gerrit.wikimedia.org/r/#/c/302775/ |
[releng] |
| 20:03 |
<urandom> |
T140825,T140869: Performing Cassandra instance rolling restart of restbase1011.eqiad.wmnet |
[production] |
| 19:59 |
<reedy@tin> |
Synchronized wmf-config/extension-list: More to extension.json in extension-list (duration: 00m 52s) |
[production] |
| 19:58 |
<ottomata> |
testing some eventbus kafka failure scenarios in codfw with test.event. (short icinga downtime has been scheduled) |
[production] |
| 19:51 |
<awight> |
paymentswiki config fix |
[production] |
| 19:50 |
<ottomata> |
now running kafka-python 1.2.5 for eventlogging-service-eventbus in codfw, removed downtime for kafka200[12] |
[analytics] |
| 19:47 |
<otto@palladium> |
conftool action : set/pooled=yes; selector: kafka2002.codfw.wmnet |
[production] |
| 19:46 |
<akosiaris> |
restart puppetmaster on palladium to activate loadfactor change. puppet related icinga spam will ensue* |
[production] |
| 19:46 |
<awight> |
enabling paymentswiki queue mirroring |
[production] |
| 19:46 |
<akosiaris> |
restart puppetmaster on palladium to activate loadfactor change. puppet related icinga spam will ensure |
[production] |
| 19:45 |
<otto@palladium> |
conftool action : set/pooled=no; selector: kafka2002.codfw.wmnet |
[production] |
| 19:43 |
<otto@palladium> |
conftool action : set/pooled=yes; selector: kafka2001.codfw.wmnet |
[production] |
| 19:43 |
<mafk> |
Sync. resources/common.css: [[gerrit:302824|Use www.wikimedia.org instead of deprecated bits.wikimedia.org]] |
[tools.stewardbots] |
| 19:42 |
<urandom> |
T140825,T140869: Restarting Cassandra, restbase1010-c.eqiad.wmnet |
[production] |
| 19:40 |
<otto@palladium> |
conftool action : set/pooled=no; selector: kafka2001.codfw.wmnet |
[production] |
| 19:40 |
<urandom> |
T140825,T140869: Restarting Cassandra, restbase1010-b.eqiad.wmnet |
[production] |
| 19:39 |
<ottomata> |
deploying eventlogging-service-eventbus to codfw hosts, depooling and pooling |
[production] |
| 19:37 |
<urandom> |
T140825,T140869: Restarting Cassandra, restbase1010-a.eqiad.wmnet |
[production] |
| 19:27 |
<paladox> |
restarting grrrit-wm bot |
[tools.lolrrit-wm] |
| 19:25 |
<paladox> |
git reset --hard origin/master && git pull origin master |
[tools.lolrrit-wm] |
| 19:09 |
<yuvipanda> |
cleaned up nginx log files in tools-docker-registry-01 to fix free space warning |
[tools] |
| 19:06 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.28.0-wmf.13 |
[production] |
| 18:49 |
<thcipriani@tin> |
Synchronized README: test logstash host (duration: 00m 51s) |
[production] |
| 18:36 |
<cwd> |
updated payments from 3a724bfb1a3e20e17b5886dae0ba7572020abd6b to b737b60c87da82543ab812ece4611c68af01307f |
[production] |
| 18:12 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1074 as backup rc node in case db1036 lags (duration: 00m 25s) |
[production] |
| 18:12 |
<awight> |
update paymentswiki config to 71cc55194f4465600ce4da0ea9f7dfaefdda5479 |
[production] |
| 18:09 |
<bd808> |
Restarted grrrit-wm pod |
[tools.lolrrit-wm] |
| 18:02 |
<halfak> |
rebooting ores-web-05 to address OOM |
[ores] |
| 17:56 |
<halfak> |
reduced web workers per core from 28 to 16 |
[ores] |
| 17:36 |
<elukey> |
added the analytics-deploy key to the Keyholder for the Analytics Refinery scap3 migration (also updated https://wikitech.wikimedia.org/wiki/Keyholder) |
[production] |
| 17:36 |
<elukey> |
added the analytics-deploy key to the Keyholder for the Analytics Refinery scap3 migration (also updated https://wikitech.wikimedia.org/wiki/Keyholder) |
[analytics] |
| 17:28 |
<elukey> |
deploying the refinery with scap3 for the first time on all nodes |
[analytics] |
| 17:18 |
<ostriches> |
gerrit: Restarting really quick, trying alternative mysql library. |
[production] |
| 17:17 |
<urandom> |
T140825,T140869: Restarting Cassandra, restbase1007-c.eqiad.wmnet |
[production] |
| 17:14 |
<urandom> |
T140825,T140869: Restarting Cassandra, restbase1007-b.eqiad.wmnet |
[production] |
| 17:08 |
<urandom> |
T140825,T140869: Restarting Cassandra, restbase1007-a.eqiad.wmnet |
[production] |
| 17:07 |
<elukey> |
restarting keyholder-proxy on tin to let the new analytics key to be picked up |
[production] |
| 17:03 |
<legoktm> |
jstart -N qamorebots /usr/lib/adminbot/adminlogbot.py --config ./confs/qa-logbot.py |
[releng] |
| 17:03 |
<legoktm> |
jstart -N qamorebots /usr/lib/adminbot/adminlogbot.py --config ./confs/qa-logbot.py |
[releng] |
| 16:53 |
<thcipriani> |
restart integration-puppetmaster puppetmaster service. Evidently OOM'd. |
[releng] |
| 15:08 |
<thcipriani@tin> |
Synchronized portals: SWAT: [[gerrit:302921|Bumping portals to master.]] (duration: 00m 37s) |
[production] |
| 15:07 |
<thcipriani@tin> |
Synchronized portals/prod/wikipedia.org/assets: SWAT: [[gerrit:302921|Bumping portals to master.]] (duration: 00m 36s) |
[production] |
| 14:11 |
<godog> |
manually config s1 dbs to scrape on prometheus2001 as a test |
[production] |
| 14:03 |
<paladox> |
cherry picking https://gerrit.wikimedia.org/r/#/c/302918/ to test |
[tools.lolrrit-wm] |
| 13:51 |
<akosiaris> |
delete+accept ganeti1004 salt minion key |
[production] |
| 13:43 |
<akosiaris@palladium> |
conftool action : set/pooled=yes; selector: wtp1012.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=parsoid', 'service=parsoid']) |
[production] |
| 13:43 |
<akosiaris@palladium> |
conftool action : set/pooled=yes; selector: wtp1011.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=parsoid', 'service=parsoid']) |
[production] |
| 13:43 |
<akosiaris@palladium> |
conftool action : set/pooled=yes; selector: wtp1010.eqiad.wmnet (tags: ['dc=eqiad', 'cluster=parsoid', 'service=parsoid']) |
[production] |