|
2015-06-17
ยง
|
| 23:35 |
<gwicke> |
rolled back restbase to 90817c2a |
[production] |
| 23:24 |
<catrope> |
Synchronized php-1.26wmf9/extensions/MobileFrontend: SWAT (duration: 00m 15s) |
[production] |
| 23:23 |
<catrope> |
Synchronized php-1.26wmf9/extensions/Flow: SWAT (duration: 00m 15s) |
[production] |
| 22:45 |
<gwicke> |
rolling restart of cassandra nodes |
[production] |
| 22:09 |
<gwicke> |
rolling restart of restbase instances to apply puppet change after puppet actually ran on all nodes |
[production] |
| 21:58 |
<gwicke> |
rolling restart of restbase instances to apply config change |
[production] |
| 21:56 |
<godog> |
restart nutcracker on mw1145 |
[production] |
| 21:35 |
<gwicke> |
restarting cassandra on restbase1005 |
[production] |
| 20:47 |
<mutante> |
temp. stopped icinga-wm |
[production] |
| 20:37 |
<gwicke> |
deployed RESTBase 7ffaf94bfc |
[production] |
| 20:25 |
<cscott> |
updated Parsoid to version 402ddf66 |
[production] |
| 20:01 |
<ottomata> |
resized antimony's / LV from 30G to 100G. looks like /var/lib/git was getting filled up |
[production] |
| 19:43 |
<jynus> |
rolling schema changes on hewiki |
[production] |
| 19:29 |
<godog> |
downgrade and restart cassandra to 2.1.3 on restbase1001, metrics not being pushed to graphite with 2.1.6 |
[production] |
| 19:05 |
<godog> |
bounce cassandra on xenon |
[production] |
| 18:46 |
<ori> |
Synchronized wmf-config/InitialiseSettings.php: Ic03b152de: Make $wgUploadPath for commons https only for benefit instant commons (duration: 00m 14s) |
[production] |
| 18:11 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.26wmf10 |
[production] |
| 17:45 |
<godog> |
bounce cassandra on restbase1001 |
[production] |
| 17:39 |
<mutante> |
repooled mw1234 |
[production] |
| 17:24 |
<ottomata> |
starting reinstall of Zookeeper analytics nodes (analytics102[345]): https://phabricator.wikimedia.org/T101713 |
[production] |
| 17:16 |
<godog> |
bounce cassandra on restbase1001 |
[production] |
| 17:14 |
<jynus> |
rolling schema changes on ruwiki master |
[production] |
| 17:13 |
<mutante> |
running puppet via salt on api appservers in batches, switch to ganglia_new and carbon |
[production] |
| 17:12 |
<godog> |
cassandra stopped sending graphite metrics after restart, investigating (test cluster works fine tho) |
[production] |
| 16:58 |
<jynus> |
rolling schema changes on ruwiki slaves |
[production] |
| 16:28 |
<godog> |
start upgrading restbase1001 to cassandra 2.1.6 T102015 |
[production] |
| 16:02 |
<thcipriani> |
Finished scap: Wikitech-Ldap host record roll-out (duration: 24m 35s) |
[production] |
| 15:37 |
<thcipriani> |
Started scap: Wikitech-Ldap host record roll-out |
[production] |
| 15:19 |
<anomie> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Give patrolmarks right to "*" on dewiki [[gerrit:218901]] (duration: 00m 13s) |
[production] |
| 15:17 |
<anomie> |
Synchronized wmf-config/throttle.php: SWAT: Add a throttle exception for United Islands of Prague [[gerrit:217413]] (duration: 00m 14s) |
[production] |
| 15:15 |
<anomie> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Disable captcha on labswiki for now [[gerrit:218908]] (duration: 00m 13s) |
[production] |
| 15:10 |
<anomie> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Add extra namespace aliases for Italian Wikipedia [[gerrit:215708]] (duration: 00m 13s) |
[production] |
| 15:08 |
<anomie> |
SWAT: Enable anti-abuse features on labswiki [[gerrit:218903]] |
[production] |
| 15:08 |
<jynus> |
testing some schema changes on testwiki |
[production] |
| 15:00 |
<aude> |
Synchronized usagetracking.dblist: Enable Wikibase usage tracking on nowiki and plwiki (duration: 00m 13s) |
[production] |
| 13:56 |
<aude> |
Synchronized usagetracking.dblist: Enable Wikibase usage tracking on fiwiki and idwiki (duration: 00m 13s) |
[production] |
| 13:26 |
<aude> |
Synchronized usagetracking.dblist: Enable Wikibase usage tracking on bgwiki and eowiki (duration: 00m 13s) |
[production] |
| 10:52 |
<akosiaris> |
reload pybal on lvs1006 |
[production] |
| 10:50 |
<mobrovac> |
finished deploying mathoid I40ef68 on SCA |
[production] |
| 10:48 |
<akosiaris> |
repooled mathoid.svc.eqiad.wmnet: sca1002 backend |
[production] |
| 10:44 |
<akosiaris> |
enable puppet on sca1002 |
[production] |
| 10:43 |
<akosiaris> |
enable puppet |
[production] |
| 10:43 |
<akosiaris> |
depool sca1002 for mathoid.svc.eqiad.wmnet |
[production] |
| 10:43 |
<akosiaris> |
reloaded pybal on lvs1003 |
[production] |
| 10:28 |
<akosiaris> |
repool sca1002, depool sca1001 |
[production] |
| 10:18 |
<mark> |
Halting pvmove of md124 on labstore1001 |
[production] |
| 09:30 |
<akosiaris> |
disable puppet on sca1001 |
[production] |
| 09:09 |
<akosiaris> |
depool sca1001, resource: mathoid |
[production] |
| 09:09 |
<akosiaris> |
puppet disabled on sca1002 |
[production] |
| 08:37 |
<YuviPanda> |
run sudo salt -t 20 -b 100 '*' cmd.run 'sudo service salt-minion restart' on virt1000, attempt to get them to answer on labcontrol1001 instead |
[production] |