| 2015-06-17
      
      § | 
    
  | 23:35 | <catrope> | Synchronized php-1.26wmf10/extensions/Gather: SWAT (duration: 00m 14s) | [production] | 
            
  | 23:35 | <gwicke> | rolled back restbase to 90817c2a | [production] | 
            
  | 23:24 | <catrope> | Synchronized php-1.26wmf9/extensions/MobileFrontend: SWAT (duration: 00m 15s) | [production] | 
            
  | 23:23 | <catrope> | Synchronized php-1.26wmf9/extensions/Flow: SWAT (duration: 00m 15s) | [production] | 
            
  | 22:45 | <gwicke> | rolling restart of cassandra nodes | [production] | 
            
  | 22:09 | <gwicke> | rolling restart of restbase instances to apply puppet change after puppet actually ran on all nodes | [production] | 
            
  | 21:58 | <gwicke> | rolling restart of restbase instances to apply config change | [production] | 
            
  | 21:56 | <godog> | restart nutcracker on mw1145 | [production] | 
            
  | 21:35 | <gwicke> | restarting cassandra on restbase1005 | [production] | 
            
  | 20:47 | <mutante> | temp. stopped icinga-wm | [production] | 
            
  | 20:37 | <gwicke> | deployed RESTBase 7ffaf94bfc | [production] | 
            
  | 20:25 | <cscott> | updated Parsoid to version 402ddf66 | [production] | 
            
  | 20:01 | <ottomata> | resized antimony's / LV from 30G to 100G.  looks like /var/lib/git was getting filled up | [production] | 
            
  | 19:43 | <jynus> | rolling schema changes on hewiki | [production] | 
            
  | 19:29 | <godog> | downgrade and restart cassandra to 2.1.3 on restbase1001, metrics not being pushed to graphite with 2.1.6 | [production] | 
            
  | 19:05 | <godog> | bounce cassandra on xenon | [production] | 
            
  | 18:46 | <ori> | Synchronized wmf-config/InitialiseSettings.php: Ic03b152de: Make $wgUploadPath for commons https only for benefit instant commons (duration: 00m 14s) | [production] | 
            
  | 18:11 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.26wmf10 | [production] | 
            
  | 17:45 | <godog> | bounce cassandra on restbase1001 | [production] | 
            
  | 17:39 | <mutante> | repooled mw1234 | [production] | 
            
  | 17:24 | <ottomata> | starting reinstall of Zookeeper analytics nodes (analytics102[345]): https://phabricator.wikimedia.org/T101713 | [production] | 
            
  | 17:16 | <godog> | bounce cassandra on restbase1001 | [production] | 
            
  | 17:14 | <jynus> | rolling schema changes on ruwiki master | [production] | 
            
  | 17:13 | <mutante> | running puppet via salt on api appservers in batches, switch to ganglia_new and carbon | [production] | 
            
  | 17:12 | <godog> | cassandra stopped sending graphite metrics after restart, investigating (test cluster works fine tho) | [production] | 
            
  | 16:58 | <jynus> | rolling schema changes on ruwiki slaves | [production] | 
            
  | 16:28 | <godog> | start upgrading restbase1001 to cassandra 2.1.6 T102015 | [production] | 
            
  | 16:02 | <thcipriani> | Finished scap: Wikitech-Ldap host record roll-out (duration: 24m 35s) | [production] | 
            
  | 15:37 | <thcipriani> | Started scap: Wikitech-Ldap host record roll-out | [production] | 
            
  | 15:19 | <anomie> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Give patrolmarks right to "*" on dewiki [[gerrit:218901]] (duration: 00m 13s) | [production] | 
            
  | 15:17 | <anomie> | Synchronized wmf-config/throttle.php: SWAT: Add a throttle exception for United Islands of Prague [[gerrit:217413]] (duration: 00m 14s) | [production] | 
            
  | 15:15 | <anomie> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Disable captcha on labswiki for now [[gerrit:218908]] (duration: 00m 13s) | [production] | 
            
  | 15:10 | <anomie> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Add extra namespace aliases for Italian Wikipedia [[gerrit:215708]] (duration: 00m 13s) | [production] | 
            
  | 15:08 | <anomie> | SWAT: Enable anti-abuse features on labswiki [[gerrit:218903]] | [production] | 
            
  | 15:08 | <jynus> | testing some schema changes on testwiki | [production] | 
            
  | 15:00 | <aude> | Synchronized usagetracking.dblist: Enable Wikibase usage tracking on nowiki and plwiki (duration: 00m 13s) | [production] | 
            
  | 13:56 | <aude> | Synchronized usagetracking.dblist: Enable Wikibase usage tracking on fiwiki and idwiki (duration: 00m 13s) | [production] | 
            
  | 13:26 | <aude> | Synchronized usagetracking.dblist: Enable Wikibase usage tracking on bgwiki and eowiki (duration: 00m 13s) | [production] | 
            
  | 10:52 | <akosiaris> | reload pybal on lvs1006 | [production] | 
            
  | 10:50 | <mobrovac> | finished deploying mathoid I40ef68 on SCA | [production] | 
            
  | 10:48 | <akosiaris> | repooled mathoid.svc.eqiad.wmnet: sca1002 backend | [production] | 
            
  | 10:44 | <akosiaris> | enable puppet on sca1002 | [production] |