| 2015-12-10
      
      ยง | 
    
  | 21:56 | <gwicke> | restbase: finished full deploy of 7398850fe9 to restbase cluster | [production] | 
            
  | 21:49 | <gwicke> | restbase: starting full deploy of 7398850fe9 to restbase cluster | [production] | 
            
  | 21:49 | <catrope@tin> | Synchronized wmf-config/: SWAT: VisualEditor config patches (duration: 00m 30s) | [production] | 
            
  | 21:46 | <gwicke> | restbase: canary deploy of 7398850fe9 to restbase1001 | [production] | 
            
  | 21:44 | <catrope@tin> | Synchronized php-1.27.0-wmf.8/extensions/Gadgets/: Fix MediaWiki:MediaWiki: (duration: 00m 29s) | [production] | 
            
  | 21:35 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: New password policy for staff group (duration: 00m 28s) | [production] | 
            
  | 21:28 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: SWAT: Echo and Flow config patches (duration: 00m 29s) | [production] | 
            
  | 21:27 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Echo and Flow config patches (duration: 00m 29s) | [production] | 
            
  | 21:20 | <Reedy> | Changed email for mbeattie on otrs_wikiwiki per request of mdennis | [production] | 
            
  | 20:32 | <subbu> | finished parsoid deploy | [production] | 
            
  | 20:14 | <subbu> | restarted parsoid on wtp1003 as canary | [production] | 
            
  | 20:10 | <subbu> | starting parsoid deploy | [production] | 
            
  | 19:11 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.8 | [production] | 
            
  | 18:40 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool es1014 at 100% load and reduce es1019 load (duration: 00m 29s) | [production] | 
            
  | 18:33 | <paravoid> | reprepro: updating cassandra to latest upstream (2.1.12) | [production] | 
            
  | 17:25 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool es1011 at 100% load, repool es1014 with low load (duration: 00m 29s) | [production] | 
            
  | 17:06 | <bd808> | Updated scholarships.wikimedia.org to a4e3dbf (Ensure that Auth\UserData is constructed with an array) | [production] | 
            
  | 16:45 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.7/extensions/MobileApp/config/config.json: SWAT: Roll out RESTBase usage to Android Beta app: 30% [[gerrit:258159]] (duration: 00m 29s) | [production] | 
            
  | 16:42 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.8/extensions/MobileApp/config/config.json: SWAT: Roll out RESTBase usage to Android Beta app: 30% [[gerrit:258161]] (duration: 00m 29s) | [production] | 
            
  | 16:32 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Add three new groups to pawiki, and allow sysops to add or remove users to them [[gerrit:257881]] (duration: 00m 29s) | [production] | 
            
  | 16:26 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Add a rollbacker group at wuuwiki [[gerrit:253084]] (duration: 00m 28s) | [production] | 
            
  | 16:19 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable filemover group at ukwiki [[gerrit:255422]] (duration: 00m 29s) | [production] | 
            
  | 16:14 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Nuke and unblockself only for bureaucrats on en.wikiversity [[gerrit:254029]] and Add new group "curator" to enwikiversity [[gerrit:252012]] (duration: 00m 36s) | [production] | 
            
  | 16:00 | <jynus> | restarting, upgrading and configuring es1014 | [production] | 
            
  | 15:44 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Switchover es3 master es1014 -> es1019 and depool es1014 (duration: 00m 31s) | [production] | 
            
  | 15:44 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Switchover es3 master es1014 -> es1019 and depool es1014 (duration: 00m 28s) | [production] | 
            
  | 15:22 | <jynus> | perming es3 master switchover of es1014 -> es1019, no production impact is expected | [production] | 
            
  | 15:13 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool es1011 with lower weight after mantenance (duration: 00m 29s) | [production] | 
            
  | 13:57 | <jynus> | restarting, upgrading and reconfiguring mysql @ es1011 | [production] | 
            
  | 13:45 | <moritzm> | stopping/starting slapd on seaborgium to update LDAP indices | [production] | 
            
  | 13:39 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Switchover codfw es2 master es1011 -> es1015 (duration: 00m 29s) | [production] | 
            
  | 13:39 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Switchover es2 master es1011 -> es1015 (duration: 00m 37s) | [production] | 
            
  | 13:30 | <paravoid> | reenabled puppet on seaborgium & forced a puppet run | [production] | 
            
  | 13:05 | <moritzm> | restarted slapd on serpens | [production] | 
            
  | 13:03 | <moritzm> | stopping slapd on serpens to update LDAP indices | [production] | 
            
  | 12:54 | <moritzm> | re-enabled puppet on serpens / restarted slapd | [production] | 
            
  | 12:53 | <jynus> | performing switchover of es2 es1011-> es1015 for master maintenance, no production impact is expected | [production] | 
            
  | 12:13 | <godog> | powercycle mw1119, login on console sluggish and no ssh | [production] | 
            
  | 12:11 | <jynus> | codfw es2 server restart finished | [production] | 
            
  | 09:04 | <jynus> | rebooting, restarting and upgrading mysql on es2 (codfw external storage servers) | [production] | 
            
  | 03:55 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Dec 10 03:55:07 UTC 2015 (duration 54m 36s) | [production] | 
            
  | 03:00 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.8) (duration: 15m 54s) | [production] | 
            
  | 02:28 | <gwicke> | updated restbase1008 to cassandra 2.1.12 | [production] | 
            
  | 02:27 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.7) (duration: 10m 53s) | [production] | 
            
  | 01:25 | <ebernhardson@tin> | Synchronized php-1.27.0-wmf.7/extensions/CirrusSearch/: Add master timeout parameter to mapping updates so it can be increased when the master is slow (duration: 00m 30s) | [production] | 
            
  | 01:23 | <ebernhardson@tin> | Synchronized php-1.27.0-wmf.8/extensions/CirrusSearch/: Add master timeout parameter to mapping updates so it can be increased when the master is slow (duration: 00m 29s) | [production] | 
            
  | 01:20 | <ebernhardson@tin> | Synchronized wmf-config/CirrusSearch-production.php: Increase elasticsearch master timeout for maint actions to 2 minutes (duration: 00m 31s) | [production] | 
            
  | 01:05 | <catrope@tin> | Synchronized php-1.27.0-wmf.8/extensions/WikimediaMaintenance/: Fix refreshMessageBlobs script (duration: 00m 29s) | [production] | 
            
  | 01:01 | <catrope@tin> | Synchronized php-1.27.0-wmf.8/extensions/WikimediaMaintenance/: Fix refreshMessageBlobs script (duration: 00m 29s) | [production] | 
            
  | 00:54 | <catrope@tin> | Synchronized php-1.27.0-wmf.8/: MessageBlobStore changes (duration: 02m 29s) | [production] |