| 2016-04-01
      
      § | 
    
  | 21:36 | <mutante> | hooft: stopping ganglia aggregators, remove from icinga/storedconfigclean | [production] | 
            
  | 21:25 | <mutante> | hooft.esams - stop puppet, stop salt | [production] | 
            
  | 20:57 | <mutante> | bast3001 - sign puppet certs, salt keys | [production] | 
            
  | 19:20 | <reedy@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: noop change for labs (duration: 00m 29s) | [production] | 
            
  | 19:10 | <mutante> | booting slauerhoff into PXE, installing as new bast3001 | [production] | 
            
  | 19:10 | <reedy@tin> | Synchronized wmf-config/CommonSettings.php: Replace old copyright image config (duration: 00m 32s) | [production] | 
            
  | 19:09 | <reedy@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: consistency (duration: 00m 28s) | [production] | 
            
  | 17:45 | <bearND> | mobileapps deployed 66f8dac | [production] | 
            
  | 17:12 | <bearND> | starting mobileapps deploy (retry of yesterdays which did not complete) | [production] | 
            
  | 15:37 | <papaul> | installing graphite2002 | [production] | 
            
  | 15:04 | <YuviPanda> | grafana made me and godog admins | [production] | 
            
  | 15:01 | <moritzm> | rebooting copper for upgrade to 4.4 | [production] | 
            
  | 11:06 | <Dereckson> | Imported Jerusalem_Old_City_Walking_to_the_Western_Wall_4K.webm to Commons (T131441) | [production] | 
            
  | 09:14 | <gehel> | testing new Cache-Control headers on mw1017 | [production] | 
            
  | 09:09 | <reedy@tin> | Synchronized wmf-config/throttle.php: Fix my quote fail (duration: 00m 29s) | [production] | 
            
  | 08:42 | <reedy@tin> | rebuilt wikiversions.php and synchronized wikiversions files: Revert labswiki to wmf.18 as 2FA seems to be broken | [production] | 
            
  | 08:38 | <godog> | bootstrap restbase2004-b | [production] | 
            
  | 08:29 | <dereckson@tin> | Synchronized wmf-config/throttle.php: Fix throttle rules (Gerrit change 280819). (duration: 00m 29s) | [production] | 
            
  | 08:11 | <_joe_> | progressively reducing weight of the older servers in the api cluster | [production] | 
            
  | 07:56 | <_joe_> | depooling mw1121-1130 from the api cluster | [production] | 
            
  | 07:15 | <reedy@tin> | Synchronized wmf-config/throttle.php: T130460 Jerusalem Hackathon (duration: 00m 40s) | [production] | 
            
  | 07:00 | <_joe_> | depooling mw1070-89 from the appserver cluster. T126242 | [production] | 
            
  | 06:53 | <_joe_> | setting all newer appservers weight to 20 in eqiad | [production] | 
            
  | 05:28 | <bearND> | mobileapps deployed 66f8dac | [production] | 
            
  | 05:04 | <bearND> | starting mobileapps deploy | [production] | 
            
  | 02:38 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Fri Apr  1 02:38:39 UTC 2016 (duration 8m 58s) | [production] | 
            
  | 02:29 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.19) (duration: 10m 29s) | [production] | 
            
  | 00:55 | <csteipp@tin> | Synchronized wmf-config/CommonSettings-labs.php: Synching labs revert (duration: 00m 31s) | [production] | 
            
  | 00:54 | <csteipp@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: Synching labs revert (duration: 00m 27s) | [production] | 
            
  | 00:37 | <csteipp@tin> | Synchronized wmf-config/CommonSettings-labs.php: Sync labs change to keep repo clean (duration: 00m 31s) | [production] | 
            
  | 00:36 | <csteipp@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 34s) | [production] | 
            
  
    | 2016-03-31
      
      § | 
    
  | 23:48 | <awight@tin> | Finished scap: (no message) (duration: 26m 44s) | [production] | 
            
  | 23:21 | <awight@tin> | Started scap: (no message) | [production] | 
            
  | 23:14 | <awight@tin> | Synchronized php-1.27.0-wmf.19/extensions/OATHAuth: SWAT deployment of OATHAuth fixes, take 2 (duration: 00m 32s) | [production] | 
            
  | 23:13 | <awight@tin> | Synchronized php-1.27.0-wmf.19/extensions/OATHAuth: SWAT deployment of OATHAuth fixes (duration: 00m 46s) | [production] | 
            
  | 22:51 | <Pchelolo> | rolling restart restbase. Apply https://gerrit.wikimedia.org/r/#/c/280711/ config change | [production] | 
            
  | 22:22 | <Pchelolo> | finished update restbase to ba39d2bc | [production] | 
            
  | 22:15 | <Pchelolo> | started update restbase to ba39d2bc | [production] | 
            
  | 22:06 | <Pchelolo> | update restbase to ba39d2bc canary on restbase1005 | [production] | 
            
  | 22:02 | <Pchelolo> | reenable puppet on cerium in restbase staging | [production] | 
            
  | 21:26 | <Pchelolo> | disabling puppet on cerium, updating config and deploying restbase to staging. Testing https://gerrit.wikimedia.org/r/#/c/280711/ | [production] | 
            
  | 21:00 | <mutante> | bast2001 has been reinstalled and can be used again. fingerprints at https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/bast2001.wikimedia.org | [production] | 
            
  | 20:38 | <mutante> | bast2001 - revoking old, signing new puppet certs, salt key.. | [production] | 
            
  | 20:21 | <mutante> | bast2001 - reinstall after disk replacement | [production] | 
            
  | 20:01 | <ottomata> | stopping eventlogging, uninstalling globally installed eventlogging python code, running puppet, restarting eventlogging from /srv/deployment/eventlogging/eventlogging | [production] | 
            
  | 19:59 | <ottomata> | temporarily stopped puppet on eventlog1001 | [production] | 
            
  | 19:12 | <volans> | Starting slave for s2 on db1047 | [production] | 
            
  | 19:04 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.19 | [production] | 
            
  | 17:46 | <mutante> | rcs1002 - stop ferm | [production] | 
            
  | 15:56 | <tgr> | running checkLocalUser.php --delete on some wikis for T119736 | [production] |