| 
      
        2016-04-01
      
      §
     | 
  
    
  | 17:45 | 
  <bearND> | 
  mobileapps deployed 66f8dac | 
  [production] | 
            
  | 17:12 | 
  <bearND> | 
  starting mobileapps deploy (retry of yesterdays which did not complete) | 
  [production] | 
            
  | 15:37 | 
  <papaul> | 
  installing graphite2002 | 
  [production] | 
            
  | 15:04 | 
  <YuviPanda> | 
  grafana made me and godog admins | 
  [production] | 
            
  | 15:01 | 
  <moritzm> | 
  rebooting copper for upgrade to 4.4 | 
  [production] | 
            
  | 11:06 | 
  <Dereckson> | 
  Imported Jerusalem_Old_City_Walking_to_the_Western_Wall_4K.webm to Commons (T131441) | 
  [production] | 
            
  | 09:14 | 
  <gehel> | 
  testing new Cache-Control headers on mw1017 | 
  [production] | 
            
  | 09:09 | 
  <reedy@tin> | 
  Synchronized wmf-config/throttle.php: Fix my quote fail (duration: 00m 29s) | 
  [production] | 
            
  | 08:42 | 
  <reedy@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: Revert labswiki to wmf.18 as 2FA seems to be broken | 
  [production] | 
            
  | 08:38 | 
  <godog> | 
  bootstrap restbase2004-b | 
  [production] | 
            
  | 08:29 | 
  <dereckson@tin> | 
  Synchronized wmf-config/throttle.php: Fix throttle rules (Gerrit change 280819). (duration: 00m 29s) | 
  [production] | 
            
  | 08:11 | 
  <_joe_> | 
  progressively reducing weight of the older servers in the api cluster | 
  [production] | 
            
  | 07:56 | 
  <_joe_> | 
  depooling mw1121-1130 from the api cluster | 
  [production] | 
            
  | 07:15 | 
  <reedy@tin> | 
  Synchronized wmf-config/throttle.php: T130460 Jerusalem Hackathon (duration: 00m 40s) | 
  [production] | 
            
  | 07:00 | 
  <_joe_> | 
  depooling mw1070-89 from the appserver cluster. T126242 | 
  [production] | 
            
  | 06:53 | 
  <_joe_> | 
  setting all newer appservers weight to 20 in eqiad | 
  [production] | 
            
  | 05:28 | 
  <bearND> | 
  mobileapps deployed 66f8dac | 
  [production] | 
            
  | 05:04 | 
  <bearND> | 
  starting mobileapps deploy | 
  [production] | 
            
  | 02:38 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Fri Apr  1 02:38:39 UTC 2016 (duration 8m 58s) | 
  [production] | 
            
  | 02:29 | 
  <mwdeploy@tin> | 
  sync-l10n completed (1.27.0-wmf.19) (duration: 10m 29s) | 
  [production] | 
            
  | 00:55 | 
  <csteipp@tin> | 
  Synchronized wmf-config/CommonSettings-labs.php: Synching labs revert (duration: 00m 31s) | 
  [production] | 
            
  | 00:54 | 
  <csteipp@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: Synching labs revert (duration: 00m 27s) | 
  [production] | 
            
  | 00:37 | 
  <csteipp@tin> | 
  Synchronized wmf-config/CommonSettings-labs.php: Sync labs change to keep repo clean (duration: 00m 31s) | 
  [production] | 
            
  | 00:36 | 
  <csteipp@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 34s) | 
  [production] | 
            
  
    | 
      
        2016-03-31
      
      §
     | 
  
    
  | 23:48 | 
  <awight@tin> | 
  Finished scap: (no message) (duration: 26m 44s) | 
  [production] | 
            
  | 23:21 | 
  <awight@tin> | 
  Started scap: (no message) | 
  [production] | 
            
  | 23:14 | 
  <awight@tin> | 
  Synchronized php-1.27.0-wmf.19/extensions/OATHAuth: SWAT deployment of OATHAuth fixes, take 2 (duration: 00m 32s) | 
  [production] | 
            
  | 23:13 | 
  <awight@tin> | 
  Synchronized php-1.27.0-wmf.19/extensions/OATHAuth: SWAT deployment of OATHAuth fixes (duration: 00m 46s) | 
  [production] | 
            
  | 22:51 | 
  <Pchelolo> | 
  rolling restart restbase. Apply https://gerrit.wikimedia.org/r/#/c/280711/ config change | 
  [production] | 
            
  | 22:22 | 
  <Pchelolo> | 
  finished update restbase to ba39d2bc | 
  [production] | 
            
  | 22:15 | 
  <Pchelolo> | 
  started update restbase to ba39d2bc | 
  [production] | 
            
  | 22:06 | 
  <Pchelolo> | 
  update restbase to ba39d2bc canary on restbase1005 | 
  [production] | 
            
  | 22:02 | 
  <Pchelolo> | 
  reenable puppet on cerium in restbase staging | 
  [production] | 
            
  | 21:26 | 
  <Pchelolo> | 
  disabling puppet on cerium, updating config and deploying restbase to staging. Testing https://gerrit.wikimedia.org/r/#/c/280711/ | 
  [production] | 
            
  | 21:00 | 
  <mutante> | 
  bast2001 has been reinstalled and can be used again. fingerprints at https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/bast2001.wikimedia.org | 
  [production] | 
            
  | 20:38 | 
  <mutante> | 
  bast2001 - revoking old, signing new puppet certs, salt key.. | 
  [production] | 
            
  | 20:21 | 
  <mutante> | 
  bast2001 - reinstall after disk replacement | 
  [production] | 
            
  | 20:01 | 
  <ottomata> | 
  stopping eventlogging, uninstalling globally installed eventlogging python code, running puppet, restarting eventlogging from /srv/deployment/eventlogging/eventlogging | 
  [production] | 
            
  | 19:59 | 
  <ottomata> | 
  temporarily stopped puppet on eventlog1001 | 
  [production] | 
            
  | 19:12 | 
  <volans> | 
  Starting slave for s2 on db1047 | 
  [production] | 
            
  | 19:04 | 
  <thcipriani@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.19 | 
  [production] | 
            
  | 17:46 | 
  <mutante> | 
  rcs1002 - stop ferm | 
  [production] | 
            
  | 15:56 | 
  <tgr> | 
  running checkLocalUser.php --delete on some wikis for T119736 | 
  [production] | 
            
  | 15:17 | 
  <thcipriani@tin> | 
  Synchronized php-1.27.0-wmf.19/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: SWAT: Ignore ResultSets that do not return pages [[gerrit:280669]] (duration: 00m 38s) | 
  [production] | 
            
  | 14:01 | 
  <volans> | 
  manually running puppet on es2018 to double verify merged changes | 
  [production] | 
            
  | 13:55 | 
  <mutante> | 
  hooft - reboot to pxe, one more time | 
  [production] | 
            
  | 13:49 | 
  <mutante> | 
  netmon1001 - re-enabled puppet (was for torrus issue earlier) | 
  [production] | 
            
  | 13:44 | 
  <elukey> | 
  forced puppet agent on netmon1001 | 
  [production] | 
            
  | 13:27 | 
  <ema> | 
  repooling cp1044, upgraded to varnish 4 (T122880) | 
  [production] | 
            
  | 13:17 | 
  <moritzm> | 
  installing rsync security updates | 
  [production] |