| 2014-12-16
      
      § | 
    
  | 17:17 | <bd808> | git-sync-upstream runs cleanly on deployment-salt again! | [releng] | 
            
  | 17:16 | <bd808> | removed cherry pick of Ib2a0401a7aa5632fb79a5b17c0d0cef8955cf990 (-2 by _joe_; replaced by Ibcad98a95413044fd6c5e9bd3c0a6fb486bd5fe9) | [releng] | 
            
  | 17:15 | <bd808> | removed cherry pick of I3b6e37a2b6b9389c1a03bd572f422f898970c5b4 (modified in gerrit by bd808 and not repicked; merged) | [releng] | 
            
  | 17:15 | <bd808> | removed cherry pick of I08c24578596506a1a8baedb7f4a42c2c78be295a (-2 by _joe_ in gerrit; replaced by Iba742c94aa3df7497fbff52a856d7ba16cf22cc7) | [releng] | 
            
  | 17:13 | <bd808> | removed cherry pick of I6084f49e97c855286b86dbbd6ce8e80e94069492 (merged by Ori with a change) | [releng] | 
            
  | 17:09 | <bd808> | trying to fix it without using important changes | [releng] | 
            
  | 17:08 | <bd808> | deployment-salt:/var/lib/git/operations/puppet is a rebase hell of cherry-picks that don't apply | [releng] | 
            
  | 16:52 | <manybubbles> | elasticsearch restart finished | [releng] | 
            
  | 16:48 | <mutante> | deployment-db2 is down | [releng] | 
            
  | 16:48 | <manybubbles> | restarting beta's elasticsearch servers to pick up a new version of a plugin.  won't interfere with current downtime. | [releng] | 
            
  | 13:51 | <hashar> | deleting integration-slave1001 and recreating it. It is blocked on boot and we can't console on it https://phabricator.wikimedia.org/T76250 | [releng] | 
            
  
    | 2014-12-13
      
      § | 
    
  | 18:51 | <bd808> | Running chmod -R g+s /data/project/upload7 on deploymnet-mediawiki02 | [releng] | 
            
  | 18:25 | <bd808> | Running chmod -R u=rwX,g=rwX,o=rX /data/project/upload7 from deployment-mediawiki02 | [releng] | 
            
  | 18:16 | <bd808> | chown done for /data/project/upload7 | [releng] | 
            
  | 17:51 | <bd808> | Running chown -R apache:apache on /data/project/upload7 from deployment-mediawiki02 | [releng] | 
            
  | 17:11 | <bd808> | Labs DNS seems to be flaking out badly and causing random scap and puppet failures | [releng] | 
            
  | 17:10 | <bd808> | Many strange puppet and scap failures in beta that look to be related to DNS failures | [releng] | 
            
  | 16:58 | <bd808> | restarted puppetmaster on deployment-salt | [releng] | 
            
  | 16:31 | <bd808> | apache user renumbered on deployment-mediawiki03 | [releng] | 
            
  | 16:23 | <bd808> | apache and hhvm restarted on beta app servers following apache user renumber | [releng] | 
            
  | 16:09 | <bd808> | apache and hhvm stopped on beta app server tier. All requests expected to return 503 from varnish | [releng] | 
            
  | 16:03 | <bd808> | Starting work on [[phab:T78076]] to renumber apache users in beta | [releng] | 
            
  | 16:03 | <bd808> | Starting work on [[phab:T78076]] to renumber apache users in beta | [releng] | 
            
  | 08:21 | <YuviPanda|zzz> | forcing puppet run on all deployment-prep hosts | [releng] |