| 2016-09-18
      
      § | 
    
  | 20:37 | <bblack> | restart upload backend: cp3036 | [production] | 
            
  | 20:22 | <bblack> | restart upload backend: cp3039 | [production] | 
            
  | 19:58 | <bblack> | restart upload backend: cp1074 (stats indicate LRU_Fail imminent) | [production] | 
            
  | 19:28 | <bblack> | restart upload backend: cp1064 (already in LRU_Fail, caught early) | [production] | 
            
  | 19:28 | <bblack> | restart up | [production] | 
            
  | 18:06 | <bblack> | restart upload varnish backend: cp1050 (already in LRU_Fail) | [production] | 
            
  | 17:58 | <bblack> | restart upload varnish backend: cp2008 | [production] | 
            
  | 17:42 | <bblack> | restart upload varnish backend: cp1071 | [production] | 
            
  | 17:32 | <bblack> | restart upload varnish backend: cp2020 | [production] | 
            
  | 17:06 | <bblack> | restart upload varnish backend: cp2026 | [production] | 
            
  | 16:35 | <bblack> | restarting upload varnish backend: cp2011 | [production] | 
            
  | 15:43 | <bblack> | restarting upload varnish backend: cp2017 | [production] | 
            
  | 15:13 | <bblack> | restarting upload varnish backend: cp2005 | [production] | 
            
  | 14:52 | <bblack> | restarting upload varnish backend: cp1049 | [production] | 
            
  | 14:42 | <bblack> | restarting upload varnish backend: cp2022 | [production] | 
            
  | 14:17 | <bblack> | restarting varnish backend on cp1073 (503 LRU_Fail pattern, has been up a few days...) | [production] | 
            
  | 13:29 | <bblack> | disabling puppet on cp1074, to experiment with vhtcpd regex filter | [production] | 
            
  | 11:17 | <ema> | repooling varnish-be in codfw | [production] | 
            
  | 10:59 | <ema> | varnish-backend restart on cp3037 | [production] | 
            
  | 10:58 | <ema> | varnish-backend restart on cp3044 | [production] | 
            
  | 10:54 | <ema> | repooling varnish on cp1050 | [production] | 
            
  | 10:53 | <ema> | repooling varnish on cp1062 | [production] | 
            
  | 10:52 | <ema> | repooling varnish on cp1064 | [production] | 
            
  | 10:50 | <ema> | repooling varnish on cp1071 | [production] | 
            
  | 10:50 | <ema> | repooling varnish on cp1072 | [production] | 
            
  | 10:49 | <ema> | repooling varnish on cp1073 | [production] | 
            
  | 10:49 | <ema> | repooling varnish on cp1074 | [production] | 
            
  | 09:47 | <_joe_> | varnish-backend-restart on cp1063 | [production] | 
            
  | 09:30 | <_joe_> | varnish-backend-restart on cp1048 | [production] | 
            
  | 06:09 | <ejegg|afk> | updated civicrm from 5ba6976f2552564b51085abff0afd5f76195229b to 1df25962834885121bd7a9ba856c0c69c2b9cfda | [production] | 
            
  | 02:32 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Sun Sep 18 02:32:49 UTC 2016 (duration 5m 58s) | [production] | 
            
  | 02:26 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.18) (duration: 10m 18s) | [production] | 
            
  
    | 2016-09-17
      
      § | 
    
  | 16:40 | <_joe_> | rolling restart of HHVM on part fo the API cluster in eqiad, T133674 | [production] | 
            
  | 08:15 | <_joe_> | enlarged puppet partition on puppetmaster1001, rendered full by reports | [production] | 
            
  | 07:06 | <p858snake> | set +z on -operations, allows messages sent by +b or +q users (normally blocked) to be seen by users that currently op'ed | [production] | 
            
  | 06:55 | <p858snake> | see T145924 or email to ops list for more info | [production] | 
            
  | 06:54 | <p858snake> | silenced (+q) icinga-wm in operations channel, due to channel spam from low disk space on puppetm1001 | [production] | 
            
  | 02:34 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Sat Sep 17 02:34:33 UTC 2016 (duration 7m 3s) | [production] | 
            
  | 02:27 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.18) (duration: 11m 40s) | [production] |