| 2014-07-07
      
      § | 
    
  | 12:11 | <hashar> | Jenkins job builder e1ddd23 fails for us :/  Moving back to parent commit | [production] | 
            
  | 12:09 | <hashar> | Updated our Jenkins job builder fork  0972985..e1ddd23 | [production] | 
            
  | 09:40 | <godog> | upgrade ms-be1003/1004/1012 (zone2) to swift icehouse | [production] | 
            
  | 09:16 | <_joe_> | restarting rhenium, pings but no ssh since 2 days, serial console is blank and unresponsive | [production] | 
            
  | 09:15 | <godog> | upgrade ms-be1002/1008 (zone1) to swift icehouse | [production] | 
            
  | 02:53 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Mon Jul  7 02:52:10 UTC 2014 (duration 52m 9s) | [production] | 
            
  | 02:24 | <LocalisationUpdate> | completed (1.24wmf12) at 2014-07-07 02:23:48+00:00 | [production] | 
            
  | 02:13 | <LocalisationUpdate> | completed (1.24wmf11) at 2014-07-07 02:12:44+00:00 | [production] | 
            
  
    | 2014-07-03
      
      § | 
    
  | 23:40 | <mutante> | osmium - libboost-dev : Depends: libboost1.54-dev but it is not going to be installed | [production] | 
            
  | 23:33 | <mutante> | rhenium (pmacct / flow) Out of memory: Kill process 3123 (pmacctd) score 1 or sacrifice child | [production] | 
            
  | 23:22 | <K4-713> | updated payments to c5689f385b2f0a7bdc55c5752010e9eb | [production] | 
            
  | 23:17 | <mwalker> | Synchronized php-1.24wmf12/extensions/VisualEditor/: Updating VisualEditor for {{gerrit|144081}} (duration: 00m 12s) | [production] | 
            
  | 21:07 | <oblivian> | gracefulled all apaches | [production] | 
            
  | 20:45 | <mutante> | deleted analytics/kraken branch from ops/puppet via gerrit ui, ack'ed by ottomata | [production] | 
            
  | 20:12 | <bd808|deploy> | Updated scap to ff04431 (restart-nutcracker script) | [production] | 
            
  | 19:53 | <reedy> | Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 14s) | [production] | 
            
  | 19:48 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 14s) | [production] | 
            
  | 19:48 | <reedy> | Synchronized docroot and w: (no message) (duration: 00m 30s) | [production] | 
            
  | 19:47 | <reedy> | Synchronized database lists: (no message) (duration: 00m 20s) | [production] | 
            
  | 19:36 | <jgage> | rebooting analytics1012 for bios change: cpufreq governor | [production] | 
            
  | 19:27 | <ottomata> | disabling puppet on hadoop related analytics nodes, preparing for reinstall | [production] | 
            
  | 19:21 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 14s) | [production] | 
            
  | 19:16 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.24wmf12 | [production] | 
            
  | 19:12 | <reedy> | Synchronized php-1.24wmf11/languages/Language.php: I039547b867b2eab47692dcc018c95b89975bc65d (duration: 00m 40s) | [production] | 
            
  | 18:49 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedia to 1.24wmf11 | [production] | 
            
  | 18:41 | <reedy> | Finished scap: testwiki to 1.24wmf12 and build l10n cache (duration: 30m 47s) | [production] | 
            
  | 18:18 | <ottomata> | doing rolling restarts of zookeeper servers and kafka brokers to load up new zk timeout changes | [production] | 
            
  | 18:10 | <reedy> | Started scap: testwiki to 1.24wmf12 and build l10n cache | [production] | 
            
  | 18:10 | <reedy> | scap aborted: testwiki to 1.24wmf12 and build l10n cache (duration: 27m 26s) | [production] | 
            
  | 17:53 | <godog> | reloading librenms, semi-broke it with a syslog search (again) | [production] | 
            
  | 17:46 | <godog> | reloading librenms, semi-broke it with a syslog search | [production] | 
            
  | 17:42 | <reedy> | Started scap: testwiki to 1.24wmf12 and build l10n cache | [production] | 
            
  | 16:39 | <maxsem> | Synchronized php-1.24wmf11/extensions/EventLogging/: bug 67420 (duration: 00m 35s) | [production] | 
            
  | 16:34 | <paravoid> | apt: uploading nutcracker backport for precise | [production] | 
            
  | 08:07 | <hashar> | Jenkins restarted | [production] | 
            
  | 08:00 | <hashar> | upgrading Jenkins (minor version bump 1.554.2 -> 1.554.3) | [production] | 
            
  | 03:39 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu Jul  3 03:38:46 UTC 2014 (duration 38m 45s) | [production] |