| 2014-01-23
      
      § | 
    
  | 09:27 | <ori> | restarting carbon on tungsten | [production] | 
            
  | 05:33 | <reedy> | updated /a/common to {{Gerrit|Iab1639fd8}}: Add meta charset | [production] | 
            
  | 05:33 | <reedy> | synchronized docroot/noc/conf/index.php | [production] | 
            
  | 05:30 | <reedy> | synchronized docroot/noc/conf/index.php | [production] | 
            
  | 05:30 | <reedy> | updated /a/common to {{Gerrit|Ie8d6d2266}}: Fix broken a tags | [production] | 
            
  | 04:10 | <reedy> | synchronized docroot/noc/conf/index.php | [production] | 
            
  | 04:07 | <reedy> | synchronized multiversion/ | [production] | 
            
  | 04:05 | <reedy> | synchronized docroot and w | [production] | 
            
  | 04:04 | <reedy> | updated /a/common to {{Gerrit|I06964b159}}: dirname( __FILE__ ) to __DIR__ | [production] | 
            
  | 02:18 | <reedy> | synchronized php-1.23wmf11/extensions/EducationProgram/ | [production] | 
            
  | 02:14 | <ori> | synchronized wmf-config/InitialiseSettings.php  'I0dbeff07d: : true everywhere' | [production] | 
            
  | 02:13 | <ori> | updated /a/common to {{Gerrit|I0dbeff07d}}: $wgResourceLoaderStorageEnabled: true everywhere | [production] | 
            
  | 02:10 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-01-23 02:09:57+00:00 | [production] | 
            
  | 02:03 | <LocalisationUpdate> | completed (1.23wmf11) at 2014-01-23 02:03:47+00:00 | [production] | 
            
  | 02:02 | <ori> | synchronized wmf-config/InitialiseSettings.php  'I16738b6ef: : true everywhere except enwiki' | [production] | 
            
  | 02:02 | <LocalisationUpdate> | completed (1.23wmf10) at 2014-01-23 02:02:42+00:00 | [production] | 
            
  | 02:02 | <ori> | updated /a/common to {{Gerrit|I16738b6ef}}: $wgResourceLoaderStorageEnabled: true everywhere except enwiki | [production] | 
            
  | 01:08 | <ori> | synchronized wmf-config/InitialiseSettings.php  'I321ce06ae: Enable module storage on it, de, es and nlwikis' | [production] | 
            
  | 01:08 | <ori> | updated /a/common to {{Gerrit|I321ce06ae}}: Enable module storage on it, de, es and nlwikis | [production] | 
            
  | 00:56 | <hasharMeeting> | integration.wikimedia.org and doc.wikimedia.org have been successfully migrated behind the misc varnish | [production] | 
            
  | 00:53 | <ori> | doing staggered graceful restarts on bits app servers for I9e311c98f | [production] | 
            
  | 00:38 | <hasharMeeting> | migrating  (with RobH) integration.wikimedia.org being misc varnish | [production] | 
            
  | 00:24 | <hasharMeeting> | restarted zuul again | [production] | 
            
  | 00:18 | <hasharMeeting> | upgrading Zuul (lightning deploy of doom) | [production] | 
            
  
    | 2014-01-22
      
      § | 
    
  | 23:41 | <hasharMeeting> | Going to upgrade Zuul, need a patch to make it send  Cache-Control: no-cache so we can migrate Zuul service behind misc varnish (being done with RobH) | [production] | 
            
  | 22:58 | <ottomata> | restarted icinga (a while ago) and puppet freshness checks are all boogery(?) will check up on this tomorrow | [production] | 
            
  | 22:51 | <RobH> | doc.wikimedia.org updates complete, once dns re-propagates it should be again reachable over https | [production] | 
            
  | 22:43 | <RobH> | doc.wikimedia.org and doc.mediawiki.org relocating behind misc-web-lb.  dns ttl was set to 5 minutes yesterday, so this change should result in less than 5 minutes of unreachability by anyone | [production] | 
            
  | 22:09 | <reedy> | updated /a/common to {{Gerrit|Ie95fdf8e9}}: Adjust linkpurge and renderfile limits | [production] | 
            
  | 21:26 | <matanya> | git.wikimedia.org is back up | [production] | 
            
  | 20:44 | <RobH> | all plugins updated except caching cuz i hate it and i plan to move it behind misc-web-lb | [production] | 
            
  | 20:42 | <RobH> | blog updated | [production] | 
            
  | 19:40 | <matanya> | git.wikimedia.org is down | [production] | 
            
  | 19:32 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 17:40 | <RobH> | virt0 puppet agent re-enabled and resumed normal service | [production] | 
            
  | 17:05 | <RobH> | issues with change, don't re-enable puppet agent on virt0 yet | [production] | 
            
  | 16:59 | <RobH> | rolling certificate changes to virt1000 and virt0.  virt0 puppet agent disabled for moment | [production] | 
            
  | 16:33 | <Nemo_bis> | still no data in Global_JobQueue_length ganglia graph on hume (3 days now) | [production] | 
            
  | 14:34 | <csteipp> | synchronized php-1.23wmf10/includes/media  'bug60339' | [production] | 
            
  | 14:33 | <csteipp> | synchronized php-1.23wmf11/includes/media  'bug60339' | [production] | 
            
  | 11:18 | <apergos> | powercycled kaulen, it had died the swap death | [production] | 
            
  | 10:02 | <apergos> | restarted mw1206 a little while ago, it was inaccessible even via mgmt | [production] | 
            
  | 06:31 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 05:37 | <ori> | restarting hafnium for kernel upgrade | [production] | 
            
  | 03:33 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-01-22 03:33:21+00:00 | [production] | 
            
  | 00:55 | <marktraceur> | deployed config for jsduck jobs for MultimediaViewer | [production] | 
            
  | 00:22 | <demon> | synchronized php-1.23wmf11/extensions/CirrusSearch  'Cirrus to master, with better job queues' | [production] | 
            
  | 00:21 | <demon> | synchronized php-1.23wmf10/extensions/CirrusSearch  'Cirrus to master, with better job queues' | [production] | 
            
  | 00:18 | <demon> | synchronized php-1.23wmf10/extensions/Elastica  'Elastica to master' | [production] | 
            
  | 00:15 | <demon> | synchronized php-1.23wmf10/extensions/CirrusSearch  'Rolling back Cirrus -- fatals' | [production] |