| 
      
        2012-06-22
      
      §
     | 
  
    
  | 11:44 | 
  <mutante> | 
  installing upgrades and kernel on pdf1, can reboot? (also needs puppetizing and precise reinstall) | 
  [production] | 
            
  | 10:46 | 
  <mutante> | 
  installing security upgrades and kernel on bast1001 (still needs reboot, but dont break user sessions) | 
  [production] | 
            
  | 10:42 | 
  <mutante> | 
  fenari upgrade - this included replace wikimedia-lvs-realserver 0.04 (using .../wikimedia-lvs-realserver_0.08 | 
  [production] | 
            
  | 10:41 | 
  <mutante> | 
  installing security upgrades on fenari | 
  [production] | 
            
  | 10:31 | 
  <mutante> | 
  installing security upgrades on formey (gerrit) | 
  [production] | 
            
  | 08:49 | 
  <hashar> | 
  synchronized wmf-config/CommonSettings.php  '{{gerrit|12569}} Load transcode conf on -e /etc/wikimedia-transcoding (wmflabs change)' | 
  [production] | 
            
  | 08:40 | 
  <hashar> | 
  synchronized wmf-config/CommonSettings.php  '{{gerrit|12568}} Disable wgNoticeInfrastructure on beta cluster' | 
  [production] | 
            
  | 08:30 | 
  <hashar> | 
  synchronized wmf-config/CommonSettings.php  '{{gerrit|12566}} labs use the same wgCentralDBname on all wiki' | 
  [production] | 
            
  | 07:59 | 
  <apergos> | 
  powercycled lvs1001, not pingable, nothing good from mgmt console, etc. | 
  [production] | 
            
  | 04:11 | 
  <aaron> | 
  synchronized php-1.20wmf5/includes/WikiPage.php  | 
  [production] | 
            
  | 02:24 | 
  <LocalisationUpdate> | 
  completed (1.20wmf5) at Fri Jun 22 02:24:15 UTC 2012 | 
  [production] | 
            
  | 01:00 | 
  <catrope> | 
  synchronized php-1.20wmf5/extensions/VisualEditor  'VisualEditor updates' | 
  [production] | 
            
  | 00:37 | 
  <LeslieCarr> | 
  restarting exim4 on mchenry with split_spool_directory = true | 
  [production] | 
            
  | 00:30 | 
  <aaron> | 
  synchronized wmf-config/PrivateSettings.php  'Updated swift user config.' | 
  [production] | 
            
  
    | 
      
        2012-06-21
      
      §
     | 
  
    
  | 23:11 | 
  <catrope> | 
  synchronized php-1.20wmf5/extensions/VisualEditor/modules/ve/ce/nodes/ve.ce.TextNode.js  | 
  [production] | 
            
  | 23:07 | 
  <catrope> | 
  synchronized php-1.20wmf5/extensions/VisualEditor/VisualEditor.php  | 
  [production] | 
            
  | 22:49 | 
  <catrope> | 
  synchronized php-1.20wmf5/extensions/VisualEditor/modules/ve/ce/nodes/ve.ce.TextNode.js  | 
  [production] | 
            
  | 22:26 | 
  <catrope> | 
  synchronized php-1.20wmf5/extensions/VisualEditor/modules/ve/ce/nodes/ve.ce.TextNode.js  | 
  [production] | 
            
  | 21:28 | 
  <notpeter> | 
  restarting all lucene instances to direct logs to oxygen | 
  [production] | 
            
  | 21:16 | 
  <binasher> | 
  deploying new mobile redirector to esams text squids | 
  [production] | 
            
  | 21:00 | 
  <catrope> | 
  synchronized php-1.20wmf5/extensions/VisualEditor  'VisualEditor bugfixes' | 
  [production] | 
            
  | 20:34 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 20:29 | 
  <binasher> | 
  deployed new squid mobile redirector, now covers additional projects | 
  [production] | 
            
  | 20:27 | 
  <reedy> | 
  synchronized wmf-config/  | 
  [production] | 
            
  | 20:10 | 
  <hashar:> | 
  on gallium, cloning mediawiki/extensions.git to /var/lib/jenkins/jobs/MediaWiki-Extensions-Fetching/workspace | 
  [production] | 
            
  | 19:25 | 
  <mark> | 
  Restarted 3 queue runners as exim -qff & | 
  [production] | 
            
  | 19:15 | 
  <catrope> | 
  Finished syncing Wikimedia installation... : VisualEditor updates | 
  [production] | 
            
  | 18:57 | 
  <catrope> | 
  Started syncing Wikimedia installation... : VisualEditor updates | 
  [production] | 
            
  | 18:50 | 
  <mark> | 
  Started 5 exim queue runners on mchenry with exim -qqff & | 
  [production] | 
            
  | 18:49 | 
  <catrope> | 
  synchronized wmf-config/CommonSettings.php  'Point $wgVisualEditorParsoidURL to cadmium' | 
  [production] | 
            
  | 18:36 | 
  <catrope> | 
  synchronized php-1.20wmf5/includes/OutputPage.php  'Core patches for VisualEditor deploy' | 
  [production] | 
            
  | 18:36 | 
  <catrope> | 
  synchronized php-1.20wmf5/resources/Resources.php  'Core patches for VisualEditor deploy' | 
  [production] | 
            
  | 18:36 | 
  <catrope> | 
  synchronized php-1.20wmf5/resources/mediawiki.page/mediawiki.page.watch.ajax.js  'Core patches for VisualEditor deploy' | 
  [production] | 
            
  | 17:51 | 
  <notpeter> | 
  restarting puppet on brewster | 
  [production] | 
            
  | 15:25 | 
  <notpeter> | 
  stopping puppet on brewster | 
  [production] | 
            
  | 14:11 | 
  <paravoid> | 
  powercycling srv272, unreachable due to load spike | 
  [production] | 
            
  | 05:30 | 
  <binasher> | 
  clearing mobile varnish cache - my friend can't expand some article categories on his iphone after rebooting and clearing cache | 
  [production] | 
            
  | 04:41 | 
  <catrope> | 
  synchronized php-1.20wmf4/extensions/LastModified/modules/lastmodified.js  | 
  [production] | 
            
  | 04:40 | 
  <catrope> | 
  synchronized php-1.20wmf5/extensions/LastModified/modules/lastmodified.js  | 
  [production] | 
            
  | 02:25 | 
  <LocalisationUpdate> | 
  completed (1.20wmf5) at Thu Jun 21 02:25:40 UTC 2012 | 
  [production] | 
            
  | 00:10 | 
  <binasher> | 
  stopped puppet on cp1020 until tomorrow - testing new build of the squid mobile redirector on one server until tomorrow | 
  [production] | 
            
  
    | 
      
        2012-06-20
      
      §
     | 
  
    
  | 23:29 | 
  <kaldari> | 
  synchronized php-1.20wmf5/extensions/LastModified/E3Experiments/js/ext.E3Experiments.Timestamp.js  'updating clicktracking for LastModified and E3Experiments exts' | 
  [production] | 
            
  | 21:52 | 
  <Reedy> | 
  pointed /usr/local/apache/common/php at /usr/local/apache/common/php-1.20wmf5 on mediawiki-installation | 
  [production] | 
            
  | 21:49 | 
  <LeslieCarr> | 
  see RT3170 for more details on above change and mchenry pain | 
  [production] | 
            
  | 21:48 | 
  <LeslieCarr> | 
  freezing many bounce messages on mchenry (all older than 2400 minutes) | 
  [production] | 
            
  | 21:04 | 
  <LeslieCarr> | 
  replaced srv268 with srv245 in memcached list | 
  [production] | 
            
  | 21:04 | 
  <lcarr> | 
  synchronized wmf-config/mc.php  'removed broken srv268' | 
  [production] | 
            
  | 21:03 | 
  <paravoid> | 
  powercycling srv268; unreachable due to load spike | 
  [production] | 
            
  | 18:31 | 
  <aaron> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files:  | 
  [production] | 
            
  | 18:23 | 
  <LeslieCarr> | 
  reloading mr1-pmtpa for sw upgrade (fixing a cpu bug) | 
  [production] |