| 2013-01-18
      
      ยง | 
    
  | 19:49 | <RobH> | authdns-update colby back into sandbox | [production] | 
            
  | 19:08 | <aaron> | synchronized wmf-config/filebackend.php  'Added ceph file backend configuration' | [production] | 
            
  | 19:08 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 19:00 | <mark> | rm -rf /usr/local/apache/common-local on eqiad apaches | [production] | 
            
  | 18:54 | <RobH> | restarted pdns on linne, it borked on authdns-update | [production] | 
            
  | 18:48 | <RobH> | authdns-update | [production] | 
            
  | 18:17 | <root> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 18:04 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 18:02 | <root> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 18:00 | <mark> | Removed mw1085 from mediawiki_installation (acting up) | [production] | 
            
  | 17:54 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 17:27 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 16:11 | <hashar> | gallium jenkins: updated tools/fetch-mw-ext script to wipe the extension destination directory {{gerrit|44648}} updated: 09bbb44..e41843b | [production] | 
            
  | 16:00 | <hashar> | gallium jenkins: updated the tools/fetch-mw-ext script which was not properly fetching the extension. {{gerrit|44644}]  updated: cfa82ff..09bbb44 | [production] | 
            
  | 15:18 | <Reedy> | recentchanges.rc_params is now a blob on all wikis | [production] | 
            
  | 15:14 | <mark> | Deployed MediaWiki on eqiad image scalers | [production] | 
            
  | 15:11 | <root> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 15:05 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 15:00 | <root> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 14:57 | <paravoid> | running /usr/local/bin/foreachwiki maintenance/cleanupUploadStash.php on hume | [production] | 
            
  | 14:57 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 14:49 | <reedy> | synchronized php-1.21wmf8/maintenance/cleanupUploadStash.php | [production] | 
            
  | 14:49 | <reedy> | synchronized php-1.21wmf7/maintenance/cleanupUploadStash.php | [production] | 
            
  | 14:31 | <mark> | Corrected srv191:/tmp permissions | [production] | 
            
  | 14:21 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 13:56 | <mark> | Set rt_cache_rebuild_count to -1 on lvs2 and lvs6 | [production] | 
            
  | 13:50 | <mark> | Powercycled lvs6 | [production] | 
            
  | 13:26 | <demon> | synchronized php-1.21wmf7/includes/resourceloader/ResourceLoaderFileModule.php  'Deploying Idc83a0fe' | [production] | 
            
  | 13:25 | <demon> | synchronized php-1.21wmf7/includes/resourceloader/ResourceLoader.php  'Deploying Idc83a0fe' | [production] | 
            
  | 13:25 | <demon> | synchronized php-1.21wmf7/includes/MessageBlobStore.php  'Deploying Idc83a0fe' | [production] | 
            
  | 12:59 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 12:58 | <Tim> | added eqiad apaches to mediawiki-installation and started scap, it will take a while | [production] | 
            
  | 12:31 | <tstarling> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 12:29 | <tstarling> | Started syncing Wikimedia installation... : | [production] | 
            
  | 12:19 | <root> | Finished syncing Wikimedia installation... : | [production] | 
            
  | 12:10 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 12:03 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 11:58 | <mark> | Mounted nfs1:/home noatime | [production] | 
            
  | 11:58 | <root> | Started syncing Wikimedia installation... : | [production] | 
            
  | 10:31 | <RoanKattouw> | Purged all URLs containing 'wikidata' from Varnish | [production] | 
            
  | 10:22 | <tstarling> | Started syncing Wikimedia installation... : | [production] | 
            
  | 05:28 | <Tim> | running sync-apache and gracefulling apaches to hopefully fix reports of occasional wikidata redirect issues | [production] | 
            
  | 05:04 | <tstarling> | synchronized live-1.5/load.php  're-enabling bits for MF' | [production] | 
            
  | 03:39 | <awjrichards> | synchronized php-1.21wmf8/extensions/MobileFrontend  'Roll back today's MobileFrontend deployment to 1ded0894cdbc089370a58ba41aeb4db53e5c7345' | [production] | 
            
  | 03:38 | <awjrichards> | synchronized php-1.21wmf7/extensions/MobileFrontend  'Roll back today's MobileFrontend deployment to 1ded0894cdbc089370a58ba41aeb4db53e5c7345' | [production] | 
            
  | 03:30 | <tstarling> | synchronized live-1.5/load.php | [production] | 
            
  | 03:24 | <Tim> | attempting to block MF in bits apache configuration | [production] | 
            
  | 02:50 | <LocalisationUpdate> | completed (1.21wmf8) at Fri Jan 18 02:50:35 UTC 2013 | [production] | 
            
  | 02:49 | <Tim> | restarted bits apaches due to overload, were totally down | [production] | 
            
  | 02:27 | <LocalisationUpdate> | completed (1.21wmf7) at Fri Jan 18 02:26:57 UTC 2013 | [production] |