| 
      
        2011-11-16
      
      §
     | 
  
    
  | 19:48 | 
  <Ryan_Lane> | 
  shut down sq82 | 
  [production] | 
            
  | 19:39 | 
  <maplebed> | 
  grew xfs on ms5 by 2T to get some breathing room | 
  [production] | 
            
  | 19:30 | 
  <cmjohnson1> | 
  sq81 down for rack relocation | 
  [production] | 
            
  | 18:19 | 
  <awjr> | 
  Increasing donatinos queue consumption script frequency to run every 2 mins rather than 3 mins via Jenkins on aluminium | 
  [production] | 
            
  | 16:26 | 
  <mutante> | 
  fixed permissions on nagios external command file (RT #1994) | 
  [production] | 
            
  | 16:12 | 
  <RobH> | 
  dns updated for db48-db58 (both mgmt and production) | 
  [production] | 
            
  | 15:39 | 
  <mutante> | 
  srv180 - powered down, was offline with disk fail anyways, created RT 1993 | 
  [production] | 
            
  | 13:36 | 
  <mutante> | 
  argon - dpkg errors processing mysql-client pkg upgrade | 
  [production] | 
            
  | 13:33 | 
  <mutante> | 
  argon - dist-upgrade/kernel,reboot (unused limesurvey box) | 
  [production] | 
            
  | 04:12 | 
  <awjr> | 
  synchronizing payments cluster, [[rev:103293|r103293]] | 
  [production] | 
            
  | 03:18 | 
  <asher> | 
  synchronized wmf-config/db.php  'adding db19 to s2, running new mysql build' | 
  [production] | 
            
  | 02:24 | 
  <LocalisationUpdate> | 
  completed (1.18) at Wed Nov 16 02:27:02 UTC 2011 | 
  [production] | 
            
  | 01:57 | 
  <LocalisationUpdate> | 
  failed | 
  [production] | 
            
  | 01:48 | 
  <^demon> | 
  leaving a git svn test running in a screen on formey. Should be fine, but feel free to kill it if svn.wm.o explodes. | 
  [production] | 
            
  | 00:43 | 
  <asher> | 
  synchronized wmf-config/db.php  'putting db11 (s3) into rotation with new mysql build' | 
  [production] | 
            
  | 00:27 | 
  <Tim> | 
  on srv255: ran manual sync-common and puppetd -tv | 
  [production] | 
            
  | 00:19 | 
  <tstarling> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files:  | 
  [production] | 
            
  | 00:18 | 
  <Tim> | 
  rebooting srv255, has been in swapdeath for a week, no response on serial console | 
  [production] | 
            
  | 00:12 | 
  <tstarling> | 
  synchronizing Wikimedia installation... :  | 
  [production] | 
            
  | 00:05 | 
  <tstarling> | 
  synchronized php-1.18/README  'sync test' | 
  [production] | 
            
  | 00:01 | 
  <binasher> | 
  rebooting db11 to new kernel (it's out of rotation) | 
  [production] | 
            
  
    | 
      
        2011-11-15
      
      §
     | 
  
    
  | 23:50 | 
  <awjr> | 
  synchronizing payments cluster to [[rev:103267|r103267]] | 
  [production] | 
            
  | 23:48 | 
  <neilk> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files:  | 
  [production] | 
            
  | 23:35 | 
  <neilk> | 
  synchronizing Wikimedia installation... : renabling multi-file select for UploadWizard on Commons, quashing some bugs with UploadStash | 
  [production] | 
            
  | 23:31 | 
  <maplebed> | 
  imported packages for swift to our repository: swift, swift-*, python-swift, python-eventlet, python-greenlet, python-webob | 
  [production] | 
            
  | 22:44 | 
  <reedy> | 
  synchronizing Wikimedia installation... :  | 
  [production] | 
            
  | 22:44 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files:  | 
  [production] | 
            
  | 22:44 | 
  <reedy> | 
  synchronizing Wikimedia installation... :  | 
  [production] | 
            
  | 22:34 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  'Test (no changes)' | 
  [production] | 
            
  | 22:22 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files:  | 
  [production] | 
            
  | 22:08 | 
  <reedy> | 
  synchronizing Wikimedia installation... : Pushing [[rev:103242|r103242]] for mathsy updates | 
  [production] | 
            
  | 21:58 | 
  <notpeter> | 
  can't pop a shell on searchidx2 via ssh or ipmi rebooting | 
  [production] | 
            
  | 21:12 | 
  <catrope> | 
  synchronized wmf-config/CommonSettings.php  'Comment out the account creation throttle hack for the Serbia event. Leaving it in as a comment so it can be used as a boilerplate for a next time' | 
  [production] | 
            
  | 19:39 | 
  <awjr> | 
  updating frequency of donations queue consumption from every 5mins to 3mins in jenkins on aluminium | 
  [production] | 
            
  | 18:35 | 
  <LeslieCarr> | 
  hostway transit moved | 
  [production] | 
            
  | 18:20 | 
  <LeslieCarr> | 
  moving hostway transit from csw5-pmtpa to cr2-pmtpa | 
  [production] | 
            
  | 16:57 | 
  <mark> | 
  shutting down sq80 for relocation | 
  [production] | 
            
  | 16:43 | 
  <reedy> | 
  synchronized php-1.18/resources/jquery/jquery.textSelection.js  '[[rev:103192|r103192]]' | 
  [production] | 
            
  | 16:37 | 
  <reedy> | 
  synchronized php-1.18/extensions/FlaggedRevs/presentation/FlaggedPageView.php  '[[rev:103191|r103191]]' | 
  [production] | 
            
  | 16:14 | 
  <Jeff_Green> | 
  testing passive-checks config on nagios | 
  [production] | 
            
  | 15:48 | 
  <mark> | 
  Converted eth0 to aggregated bond0 on sq69 | 
  [production] | 
            
  | 15:12 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 31842 - Change of namespace names in Old Church Slavonic Wikipedia' | 
  [production] | 
            
  | 14:49 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 32119 - Change the diacritics in the namespaces at ro.wikinews' | 
  [production] | 
            
  | 14:41 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 32145 - Namespaces/Aliases for Wikipedia, Project and Portal namespace in Assamese wikipedia' | 
  [production] | 
            
  | 14:31 | 
  <mutante> | 
  commented cronjob on locke, user nobody, that shipped aggregations to dammit.lt and got connection refused | 
  [production] | 
            
  | 14:07 | 
  <mark> | 
  Shutting down sq75-78 for relocation | 
  [production] | 
            
  | 14:05 | 
  <mutante> | 
  cleared exim paniclog on mchenry,lily & williams - all just temp. spamd failures, mostly from October | 
  [production] | 
            
  | 13:59 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 32340 - Translation of project namespace and sitename for tawikibooks' | 
  [production] | 
            
  | 13:51 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 32357 - Alias for project space in ko.wikiquote' | 
  [production] | 
            
  | 13:46 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 32370 - Enable Rollbacker, Eliminator and Interface_editors on jawiki and grant bureaucrats to add/remove these flags' | 
  [production] |