| 
      
        2014-09-15
      
      §
     | 
  
    
  | 15:17 | 
  <marktraceur> | 
  Synchronized php-1.24wmf20/extensions/GeoCrumbs/GeoCrumbs.class.php: [SWAT] Handle return value NULL of GeoCrumbs::getParserCache (duration: 00m 07s) | 
  [production] | 
            
  | 15:06 | 
  <marktraceur> | 
  Synchronized wmf-config/: [SWAT] Remove 'renameuser' right from bureaucrats on CentralAuth wikis (duration: 00m 09s) | 
  [production] | 
            
  | 14:54 | 
  <aude> | 
  Synchronized wmf-config/Wikibase.php: Bump wikibase memcached key for test.wikidata, test, test2 (duration: 00m 16s) | 
  [production] | 
            
  | 14:54 | 
  <hashar> | 
  Updated Jenkins Job Builder fork:  e5c0c61..2d74b16  | 
  [production] | 
            
  | 14:50 | 
  <aude> | 
  Finished scap: Put test.wikidata back on mw1.24-wmf19 extension branch (duration: 37m 27s) | 
  [production] | 
            
  | 14:43 | 
  <manybubbles> | 
  restarting the enwiki cirrus reindex process - it crashed over the weekend.  why you crash and leave error message "1".  "1" is not a useful error message. | 
  [production] | 
            
  | 14:13 | 
  <aude> | 
  Started scap: Put test.wikidata back on mw1.24-wmf19 extension branch | 
  [production] | 
            
  | 13:03 | 
  <_joe_> | 
  fenari is swapping hard, restarting apache who was eating up all the RAM | 
  [production] | 
            
  | 09:20 | 
  <hashar> | 
  Synchronized wmf-config/InitialiseSettings.php: *.scienceimage.csiro.au to the wgCopyUploadsDomains {{gerrit|159999}} {{bug|70771}} (duration: 00m 06s) | 
  [production] | 
            
  | 09:16 | 
  <hashar> | 
  Jenkins: apt-get upgrade on prod slaves (updates php5 / libc / jdk 7) | 
  [production] | 
            
  | 03:09 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1036 (duration: 00m 09s) | 
  [production] | 
            
  | 02:03 | 
  <LocalisationUpdate> | 
  failed: mwversionsinuse returned empty list | 
  [production] | 
            
  | 01:47 | 
  <hoo> | 
  Synchronized wmf-config/liquidthreads.php: Remove global $path (duration: 00m 07s) | 
  [production] | 
            
  | 01:47 | 
  <hoo> | 
  Synchronized wmf-config/flaggedrevs.php: Remove global $path (duration: 00m 10s) | 
  [production] | 
            
  
    | 
      
        2014-09-14
      
      §
     | 
  
    
  | 20:37 | 
  <ori_> | 
  enabling puppet on mw1053 | 
  [production] | 
            
  | 20:11 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1062, locked up (duration: 00m 09s) | 
  [production] | 
            
  | 13:24 | 
  <_joe_> | 
  stopped puppet aand the JR on mw1053 | 
  [production] | 
            
  | 12:42 | 
  <hoo> | 
  Ran sync-common on mw1053 to stop "Unrecognized job type 'ChangeNotification'." exceptions | 
  [production] | 
            
  | 11:14 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool es1005 (duration: 00m 07s) | 
  [production] | 
            
  | 10:37 | 
  <springle> | 
  restart es1005 | 
  [production] | 
            
  | 09:56 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool es1007, depool es1005 (duration: 00m 10s) | 
  [production] | 
            
  | 02:01 | 
  <LocalisationUpdate> | 
  failed: mwversionsinuse returned empty list | 
  [production] | 
            
  | 00:45 | 
  <ori_> | 
  fenari appears to still have twemproxy (in addition to nutcracker); decom'ing. | 
  [production] | 
            
  | 00:29 | 
  <ori_> | 
  restarting apache2 on fenari | 
  [production] | 
            
  
    | 
      
        2014-09-13
      
      §
     | 
  
    
  | 04:42 | 
  <legoktm> | 
  global rename for Trevor Parscal (WMF) unstuck itself, yay | 
  [production] | 
            
  | 04:22 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Sat Sep 13 04:22:04 UTC 2014 (duration 22m 3s) | 
  [production] | 
            
  | 03:51 | 
  <legoktm> | 
  global rename for Trevor Parscal --> Trevor Parscal (WMF) looks stuck on metawiki and mswiki, in queued state for both but showJobs.php says the jobs are active and claimed | 
  [production] | 
            
  | 03:11 | 
  <LocalisationUpdate> | 
  completed (1.24wmf21) at 2014-09-13 03:11:40+00:00 | 
  [production] | 
            
  | 02:38 | 
  <LocalisationUpdate> | 
  completed (1.24wmf20) at 2014-09-13 02:38:26+00:00 | 
  [production] | 
            
  | 01:45 | 
  <ori> | 
  Synchronized php-1.24wmf21/extensions/Flow: Update flow for I4da934dfe (duration: 00m 06s) | 
  [production] | 
            
  | 01:45 | 
  <ori> | 
  Synchronized php-1.24wmf20/extensions/Flow: Update flow for I4da934dfe (duration: 00m 06s) | 
  [production] | 
            
  | 01:41 | 
  <ori> | 
  Synchronized php-1.24wmf20/extensions/Flow: Update flow for I4da934dfe (duration: 00m 08s) | 
  [production] | 
            
  
    | 
      
        2014-09-12
      
      §
     | 
  
    
  | 21:26 | 
  <csteipp> | 
  deployed fixes for bugs 70620, 69008 | 
  [production] | 
            
  | 20:37 | 
  <mattflaschen> | 
  Synchronized php-1.24wmf21/extensions/GettingStarted/: Deploy to fix GettingStarted bucketting for users with null registration date (duration: 00m 05s) | 
  [production] | 
            
  | 20:37 | 
  <mattflaschen> | 
  Synchronized php-1.24wmf20/extensions/GettingStarted/: Deploy to fix GettingStarted bucketting for users with null registration date (duration: 00m 07s) | 
  [production] | 
            
  | 19:34 | 
  <legoktm> | 
  running migratePass0.php across all CentralAuth wikis | 
  [production] | 
            
  | 17:43 | 
  <ori> | 
  updated /a/common to {{Gerrit|I4e4187285}}: Rename some constants to clarify their meaning and purpose | 
  [production] | 
            
  | 14:52 | 
  <manybubbles> | 
  rebuilding enwiki's Cirrus index for more performance testing.  Please be faster now.  k? | 
  [production] | 
            
  | 08:37 | 
  <_joe_> | 
  rolling restart of pybal finished. Adding note on Fenari | 
  [production] | 
            
  | 08:19 | 
  <_joe_> | 
  reactivated puppet on all lvs hosts, esams almost done, pending eqiad | 
  [production] | 
            
  | 08:06 | 
  <_joe_> | 
  new pybal conf applied in all of ulsfo | 
  [production] | 
            
  | 07:39 | 
  <_joe_> | 
  changing pybal config place; stopping puppet on all loadbalancers | 
  [production] | 
            
  | 04:27 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Sep 12 04:27:17 UTC 2014 (duration 27m 16s) | 
  [production] | 
            
  | 03:16 | 
  <LocalisationUpdate> | 
  completed (1.24wmf21) at 2014-09-12 03:15:57+00:00 | 
  [production] | 
            
  | 03:08 | 
  <mattflaschen> | 
  Finished scap: One last CSS fix (wrapping issue for error state) for GettingStarted A/B test (duration: 24m 38s) | 
  [production] | 
            
  | 02:43 | 
  <mattflaschen> | 
  Started scap: One last CSS fix (wrapping issue for error state) for GettingStarted A/B test | 
  [production] | 
            
  | 02:39 | 
  <LocalisationUpdate> | 
  completed (1.24wmf20) at 2014-09-12 02:39:35+00:00 | 
  [production] | 
            
  | 01:33 | 
  <mattflaschen> | 
  Synchronized php-1.24wmf21/extensions/GettingStarted/: CSS tweaks for GettingStarted A/B test (duration: 00m 07s) | 
  [production] | 
            
  | 01:32 | 
  <mattflaschen> | 
  Synchronized php-1.24wmf20/extensions/GettingStarted/: CSS tweaks for GettingStarted A/B test (duration: 00m 21s) | 
  [production] | 
            
  | 01:29 | 
  <ori> | 
  Synchronized wmf-config/wikitech.php: Ia5b81076e: Update path reference for /srv/mediawiki (duration: 00m 04s) | 
  [production] |