| 
      
        2013-10-30
      
      §
     | 
  
    
  | 23:29 | 
  <demon> | 
  synchronized wmf-config/PoolCounterSettings-pmtpa.php  'Upping search maxqueue to 600' | 
  [production] | 
            
  | 23:29 | 
  <demon> | 
  synchronized wmf-config/PoolCounterSettings-eqiad.php  'Upping search maxqueue to 600' | 
  [production] | 
            
  | 23:28 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Disable Zero on wikidataw' | 
  [production] | 
            
  | 23:17 | 
  <maxsem> | 
  synchronized php-1.23wmf1/extensions/MobileFrontend/  'Shit hit fan' | 
  [production] | 
            
  | 23:08 | 
  <maxsem> | 
  synchronized php-1.23wmf1/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/#/c/92676/' | 
  [production] | 
            
  | 23:07 | 
  <maxsem> | 
  synchronized php-1.22wmf22/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/#/c/92676/' | 
  [production] | 
            
  | 23:02 | 
  <yurik> | 
  synchronized php-1.23wmf1/extensions/ZeroRatedMobileAccess/ | 
  [production] | 
            
  | 22:48 | 
  <yurik> | 
  synchronized php-1.22wmf22/extensions/ZeroRatedMobileAccess/ | 
  [production] | 
            
  | 22:00 | 
  <K4-713> | 
  updated civicrm to 06348e7eea3 | 
  [production] | 
            
  | 19:56 | 
  <demon> | 
  Finished syncing Wikimedia installation... : No changes, making sure arsenic is synced | 
  [production] | 
            
  | 19:46 | 
  <demon> | 
  Started syncing Wikimedia installation... : No changes, making sure arsenic is synced | 
  [production] | 
            
  | 19:45 | 
  <demon> | 
  synchronized multiversion  'Making sure multiversion is up to date on arsenic' | 
  [production] | 
            
  | 19:44 | 
  <demon> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Making sure wikiversions is up to date on arsenic | 
  [production] | 
            
  | 19:27 | 
  <cmjohnson> | 
  removing decom'd srv* from /etc/dsh/groups nrpe-ext-stores, nagios | 
  [production] | 
            
  | 19:22 | 
  <cmjohnson> | 
  dns update | 
  [production] | 
            
  | 18:39 | 
  <demon> | 
  synchronized robots.txt  'New robots.txt changes for mobileview' | 
  [production] | 
            
  | 17:47 | 
  <awight> | 
  dedupe updated from 5b9f19ba719dc6 to 922b3aa6b797a | 
  [production] | 
            
  | 17:38 | 
  <demon> | 
  Finished syncing Wikimedia installation... : No changes, making sure arsenic is synced | 
  [production] | 
            
  | 17:36 | 
  <awight> | 
  updated crm from 6f370cccad7f9 to 117025ce5c993d | 
  [production] | 
            
  | 17:20 | 
  <demon> | 
  Started syncing Wikimedia installation... : No changes, making sure arsenic is synced | 
  [production] | 
            
  | 17:17 | 
  <^d> | 
  aborted sync | 
  [production] | 
            
  | 17:17 | 
  <demon> | 
  Started syncing Wikimedia installation... : No changes, making sure arsenic is synced | 
  [production] | 
            
  | 15:40 | 
  <hashar> | 
  Broke jenkins for a couple hours while doing some path migration.  All jobs got updated in the process and failed jobs got retriggered. | 
  [production] | 
            
  | 10:40 | 
  <apergos> | 
  restarted morebots (failed to rejoin after netsplit) | 
  [production] | 
            
  
    | 
      
        2013-10-29
      
      §
     | 
  
    
  | 21:14 | 
  <maxsem> | 
  synchronized php-1.23wmf1/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/#/c/92474/' | 
  [production] | 
            
  | 21:12 | 
  <maxsem> | 
  synchronized php-1.22wmf22/extensions/MobileFrontend/  'https://gerrit.wikimedia.org/r/#/c/92474/' | 
  [production] | 
            
  | 21:05 | 
  <LeslieCarr> | 
  one of the sdtpa <-> eqiad links is down | 
  [production] | 
            
  | 20:50 | 
  <^demon|lunch> | 
  elasticsearch: stopped wikidatawiki indexing processes on terbium, causing too much load. working with RobH on getting an alternative place to run these scripts. | 
  [production] | 
            
  | 20:08 | 
  <mwalker> | 
  fundraising cluster back to normal | 
  [production] | 
            
  | 20:06 | 
  <cmjohnson1> | 
  dns update | 
  [production] | 
            
  | 19:14 | 
  <mutante> | 
  create mw/1.22 dir on dumps.wm.org, publish mw 1.22.0rc0 - http://dumps.wikimedia.org/mediawiki/1.22/ | 
  [production] | 
            
  | 19:06 | 
  <cmjohnson1> | 
  dns update | 
  [production] | 
            
  | 18:49 | 
  <cmjohnson1> | 
  dns update | 
  [production] | 
            
  | 17:51 | 
  <mwalker> | 
  frack cluster back in maintenance mode while I figure out why persistent STOMP messages fail to be stored | 
  [production] | 
            
  | 17:46 | 
  <bsitu> | 
  synchronized wmf-config/InitialiseSettings.php  'Switch all small wikis to use extension1 db' | 
  [production] | 
            
  | 17:31 | 
  <mwalker> | 
  taking the payments cluster out of maintenance mode for testing | 
  [production] | 
            
  | 16:49 | 
  <reedy> | 
  synchronized php-1.23wmf1/extensions/Wikibase | 
  [production] | 
            
  | 16:35 | 
  <mwalker> | 
  placing fundraising cluster into maintenance mode for ActiveMQ and database maintenance | 
  [production] | 
            
  | 16:23 | 
  <mwalker> | 
  updating payments from e47560375bfa5515cf27b0f4169f51011e883fd5 | 
  [production] | 
            
  | 16:13 | 
  <^d> | 
  elasticsearch: created wikidatawiki index, running indexing from 8 processes on terbium. | 
  [production] | 
            
  | 16:08 | 
  <reedy> | 
  synchronized php-1.23wmf1/includes/upload/UploadStash.php | 
  [production] | 
            
  | 16:07 | 
  <reedy> | 
  synchronized php-1.22wmf22/includes/upload/UploadStash.php | 
  [production] | 
            
  | 16:04 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Cirrus on wikidatawiki as secondary index' | 
  [production] | 
            
  | 14:49 | 
  <cmjohnson1> | 
  dns update | 
  [production] | 
            
  | 13:15 | 
  <paravoid> | 
  reprepro: include backported graphite-carbon, graphite-web, python-whisper (plus deps python-django-tagging, flot) from saucy; replace custom-built python-whisper | 
  [production] | 
            
  | 12:09 | 
  <hashar> | 
  jenkins refreshing jobs to let them recurse in git submodules {{bug|55614}} {{gerrit|92511}} | 
  [production] | 
            
  | 11:56 | 
  <hashar> | 
  lanthanum.eqiad.wmnet running apt-get upgrade | 
  [production] | 
            
  | 11:36 | 
  <hashar> | 
  gerrit: creating integration/phpunit to hold PHPUnit files for deployment | 
  [production] | 
            
  | 07:13 | 
  <paravoid> | 
  "ln -s /apache /a" on terbium; foreachwiki and friends was broken (for a while) due to /usr/local/lib/mw-deployment-vars.sh pointing to non-existent /a | 
  [production] | 
            
  | 03:01 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Tue Oct 29 03:01:47 UTC 2013 | 
  [production] |