| 2015-06-12
      
      § | 
    
  | 12:11 | <jynus> | restarting mariadb at labsdb1003 | [production] | 
            
  | 11:59 | <moritzm> | updated rdb200* to 3.19 kernel | [production] | 
            
  | 11:31 | <jynus> | db2068 up but all services and console login unresponsive, powercycling | [production] | 
            
  | 10:06 | <springle> | killed a bunch of queries hammering labsdb1003 for days | [production] | 
            
  | 09:58 | <moritzm> | updated mc2004 to mc2016 to 3.19 kernel | [production] | 
            
  | 06:06 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri Jun 12 06:06:55 UTC 2015 (duration 6m 54s) | [production] | 
            
  | 04:37 | <ori> | Synchronized php-1.26wmf9/extensions/FlaggedRevs: I4cfb47b41: Avoid post-redirect parse for certain edits (duration: 00m 14s) | [production] | 
            
  | 02:40 | <LocalisationUpdate> | completed (1.26wmf9) at 2015-06-12 02:40:36+00:00 | [production] | 
            
  | 02:34 | <l10nupdate> | Synchronized php-1.26wmf9/cache/l10n: (no message) (duration: 10m 00s) | [production] | 
            
  | 00:40 | <legoktm> | Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/217759 (duration: 00m 15s) | [production] | 
            
  | 00:07 | <catrope> | Synchronized wmf-config/InitialiseSettings-labs.php: (no message) (duration: 00m 14s) | [production] | 
            
  
    | 2015-06-11
      
      § | 
    
  | 23:59 | <legoktm> | Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/217753 (duration: 00m 16s) | [production] | 
            
  | 23:54 | <ori> | Synchronized php-1.26wmf9/includes/EditPage.php: cf7df757f2: Instrument edit failures (duration: 00m 14s) | [production] | 
            
  | 23:41 | <ebernhardson> | Synchronized php-1.26wmf9/extensions/MobileFrontend: Bump MobileFrontend in 1.26wmf9 for SWAT (duration: 00m 14s) | [production] | 
            
  | 23:41 | <ejegg> | updated civicrm from 7ffe0cefb019828a09c9369187f14518847b5f41 to d13aaa4e9e937b0b1ae1f5de61ea7ff1f316d58f | [production] | 
            
  | 23:25 | <ebernhardson> | Synchronized php-1.26wmf9/extensions/CirrusSearch/: Fix prefer-recent queries in cirrussearch (duration: 00m 13s) | [production] | 
            
  | 23:02 | <ejegg> | updated SmashPig on the rest of the cluster from 477e8a8be5ea895262031c147330de5a651cc3ac to 7fed22ad933a6d3e371d60dfc6f8fdd0f9131510 | [production] | 
            
  | 22:17 | <godog> | temporary bump php memory_limit on magnesium to test T102092 | [production] | 
            
  | 22:11 | <ejegg> | updated SmashPig on payments-listener from 477e8a8be5ea895262031c147330de5a651cc3ac to 7fed22ad933a6d3e371d60dfc6f8fdd0f9131510 | [production] | 
            
  | 21:54 | <ori> | Widespread TC cache exhaustion again, doing rolling restart of HHVMs | [production] | 
            
  | 21:46 | <ori> | Synchronized wmf-config/CommonSettings.php: I3d3ed7647: Test LCStoreStaticArray on test2wiki (duration: 00m 14s) | [production] | 
            
  | 21:01 | <godog> | NPE while trying to make restbase1007 (cassandra 2.1.5) join the cluster, trying matching the same cassandra version (2.1.3) | [production] | 
            
  | 20:57 | <krenair> | Synchronized wmf-config/InitialiseSettings.php: fix last commit, did not have any affect (duration: 00m 16s) | [production] | 
            
  | 20:55 | <ejegg> | updated payments from 43c7952d2a31deaea97e8319f5612d644dce43c8 to f33d0a8687a120a2057a7e6acad67da63b17f97e | [production] | 
            
  | 20:54 | <krenair> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/217688/1 (duration: 00m 13s) | [production] | 
            
  | 20:11 | <godog> | sign restbase1007 puppet key and first puppet run | [production] | 
            
  | 19:10 | <legoktm> | Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/217591 (duration: 00m 13s) | [production] | 
            
  | 18:58 | <legoktm> | Synchronized wmf-config/InitialiseSettings-labs.php: beta only change - https://gerrit.wikimedia.org/r/217560 (duration: 00m 12s) | [production] | 
            
  | 18:55 | <krinkle> | Synchronized php-1.26wmf9/extensions/WikimediaEvents: T101806 (duration: 00m 14s) | [production] | 
            
  | 18:43 | <twentyafterfour> | Synchronized php-1.26wmf9/includes/AjaxResponse.php: Hotfix Iafff9982bbbee893c13f891901dde88f998db7a6 (duration: 00m 14s) | [production] | 
            
  | 18:16 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: all wikis to 1.26wmf9 | [production] | 
            
  | 17:44 | <ejegg> | rolled back payments to 43c7952d2a31deaea97e8319f5612d644dce43c8 | [production] | 
            
  | 17:41 | <ejegg> | updated payments from 43c7952d2a31deaea97e8319f5612d644dce43c8 to 15f24d24b150d5d774314b0c1b40ae26a73185f2 | [production] | 
            
  | 17:00 | <moritzm> | updated mc200[1-3] to linux 3.19 | [production] | 
            
  | 16:28 | <aude> | Synchronized wmf-config/InitialiseSettings.php: Use arbitrary access tag (duration: 00m 12s) | [production] | 
            
  | 16:27 | <aude> | Synchronized wmf-config/CommonSettings.php: Add arbitrary access group tag (duration: 00m 13s) | [production] | 
            
  | 16:27 | <aude> | Synchronized arbitraryaccess.dblist: Add dblist for arbitrary access wikis (duration: 00m 13s) | [production] | 
            
  | 16:24 | <aude> | Synchronized wmf-config/InitialiseSettings.php: Use usagetracking tag (duration: 00m 13s) | [production] | 
            
  | 16:24 | <aude> | Synchronized wmf-config/CommonSettings.php: Add usagetracking group tag (duration: 00m 16s) | [production] | 
            
  | 16:23 | <ori> | Scap + deployments exhausted TC cache on Apaches; performed a rolling restart of HHVM | [production] | 
            
  | 16:21 | <aude> | Synchronized usagetracking.dblist: Add dblist for usage tracking wikis (duration: 00m 25s) | [production] | 
            
  | 16:19 | <ori> | Synchronized wmf-config/CommonSettings.php: Disable Parsoid update jobs (duration: 00m 14s) | [production] | 
            
  | 16:18 | <thcipriani> | Finished scap: SWAT: Update namespaces and special pages for Northern Luri (lrc) from translatewiki [[gerrit:216533]] [[gerrit:217327]] (duration: 32m 11s) | [production] | 
            
  | 15:46 | <thcipriani> | Started scap: SWAT: Update namespaces and special pages for Northern Luri (lrc) from translatewiki [[gerrit:216533]] [[gerrit:217327]] | [production] | 
            
  | 15:27 | <thcipriani> | Synchronized php-1.26wmf9/extensions/OpenStackManager: SWAT: update OpenStackManager to disable unused sudoer features [[gerrit:217407]] (duration: 00m 13s) | [production] | 
            
  | 15:11 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Make VisualEditor access RESTbase directly on all public wikis [[gerrit:214833]] (duration: 00m 12s) | [production] | 
            
  | 15:05 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT: CX: Add wikis for deployment on 20150611 [[gerrit:217460 ]] (duration: 00m 12s) | [production] | 
            
  | 14:33 | <aude> | Synchronized wmf-config/InitialiseSettings.php: Enable usage tracking on jawiki (duration: 00m 12s) | [production] | 
            
  | 13:40 | <_joe_> | rolling restart of all the restbase instances | [production] | 
            
  | 13:33 | <aude> | Synchronized wmf-config/InitialiseSettings.php: Enable usage tracking on frwiki (duration: 00m 12s) | [production] |