| 
      
        2017-03-16
      
      §
     | 
  
    
  | 11:01 | 
  <joal@tin> | 
  Started deploy [analytics/aqs/deploy@006bf8c]: (no justification provided) | 
  [production] | 
            
  | 10:59 | 
  <joal@tin> | 
  Finished deploy [analytics/aqs/deploy@006bf8c]: (no justification provided) (duration: 02m 13s) | 
  [production] | 
            
  | 10:56 | 
  <joal@tin> | 
  Started deploy [analytics/aqs/deploy@006bf8c]: (no justification provided) | 
  [production] | 
            
  | 10:12 | 
  <volans> | 
  upgraded cumin to version 0.0.2 in the repository and on neodymium/sarin | 
  [production] | 
            
  | 10:07 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1051 - T160415 (duration: 00m 41s) | 
  [production] | 
            
  | 09:56 | 
  <moritzm> | 
  installing libevent security updates | 
  [production] | 
            
  | 09:55 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1051 - T160415 (duration: 00m 42s) | 
  [production] | 
            
  | 09:46 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1055 - T160415 (duration: 00m 42s) | 
  [production] | 
            
  | 09:46 | 
  <moritzm> | 
  upgrading apache on cobalt/gerrit | 
  [production] | 
            
  | 09:38 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1055 - T160415 (duration: 00m 47s) | 
  [production] | 
            
  | 09:22 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1066 - T160415 (duration: 00m 42s) | 
  [production] | 
            
  | 09:15 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1066 - T160415 (duration: 00m 42s) | 
  [production] | 
            
  | 09:11 | 
  <moritzm> | 
  upgrading apache on fermium/lists.wikimedia.org | 
  [production] | 
            
  | 09:10 | 
  <moritzm> | 
  upgrading apache on mendelevium/OTRS | 
  [production] | 
            
  | 09:08 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1072 - T160415 (duration: 00m 42s) | 
  [production] | 
            
  | 09:03 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1072 - T160415 (duration: 00m 41s) | 
  [production] | 
            
  | 08:57 | 
  <godog> | 
  codfw-prod: add ms-be203[1-9] - T158337 | 
  [production] | 
            
  | 08:55 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1073 - T160415 (duration: 00m 41s) | 
  [production] | 
            
  | 08:51 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1073 - T160415 (duration: 00m 41s) | 
  [production] | 
            
  | 08:41 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1089 - T160415 (duration: 00m 43s) | 
  [production] | 
            
  | 08:33 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1089 - T160415 (duration: 00m 41s) | 
  [production] | 
            
  | 08:26 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1080 - T160415 (duration: 00m 46s) | 
  [production] | 
            
  | 08:20 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1080 - T160415 (duration: 00m 47s) | 
  [production] | 
            
  | 08:12 | 
  <moritzm> | 
  upgrading apache on einsteinium/icinga.wikimedia.org | 
  [production] | 
            
  | 07:51 | 
  <marostegui> | 
  Deploy schema change on s1 - T160415 | 
  [production] | 
            
  | 07:36 | 
  <marostegui> | 
  Deploy schema change on s7 - T160415 | 
  [production] | 
            
  | 07:08 | 
  <marostegui> | 
  Starting pt-table-checksum on s6 (frwiki) - T160509 | 
  [production] | 
            
  | 03:01 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Thu Mar 16 03:01:37 UTC 2017 (duration 5m 50s) | 
  [production] | 
            
  | 02:55 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.29.0-wmf.16) (duration: 13m 39s) | 
  [production] | 
            
  | 02:24 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.29.0-wmf.15) (duration: 08m 51s) | 
  [production] | 
            
  | 00:14 | 
  <eileen> | 
  updated civicrm from f1a3d647bc7d7fa4d7bbbff291d4a160ac8777c0 to cca5921a2a63346bb0d98dbaa5710952dafd1b21 | 
  [production] | 
            
  
    | 
      
        2017-03-15
      
      §
     | 
  
    
  | 23:36 | 
  <twentyafterfour> | 
  train unblocked and wmf.16 is deployed to group1 wikis. | 
  [production] | 
            
  | 23:32 | 
  <twentyafterfour@tin> | 
  Synchronized php-1.29.0-wmf.16/extensions/ApiFeatureUsage/ApiFeatureUsageQueryEngineElastica.php: deploy  I2d8603cef61723ddcc8c1d37566c334333422248 refs T160578 T158997 (duration: 00m 42s) | 
  [production] | 
            
  | 23:25 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:342696|Restrict page images to lead section on cawiki]] T152115 (duration: 00m 42s) | 
  [production] | 
            
  | 23:17 | 
  <thcipriani@tin> | 
  Synchronized wmf-config: SWAT: [[gerrit:342947|Set $wgOresExtension for I63b11eff3a4]] T159763 (duration: 00m 44s) | 
  [production] | 
            
  | 23:12 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:342684|Deploy PageViewInfo to group1]] T125917 (duration: 00m 43s) | 
  [production] | 
            
  | 22:51 | 
  <twentyafterfour@tin> | 
  Synchronized wmf-config/CirrusSearch-common.php: Deploy I4980daf73d5a98afedfae4a254a88ea078912296 refs T160569 and T158997 (duration: 00m 42s) | 
  [production] | 
            
  | 22:34 | 
  <mutante> | 
  Cassandra test hosts: deploy break-fix gerrit:342912 , run puppet on cerium and praseodymium. on xenon puppet is disabled. | 
  [production] | 
            
  | 21:54 | 
  <twentyafterfour@tin> | 
  Synchronized wmf-config/CirrusSearch-common.php: Deploy I67d71217d16cd75a2646cf658cead2c5fd740cd9 refs T160569 and T158997 (duration: 00m 42s) | 
  [production] | 
            
  | 21:52 | 
  <eileen> | 
  civicrm update from 639eb688db02911948814fe8749523145d1a2df6 to f1a3d647bc7d7fa4d7bbbff291d4a160ac8777c0 | 
  [production] | 
            
  | 21:29 | 
  <twentyafterfour@tin> | 
  Synchronized wmf-config: deploy Iad984924671b499aea19ef536ec6367e73a84a7c to fix 160569 and unblock the train refs T158997 (duration: 00m 49s) | 
  [production] | 
            
  | 21:01 | 
  <twentyafterfour@tin> | 
  Synchronized wmf-config: deploy I489c4aa1b862f053b0cb385d509f9ac5f8c6deca to fix 160569 and unblock the train refs T158997 (duration: 00m 45s) | 
  [production] | 
            
  | 20:54 | 
  <ladsgroup@tin> | 
  Finished deploy [ores/deploy@bc0bc74]: Mid-March deploy of ORES (T160279) (duration: 26m 46s) | 
  [production] | 
            
  | 20:44 | 
  <gehel> | 
  restarting postgresql on maps clusters - T160209 | 
  [production] | 
            
  | 20:38 | 
  <urandom> | 
  T111113: Restarting xenon (RESTBase Staging) to enable client encryption (canary) | 
  [production] | 
            
  | 20:34 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Move db1067 from s2 to s1 as a db1057 replacement (duration: 00m 42s) | 
  [production] | 
            
  | 20:30 | 
  <twentyafterfour> | 
  T160569 blocks the train until I can figure out what is causing it. The frequency is low so I haven't reverted to wmf.15, group 1 remains on wmf.16 refs T158997 | 
  [production] | 
            
  | 20:27 | 
  <ladsgroup@tin> | 
  Started deploy [ores/deploy@bc0bc74]: Mid-March deploy of ORES (T160279) | 
  [production] | 
            
  | 20:13 | 
  <bsitzmann@tin> | 
  Finished deploy [mobileapps/deploy@fa43048]: Update mobileapps to bb8fcf2 (duration: 03m 51s) | 
  [production] | 
            
  | 20:09 | 
  <bsitzmann@tin> | 
  Started deploy [mobileapps/deploy@fa43048]: Update mobileapps to bb8fcf2 | 
  [production] |