| 
      
        2018-09-18
      
      §
     | 
  
    
  | 23:21 | 
  <twentyafterfour@deploy1001> | 
  Synchronized wmf-config/: SWAT: tested on mwdebug1001, now syncing wmf-config settings to the whole cluster (duration: 00m 59s) | 
  [production] | 
            
  | 23:15 | 
  <twentyafterfour> | 
  SWAT: deploying 3 patches for ebernhardson: 77deae12f 7009fe473 and 12cc420fc | 
  [production] | 
            
  | 22:10 | 
  <bblack> | 
  depooled ulsfo, some uknown router issue | 
  [production] | 
            
  | 21:42 | 
  <bblack> | 
  authdns1001 seems stable/fine running gdnsd-2.99.9-beta so far.  If issues crop up later, don't hesitate to (a) downgrade back to stretch-backports gdnsd-2.3.0-1~bpo9+1 or (b) call me! | 
  [production] | 
            
  | 21:25 | 
  <bblack> | 
  authdns1001: testing gdnsd version update (2.99.9-beta) | 
  [production] | 
            
  | 20:58 | 
  <catrope@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Set ActorTableSchemaMigrationStage back to OLD on test wikis, mediawikiwiki (T188327, T204669) (duration: 00m 57s) | 
  [production] | 
            
  | 20:57 | 
  <catrope@deploy1001> | 
  sync-file aborted: Set ActorTableSchemaMigrationStage back to OLD on test wikis, mediawikiwiki (duration: 00m 10s) | 
  [production] | 
            
  | 20:05 | 
  <hashar@deploy1001> | 
  Synchronized php-1.32.0-wmf.22/extensions/OAuth/frontend/specialpages/SpecialMWOAuthListConsumers.php: Fix escapeForHtml method name - T204757 (duration: 00m 58s) | 
  [production] | 
            
  | 19:56 | 
  <ppchelko@deploy1001> | 
  Finished deploy [cpjobqueue/deploy@58f9ed3]: Log errors for HTTP error T203929 (duration: 00m 49s) | 
  [production] | 
            
  | 19:55 | 
  <ppchelko@deploy1001> | 
  Started deploy [cpjobqueue/deploy@58f9ed3]: Log errors for HTTP error T203929 | 
  [production] | 
            
  | 19:43 | 
  <bblack> | 
  uploaded gdnsd-2.99.9-beta1-1+wmf1 to reprepro for stretch-wikimedia | 
  [production] | 
            
  | 19:20 | 
  <hashar@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 to 1.32.0-wmf.22 | 
  [production] | 
            
  | 19:18 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@4c3128f] (dev-cluster): Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out (duration: 06m 32s) | 
  [production] | 
            
  | 19:11 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@4c3128f] (dev-cluster): Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out | 
  [production] | 
            
  | 18:49 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out (duration: 06m 14s) | 
  [production] | 
            
  | 18:43 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out | 
  [production] | 
            
  | 18:40 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out (duration: 07m 58s) | 
  [production] | 
            
  | 18:32 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints, take 2, feed timing out | 
  [production] | 
            
  | 18:31 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints (duration: 08m 55s) | 
  [production] | 
            
  | 18:22 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@4c3128f]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements, don't monitor PDF endpoints | 
  [production] | 
            
  | 18:08 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@55100d4]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements (duration: 03m 37s) | 
  [production] | 
            
  | 18:04 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@55100d4]: Remove restrictions table T203835, Split 25% PDF traffic to Proton T186748, Metrics endpoints improvements | 
  [production] | 
            
  | 17:29 | 
  <mutante> | 
  scandium - move from role(spare) to role(parsoid_testing), making it equal to ruthenium (T201366) | 
  [production] | 
            
  | 16:45 | 
  <addshore@deploy1001> | 
  Synchronized php-1.32.0-wmf.22/includes/watcheditem/WatchedItemStore.php: [[gerrit:461155|WatchedItemStore::countVisitingWatchersMultiple() fix]] T204729 (duration: 00m 57s) | 
  [production] | 
            
  | 16:43 | 
  <addshore@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/includes/watcheditem/WatchedItemStore.php: [[gerrit:461154|WatchedItemStore::countVisitingWatchersMultiple() fix]] T204729 (duration: 00m 59s) | 
  [production] | 
            
  | 16:33 | 
  <XioNoX> | 
  delete `filter common-infrastructure4` on cr1/2-eqiad, unused/obsolete after T198623 | 
  [production] | 
            
  | 16:33 | 
  <mutante> | 
  mwmaint2001 - nagios-nrpe-server had status 'failed' and caused all NRPE Icinga checks to fail but recovered after simply starting it again | 
  [production] | 
            
  | 16:32 | 
  <mutante> | 
  radon - re-enabled disabled puppet without reason (decom) T202040 | 
  [production] | 
            
  | 16:32 | 
  <mutante> | 
  mwmaint2001 - starting nagios-nrpe-server | 
  [production] | 
            
  | 16:22 | 
  <XioNoX> | 
  delete `term cumin` from cr1/2-eqiad analytics filter (already permited by established-tcp term) | 
  [production] | 
            
  | 16:02 | 
  <moritzm> | 
  installing policykit-1 security updates on jessie | 
  [production] | 
            
  | 15:55 | 
  <elukey@deploy1001> | 
  Finished deploy [analytics/refinery@1a6235a]: Fix cron scrips from the NYC offsite (duration: 09m 32s) | 
  [production] | 
            
  | 15:53 | 
  <XioNoX> | 
  add cumin2001 to mr* security policies - T204730 | 
  [production] | 
            
  | 15:47 | 
  <moritzm> | 
  installing spice security updates | 
  [production] | 
            
  | 15:46 | 
  <elukey@deploy1001> | 
  Started deploy [analytics/refinery@1a6235a]: Fix cron scrips from the NYC offsite | 
  [production] | 
            
  | 15:26 | 
  <XioNoX> | 
  add cumin2001 to labs-in4 on cr1/2-eqiad | 
  [production] | 
            
  | 15:23 | 
  <hashar@deploy1001> | 
  Finished scap: Sync again testwiki to php-1.32.0-wmf.20, I might have screwed up l10ncache (duration: 06m 31s) | 
  [production] | 
            
  | 15:20 | 
  <hashar> | 
  hashar scap for testwiki is actually php-1.32.0-wmf.22 | 
  [production] | 
            
  | 15:19 | 
  <mepps> | 
  updated process-control to c03857f6a6 | 
  [production] | 
            
  | 15:17 | 
  <hashar@deploy1001> | 
  Started scap: Sync again testwiki to php-1.32.0-wmf.20, I might have screwed up l10ncache | 
  [production] | 
            
  | 15:15 | 
  <mepps> | 
  updated process-control to f9ab984c5d | 
  [production] | 
            
  | 15:07 | 
  <hashar@deploy1001> | 
  Finished scap: testwiki to php-1.32.0-wmf.20 (duration: 35m 46s) | 
  [production] | 
            
  | 14:31 | 
  <hashar@deploy1001> | 
  Started scap: testwiki to php-1.32.0-wmf.20 | 
  [production] |