| 
      
        2019-10-02
      
      §
     | 
  
    
  | 11:28 | 
  <moritzm> | 
  installing cryptsetup bugfix from buster 10.1 point release | 
  [production] | 
            
  | 11:26 | 
  <Urbanecm> | 
  EU SWAT done | 
  [production] | 
            
  | 11:26 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: 01711d5: Enable partial blocks at ptwiki (T233754) (duration: 00m 55s) | 
  [production] | 
            
  | 11:26 | 
  <jbond42> | 
  update puppet.eqiad.wmnet to puppetmaster2001 | 
  [production] | 
            
  | 11:24 | 
  <jbond42> | 
  update puppet.esams.wmnet to puppetmaster2001 | 
  [production] | 
            
  | 11:20 | 
  <pmiazga@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:540107|Set new MFMobileFormatterOptions config using old config (T232690)]] (duration: 01m 01s) | 
  [production] | 
            
  | 11:15 | 
  <_joe_> | 
  testing the package on restbase-dev1006 | 
  [production] | 
            
  | 11:14 | 
  <_joe_> | 
  uploaded service-checker 0.2.0 to stretch-wikimedia | 
  [production] | 
            
  | 11:12 | 
  <pmiazga@deploy1001> | 
  Synchronized wmf-config/mobile.php: SWAT: [[gerrit:540193|Do not set wgMFNoindexPages config flag in mobile.php (T206497)]] (duration: 01m 14s) | 
  [production] | 
            
  | 10:17 | 
  <gehel@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) | 
  [production] | 
            
  | 10:17 | 
  <gehel@cumin1001> | 
  START - Cookbook sre.hosts.decommission | 
  [production] | 
            
  | 09:41 | 
  <moritzm> | 
  rebalancing Ganeti/codfw Row A after rolling reboot of Ganeti nodes | 
  [production] | 
            
  | 07:46 | 
  <moritzm> | 
  upgrading remaining stretch hosts to ferm 2.4.2pre | 
  [production] | 
            
  | 06:23 | 
  <marostegui> | 
  Fix replication on labsdb1011:s7 - T233986 | 
  [production] | 
            
  | 06:17 | 
  <marostegui> | 
  Fix replication on labsdb1011:s1 - T233986 | 
  [production] | 
            
  | 06:09 | 
  <marostegui@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | 
  [production] | 
            
  | 06:09 | 
  <marostegui@cumin1001> | 
  START - Cookbook sre.hosts.decommission | 
  [production] | 
            
  | 04:07 | 
  <vgutierrez> | 
  restarting trafficserver-tls on cp5007 | 
  [production] | 
            
  | 00:54 | 
  <ejegg> | 
  updated fundraising CiviCRM from 6d90d0cf06 to 12c5727a23 | 
  [production] | 
            
  | 00:34 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.34.0-wmf.25/resources/src: 5eb3ae1e888e353 (duration: 01m 00s) | 
  [production] | 
            
  | 00:30 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.34.0-wmf.25/skins/Vector/: d30064229f9 (duration: 00m 59s) | 
  [production] | 
            
  
    | 
      
        2019-10-01
      
      §
     | 
  
    
  | 23:46 | 
  <ebernhardson@deploy1001> | 
  Synchronized php-1.34.0-wmf.25/extensions/VisualEditor/includes/ApiVisualEditor.php: T233127: ApiVisualEditor: Add logging for RESTBase HTTP errors (duration: 00m 58s) | 
  [production] | 
            
  | 23:44 | 
  <ebernhardson@deploy1001> | 
  Synchronized php-1.34.0-wmf.24/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: T233211: Deploy cirrussearch glent m0 a/b test (duration: 00m 59s) | 
  [production] | 
            
  | 23:43 | 
  <ebernhardson@deploy1001> | 
  Synchronized php-1.34.0-wmf.25/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: T233211: Deploy cirrussearch glent m0 a/b test (duration: 00m 59s) | 
  [production] | 
            
  | 23:28 | 
  <mutante> | 
  cobalt (gerrit) rsyncing /srv/gerrit/plugins dir, push to new server gerrit1001 (T222391) | 
  [production] | 
            
  | 23:21 | 
  <mutante> | 
  gerrit1001 - chown -R gerrit2:gerrit2 /srv/gerrit/git/ (T222391) | 
  [production] | 
            
  | 23:20 | 
  <ebernhardson@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T233211: CirrusSearch: Configuration for glent m0 AB test (duration: 00m 58s) | 
  [production] | 
            
  | 23:12 | 
  <ebernhardson@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T233127: Add VisualEditor logging channel to wmgMonologChannels (duration: 00m 59s) | 
  [production] | 
            
  | 22:30 | 
  <ayounsi@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.rotate-pdu-password (exit_code=1) | 
  [production] | 
            
  | 22:19 | 
  <ayounsi@cumin1001> | 
  START - Cookbook sre.hosts.rotate-pdu-password | 
  [production] | 
            
  | 21:34 | 
  <godog> | 
  swift codfw-prod: add ms-be2051 with minimal weight - T233638 T222366 | 
  [production] | 
            
  | 21:33 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.34.0-wmf.25/skins/Vector/: bb2fd9cf9c22cc (duration: 01m 00s) | 
  [production] | 
            
  | 21:29 | 
  <ayounsi@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.rotate-pdu-password (exit_code=1) | 
  [production] | 
            
  | 21:29 | 
  <ayounsi@cumin1001> | 
  START - Cookbook sre.hosts.rotate-pdu-password | 
  [production] | 
            
  | 20:11 | 
  <ayounsi@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.rotate-pdu-password (exit_code=1) | 
  [production] | 
            
  | 20:10 | 
  <ayounsi@cumin1001> | 
  START - Cookbook sre.hosts.rotate-pdu-password | 
  [production] | 
            
  | 19:58 | 
  <mutante> | 
  cobalt (gerrit) - rsyncing gerrit data to gerrit1001 in a screen session (T222391) | 
  [production] | 
            
  | 19:47 | 
  <ayounsi@cumin1001> | 
  END (ERROR) - Cookbook sre.hosts.rotate-pdu-password (exit_code=97) | 
  [production] | 
            
  | 19:47 | 
  <ayounsi@cumin1001> | 
  START - Cookbook sre.hosts.rotate-pdu-password | 
  [production] | 
            
  | 19:42 | 
  <marxarelli> | 
  1.34.0-wmf.25 promoted to group0 cc: T220750. no rise in relevant error rates | 
  [production] | 
            
  | 19:34 | 
  <dduvall@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 to 1.34.0-wmf.25 | 
  [production] | 
            
  | 19:30 | 
  <marxarelli> | 
  promoting 1.34.0-wmf.25 to group0 | 
  [production] | 
            
  | 19:28 | 
  <dduvall@deploy1001> | 
  Finished scap: testwiki to php-1.34.0-wmf.25 and rebuild l10n cache (duration: 19m 31s) | 
  [production] | 
            
  | 19:08 | 
  <dduvall@deploy1001> | 
  Started scap: testwiki to php-1.34.0-wmf.25 and rebuild l10n cache | 
  [production] | 
            
  | 19:07 | 
  <dduvall@deploy1001> | 
  Pruned MediaWiki: 1.34.0-wmf.23 (duration: 01m 32s) | 
  [production] | 
            
  | 19:03 | 
  <dduvall@deploy1001> | 
  Pruned MediaWiki: 1.34.0-wmf.22 (duration: 01m 41s) | 
  [production] | 
            
  | 19:02 | 
  <dduvall@deploy1001> | 
  Pruned MediaWiki: 1.34.0-wmf.21 (duration: 01m 57s) | 
  [production] | 
            
  | 19:01 | 
  <ayounsi@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.rotate-pdu-password (exit_code=1) | 
  [production] | 
            
  | 19:00 | 
  <ayounsi@cumin1001> | 
  START - Cookbook sre.hosts.rotate-pdu-password | 
  [production] | 
            
  | 18:59 | 
  <dduvall@deploy1001> | 
  Pruned MediaWiki: 1.34.0-wmf.20 (duration: 02m 11s) | 
  [production] |