| 2019-04-24
      
      § | 
    
  | 08:04 | <gehel@cumin1001> | START - Cookbook sre.elasticsearch.force-shard-allocation | [production] | 
            
  | 08:02 | <gehel@cumin1001> | END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) | [production] | 
            
  | 08:02 | <gehel@cumin1001> | START - Cookbook sre.elasticsearch.force-shard-allocation | [production] | 
            
  | 06:41 | <marostegui> | Optimize tables on pc1010 | [production] | 
            
  | 06:38 | <elukey> | restart pdfrender on scb1003 | [production] | 
            
  | 06:37 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2082 (duration: 00m 52s) | [production] | 
            
  | 06:22 | <marostegui> | Upgrade db2082 | [production] | 
            
  | 06:22 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2079, depool db2082 (duration: 00m 55s) | [production] | 
            
  | 06:18 | <marostegui> | Upgrade db2081 | [production] | 
            
  | 06:10 | <marostegui> | Upgrade db2079 | [production] | 
            
  | 06:10 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2086, depool db2079 (duration: 00m 53s) | [production] | 
            
  | 05:55 | <marostegui> | Upgrade db2086 | [production] | 
            
  | 05:55 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2083 and depool db2086 (duration: 00m 52s) | [production] | 
            
  | 05:38 | <marostegui> | Upgrade db2080 and db2083 | [production] | 
            
  | 05:37 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2080 and db2083 (duration: 00m 54s) | [production] | 
            
  | 03:45 | <SMalyshev> | repooled wdqs1003, it's good now | [production] | 
            
  | 01:26 | <eileen> | jobs restarted  process-control config revision is ef6d4761e5 | [production] | 
            
  | 01:06 | <eileen> | civicrm revision changed from 31982324b8 to 468f85e524, config revision is 13b9eefe7b | [production] | 
            
  | 01:02 | <eileen> | process-control config revision is 13b9eefe7b | [production] | 
            
  | 00:29 | <mutante> | mw1297 - rebooting for nutcracker issue | [production] | 
            
  | 00:28 | <mutante> | mw1297 - scap pull | [production] | 
            
  | 00:08 | <mutante> | DNS - add initiatives.wikimedia.org (and initiaves.m) for campaign wiki requested at T167375 | [production] | 
            
  
    | 2019-04-23
      
      § | 
    
  | 23:51 | <mutante> | mw1297 - initial puppet run - will show up in Icinga in a little while but not pooled yet.. all the things are being installed right now | [production] | 
            
  | 23:48 | <ejegg> | updated payments-wiki (inactive cluster) from 7a312e371a to aa8dad50e7 | [production] | 
            
  | 23:39 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/GrowthExperiments/modules/homepage/ext.growthExperiments.Homepage.Logger.js: SWAT GrowthExperiments: Fix validation errors due to state='' (duration: 00m 53s) | [production] | 
            
  | 23:38 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/GrowthExperiments/includes/EventLogging/SpecialHomepageLogger.php: SWAT GrowthExperiments: Fix EventLogging errors (duration: 00m 53s) | [production] | 
            
  | 23:25 | <mutante> | generating mcrouter certs for appservers, added mw1297.eqiad.wmnet (T192457) | [production] | 
            
  | 23:23 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/languages/Language.php: SWAT T219728 Add support for new Japanese era name 'Reiwa' (duration: 00m 52s) | [production] | 
            
  | 23:20 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.Target.js: SWAT T221668 VisualEditor: Restore external paste sanitization of DOM elements (duration: 00m 55s) | [production] | 
            
  | 23:09 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT T221521 Add autoreviewer to wgRestrictionLevels on ptwikinews (duration: 00m 54s) | [production] | 
            
  | 22:35 | <XioNoX> | push firewall rule to pfw3-eqiad - T221475 | [production] | 
            
  | 22:33 | <XioNoX> | push firewall rule to pfw3-codfw - T221475 | [production] | 
            
  | 21:53 | <reedy@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/ORES/includes/Specials/SpecialORESModels.php: T221696 (duration: 00m 55s) | [production] | 
            
  | 21:43 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 21:43 | <robh@cumin1001> | START - Cookbook sre.hosts.decommission | [production] | 
            
  | 21:33 | <thcipriani> | restarting gerrit to pickup config changes | [production] | 
            
  | 20:55 | <smalyshev@deploy1001> | Finished deploy [wdqs/wdqs@51b4728]: Deploy new Updater fix for cnstraints (T221407) (duration: 13m 03s) | [production] | 
            
  | 20:43 | <andrewbogott> | updating designate pools on cloudservices1003 and 1004 using eqiad1_pool_config.yml template from the puppet repo | [production] | 
            
  | 20:42 | <smalyshev@deploy1001> | Started deploy [wdqs/wdqs@51b4728]: Deploy new Updater fix for cnstraints (T221407) | [production] | 
            
  | 20:26 | <urandom> | dropping disused restbase keyspaces -- T221530 | [production] | 
            
  | 19:57 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 19:57 | <robh@cumin1001> | START - Cookbook sre.hosts.decommission | [production] | 
            
  | 19:32 | <mutante> | webperf* - running puppet to git pull docroot | [production] | 
            
  | 19:11 | <thcipriani> | gerrit restart | [production] | 
            
  | 18:59 | <krinkle@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/MassMessage: c640195 (duration: 00m 56s) | [production] | 
            
  | 18:09 | <SMalyshev> | depool wdqs1003 to let it catch up | [production] | 
            
  | 18:03 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 18:03 | <robh@cumin1001> | START - Cookbook sre.hosts.decommission | [production] | 
            
  | 18:03 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 18:02 | <robh@cumin1001> | START - Cookbook sre.hosts.decommission | [production] |