| 2019-04-24
      
      § | 
    
  | 11:07 | <ladsgroup@deploy1001> | Started deploy [ores/deploy@060fc37]: (no justification provided) | [production] | 
            
  | 10:28 | <akosiaris@deploy1001> | scap-helm cxserver finished | [production] | 
            
  | 10:28 | <akosiaris@deploy1001> | scap-helm cxserver cluster staging completed | [production] | 
            
  | 10:28 | <akosiaris@deploy1001> | scap-helm cxserver upgrade -f cxserver-staging-values.yaml staging stable/cxserver [namespace: cxserver, clusters: staging] | [production] | 
            
  | 10:23 | <jijiki> | Restarting php-fpm on mw1238 for 505383 and T211488 | [production] | 
            
  | 09:58 | <moritzm> | installing rsync security updates on jessie | [production] | 
            
  | 08:44 | <moritzm> | rolling restart of Cassandra on restbase/codfw to pick up Java security update | [production] | 
            
  | 08:29 | <godog> | swift eqiad-prod: start decom for ms-be101[45] - T220590 | [production] | 
            
  | 08:17 | <godog> | bounce prometheus on bast5001 after migration and backfill | [production] | 
            
  | 08:04 | <gehel@cumin1001> | END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) | [production] | 
            
  | 08:04 | <gehel@cumin1001> | START - Cookbook sre.elasticsearch.force-shard-allocation | [production] | 
            
  | 08:02 | <gehel@cumin1001> | END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) | [production] | 
            
  | 08:02 | <gehel@cumin1001> | START - Cookbook sre.elasticsearch.force-shard-allocation | [production] | 
            
  | 06:41 | <marostegui> | Optimize tables on pc1010 | [production] | 
            
  | 06:38 | <elukey> | restart pdfrender on scb1003 | [production] | 
            
  | 06:37 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2082 (duration: 00m 52s) | [production] | 
            
  | 06:22 | <marostegui> | Upgrade db2082 | [production] | 
            
  | 06:22 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2079, depool db2082 (duration: 00m 55s) | [production] | 
            
  | 06:18 | <marostegui> | Upgrade db2081 | [production] | 
            
  | 06:10 | <marostegui> | Upgrade db2079 | [production] | 
            
  | 06:10 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2086, depool db2079 (duration: 00m 53s) | [production] | 
            
  | 05:55 | <marostegui> | Upgrade db2086 | [production] | 
            
  | 05:55 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2083 and depool db2086 (duration: 00m 52s) | [production] | 
            
  | 05:38 | <marostegui> | Upgrade db2080 and db2083 | [production] | 
            
  | 05:37 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2080 and db2083 (duration: 00m 54s) | [production] | 
            
  | 03:45 | <SMalyshev> | repooled wdqs1003, it's good now | [production] | 
            
  | 01:26 | <eileen> | jobs restarted  process-control config revision is ef6d4761e5 | [production] | 
            
  | 01:06 | <eileen> | civicrm revision changed from 31982324b8 to 468f85e524, config revision is 13b9eefe7b | [production] | 
            
  | 01:02 | <eileen> | process-control config revision is 13b9eefe7b | [production] | 
            
  | 00:29 | <mutante> | mw1297 - rebooting for nutcracker issue | [production] | 
            
  | 00:28 | <mutante> | mw1297 - scap pull | [production] | 
            
  | 00:08 | <mutante> | DNS - add initiatives.wikimedia.org (and initiaves.m) for campaign wiki requested at T167375 | [production] | 
            
  
    | 2019-04-23
      
      § | 
    
  | 23:51 | <mutante> | mw1297 - initial puppet run - will show up in Icinga in a little while but not pooled yet.. all the things are being installed right now | [production] | 
            
  | 23:48 | <ejegg> | updated payments-wiki (inactive cluster) from 7a312e371a to aa8dad50e7 | [production] | 
            
  | 23:39 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/GrowthExperiments/modules/homepage/ext.growthExperiments.Homepage.Logger.js: SWAT GrowthExperiments: Fix validation errors due to state='' (duration: 00m 53s) | [production] | 
            
  | 23:38 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/GrowthExperiments/includes/EventLogging/SpecialHomepageLogger.php: SWAT GrowthExperiments: Fix EventLogging errors (duration: 00m 53s) | [production] | 
            
  | 23:25 | <mutante> | generating mcrouter certs for appservers, added mw1297.eqiad.wmnet (T192457) | [production] | 
            
  | 23:23 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/languages/Language.php: SWAT T219728 Add support for new Japanese era name 'Reiwa' (duration: 00m 52s) | [production] | 
            
  | 23:20 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.Target.js: SWAT T221668 VisualEditor: Restore external paste sanitization of DOM elements (duration: 00m 55s) | [production] | 
            
  | 23:09 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT T221521 Add autoreviewer to wgRestrictionLevels on ptwikinews (duration: 00m 54s) | [production] | 
            
  | 22:35 | <XioNoX> | push firewall rule to pfw3-eqiad - T221475 | [production] | 
            
  | 22:33 | <XioNoX> | push firewall rule to pfw3-codfw - T221475 | [production] | 
            
  | 21:53 | <reedy@deploy1001> | Synchronized php-1.34.0-wmf.1/extensions/ORES/includes/Specials/SpecialORESModels.php: T221696 (duration: 00m 55s) | [production] | 
            
  | 21:43 | <robh@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 21:43 | <robh@cumin1001> | START - Cookbook sre.hosts.decommission | [production] | 
            
  | 21:33 | <thcipriani> | restarting gerrit to pickup config changes | [production] | 
            
  | 20:55 | <smalyshev@deploy1001> | Finished deploy [wdqs/wdqs@51b4728]: Deploy new Updater fix for cnstraints (T221407) (duration: 13m 03s) | [production] | 
            
  | 20:43 | <andrewbogott> | updating designate pools on cloudservices1003 and 1004 using eqiad1_pool_config.yml template from the puppet repo | [production] | 
            
  | 20:42 | <smalyshev@deploy1001> | Started deploy [wdqs/wdqs@51b4728]: Deploy new Updater fix for cnstraints (T221407) | [production] | 
            
  | 20:26 | <urandom> | dropping disused restbase keyspaces -- T221530 | [production] |