| 2019-08-29
      
      § | 
    
  | 08:41 | <mutante> | cp1085 - puppet run stuck after Loading facts, possibly related to ACKed IPMI sensor status issue in Icinga | [production] | 
            
  | 08:39 | <mutante> | cp1085 - kill stuck puppet processes and run manually | [production] | 
            
  | 08:36 | <marostegui> | Change min_replicas to 4 on s4 for eqiad and codfw T231019 | [production] | 
            
  | 08:30 | <marostegui> | Change min_replicas to 2 on s3 for eqiad and codfw T231019 | [production] | 
            
  | 08:26 | <mutante> | running puppet on cp-text_eqiad | [production] | 
            
  | 08:23 | <mutante> | switching iegreview app to stretch backend with TLS and discovery record | [production] | 
            
  | 08:23 | <kart_> | Updated cxserver to 2019-08-29-074757-production (T230200) | [production] | 
            
  | 08:21 | <@> | helmfile [EQIAD] Ran 'apply' command on namespace 'cxserver' for release 'production' . | [production] | 
            
  | 08:18 | <@> | helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . | [production] | 
            
  | 08:15 | <@> | helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . | [production] | 
            
  | 08:11 | <_joe_> | disabling zend GC on mw1347, testing an hypothesis for T231011 | [production] | 
            
  | 08:03 | <_joe_> | live tweak on mw1270: apc.ttl removed; apc size 4 GB; tideways disabled. | [production] | 
            
  | 05:00 | <marostegui> | Stop MySQL on db2053 for decommission T231407 | [production] | 
            
  | 04:59 | <marostegui> | Remove db2053 from tendril and zarcillo T231407 | [production] | 
            
  | 03:28 | <ejegg> | updated payments-wiki from 231b7b0850 to 1d5d7503b0 | [production] | 
            
  
    | 2019-08-28
      
      § | 
    
  | 23:17 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 8bfe43a: Add scielo.br to wgCopyUploadsDomains for commonswiki (T231402) (duration: 00m 55s) | [production] | 
            
  | 21:39 | <bd808> | Set downtime/ack for showmount on labstore1004 (T229448) | [production] | 
            
  | 21:03 | <ejegg> | deleted fredge_multiqueue_consumer process-control job | [production] | 
            
  | 19:39 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.20/includes/upload/UploadFromChunks.php: T231488 Speculatively hot-deploy fix ahead of landing in git (duration: 00m 54s) | [production] | 
            
  | 19:15 | <James_F> | Live hacking php-1.34.0-wmf.20/includes/upload/UploadFromChunks.php on mwdebug1002 for T231488 | [production] | 
            
  | 18:57 | <XioNoX> | update cloud firewall policies on cr1/2-eqiad - T231418 | [production] | 
            
  | 18:32 | <urandom> | rebooting restbase-dev1006 -- T229421 | [production] | 
            
  | 18:25 | <zfilipin@deploy1001> | Synchronized php: group1 wikis to 1.34.0-wmf.20 (duration: 00m 53s) | [production] | 
            
  | 18:24 | <zfilipin@deploy1001> | rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.20 | [production] | 
            
  | 17:42 | <XioNoX> | re-enable both sides of the reline link between knams and esams - T230448 | [production] | 
            
  | 17:14 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.20/extensions/Kartographer/includes/ApiQueryMapData.php: T231453 Fix array access as object (duration: 00m 54s) | [production] | 
            
  | 17:06 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.19/extensions/MobileFrontend/includes: T231014 Postpone call to MobileContext::shouldDisplayMobileView() (duration: 00m 55s) | [production] | 
            
  | 16:51 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Stop setting wgGraphIsTrusted (no longer used) (duration: 00m 56s) | [production] | 
            
  | 16:06 | <hashar> | upgrading Jenkins on contint1001 | [production] | 
            
  | 16:03 | <mutante> | imported new jenkins package to thirdparty/ci stretch-wikimedia | [production] | 
            
  | 16:01 | <hashar> | contint2001: upgraded Debian packages / Jenkins | [production] | 
            
  | 15:15 | <jeh> | restart puppetdb on compiler1002.puppet-diffs.eqiad.wmflabs | [production] | 
            
  | 14:35 | <mutante> | racktables - down for maintenance | [production] | 
            
  | 13:59 | <ema> | cp1075 ats-be repooled to resume testing T228629 | [production] | 
            
  | 13:58 | <ema@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=cp1075.eqiad.wmnet,service=ats-be | [production] | 
            
  | 13:38 | <zfilipin@deploy1001> | rebuilt and synchronized wikiversions files: Revert "group1 wikis to 1.34.0-wmf.20" | [production] | 
            
  | 13:30 | <zfilipin@deploy1001> | Synchronized php: group1 wikis to 1.34.0-wmf.20 (duration: 00m 55s) | [production] | 
            
  | 13:29 | <zfilipin@deploy1001> | rebuilt and synchronized wikiversions files: group1 wikis to 1.34.0-wmf.20 | [production] | 
            
  | 13:20 | <marostegui> | Change min_replicas to 4 on s8 for eqiad and codfw T231019 | [production] | 
            
  | 13:18 | <marostegui> | Change min_replicas to 3 on s6 for eqiad and codfw T231019 | [production] | 
            
  | 13:15 | <marostegui> | Optimize pc2010 after deleting old rows - T210725 | [production] | 
            
  | 12:16 | <hashar> | contint1001: manually gzip a few mw-debug-cli.log.gz files # T219850 | [production] | 
            
  | 12:06 | <Urbanecm> | Closing EU SWAT | [production] | 
            
  | 12:05 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 389919f: [rowiki] Allow sysops to name patrollers (T231099) (duration: 00m 53s) | [production] | 
            
  | 12:03 | <Urbanecm> | EU SWAT is taking few mins out of the sanity break, last patch | [production] | 
            
  | 12:01 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 34f1552: Disable search engine indexing in some namespaces of Icelandic Wikipedia (T231179) (duration: 00m 54s) | [production] | 
            
  | 11:57 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 2aebc15: Enable Page Previews as default on zhwikivoyage (T230624) (duration: 00m 52s) | [production] | 
            
  | 11:55 | <Urbanecm> | Purge /static/images/project-logos/specieswiki-1.5x.png and /static/images/project-logos/specieswiki-2x.png (T230113) | [production] | 
            
  | 11:54 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: f86baa3: Create HIDPI logo for Wikispecies (T230113) (duration: 00m 52s) | [production] | 
            
  | 11:52 | <urbanecm@deploy1001> | Synchronized static/images/project-logos/: SWAT: f86baa3: Create HIDPI logo for Wikispecies (1/2, T230113) (duration: 00m 54s) | [production] |