| 2018-10-02
      
      ยง | 
    
  | 17:37 | <XioNoX> | update NAT for frbast2001 on pfw3-codfw - T204271 | [production] | 
            
  | 17:25 | <XioNoX> | update fw policies on pfw3-eqiad - T204271 | [production] | 
            
  | 17:22 | <XioNoX> | update fw policies on pfw3-codfw - T204271 | [production] | 
            
  | 17:22 | <andrewbogott> | upgraded wikitech-static to remotes/origin/REL1_31 | [production] | 
            
  | 17:18 | <andrewbogott> | upgrading debian packages and MediaWiki version on wikitech-static | [production] | 
            
  | 16:53 | <jynus> | setup test s3 replication channel on db1110 (filtered) | [production] | 
            
  | 16:49 | <XioNoX> | assign 10.195.0.129/29 to pfw3-codfw:reth0.2133 - T204271 | [production] | 
            
  | 16:38 | <cmjohnson1> | swapping failed disk db1067 T205780 | [production] | 
            
  | 16:04 | <ppchelko@deploy1001> | Finished deploy [cpjobqueue/deploy@093551f]: Increase cirrusSearchLinksUpdate concurrency (duration: 01m 06s) | [production] | 
            
  | 16:03 | <ppchelko@deploy1001> | Started deploy [cpjobqueue/deploy@093551f]: Increase cirrusSearchLinksUpdate concurrency | [production] | 
            
  | 15:50 | <marxarelli> | cutting 1.32.0-wmf.24 branch | [production] | 
            
  | 15:33 | <gehel> | cleanup old cronjob (cleanup GC logs) on all elasticsearch servers | [production] | 
            
  | 15:24 | <akosiaris> | upgrade mathoid chart version to 0.0.11 | [production] | 
            
  | 15:24 | <akosiaris@deploy1001> | scap-helm mathoid finished | [production] | 
            
  | 15:23 | <akosiaris@deploy1001> | scap-helm mathoid cluster codfw completed | [production] | 
            
  | 15:23 | <akosiaris@deploy1001> | scap-helm mathoid cluster eqiad completed | [production] | 
            
  | 15:23 | <akosiaris@deploy1001> | scap-helm mathoid upgrade production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] | [production] | 
            
  | 15:21 | <akosiaris@deploy1001> | scap-helm mathoid finished | [production] | 
            
  | 15:21 | <akosiaris@deploy1001> | scap-helm mathoid cluster codfw completed | [production] | 
            
  | 15:21 | <akosiaris@deploy1001> | scap-helm mathoid cluster eqiad completed | [production] | 
            
  | 15:21 | <akosiaris@deploy1001> | scap-helm mathoid upgrade -h [namespace: mathoid, clusters: eqiad,codfw] | [production] | 
            
  | 14:11 | <banyek> | powering off dbstore2002.codfw.wmnet for BBU change (T205257) | [production] | 
            
  | 13:46 | <marostegui> | Deploy schema change on s4 eqiad, this will generate lag on eqiad - T205913 | [production] | 
            
  | 13:06 | <marostegui> | Deploy schema change on s7 eqiad, this will generate lag on eqiad - T205913 | [production] | 
            
  | 12:47 | <banyek> | converting enwiki.content to TokuDB on host dbstrore1002 (T205544) | [production] | 
            
  | 12:47 | <banyek> | converting enwiki.contents to TokuDB on host dbstrore1002 (T205544) | [production] | 
            
  | 11:58 | <banyek> | converting wikidatawiki.slots to TokuDB on host dbstrore1002 (T205544) | [production] | 
            
  | 11:41 | <arturo> | downtime labstore1007 load check in icinga for 1d | [production] | 
            
  | 11:21 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 11:19 | <ladsgroup@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/FlaggedRevs/frontend/specialpages/reports/ProblemChanges_body.php: SWAT: [[gerrit:463917|Use proper index on change_tag table (T205904)]] (duration: 00m 57s) | [production] | 
            
  | 10:58 | <mobrovac@deploy1001> | Synchronized rpc/RunSingleJob.php: RunSingleJob: Delay job execution while in read-only mode - T204154 (duration: 00m 57s) | [production] | 
            
  | 10:34 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2092 (duration: 00m 56s) | [production] | 
            
  | 10:24 | <marostegui> | Deploy schema change on db2092 - T203709 | [production] | 
            
  | 10:24 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2092 (duration: 00m 56s) | [production] | 
            
  | 09:30 | <marostegui> | Deploy schema change on s2 eqiad master, lag will be generated T205913 | [production] | 
            
  | 08:43 | <banyek> | disabling puppet on es2001 and disabling backups too | [production] | 
            
  | 08:28 | <marostegui> | Deploy schema change on s6 eqiad master, lag will be generated T205913 | [production] | 
            
  | 08:16 | <jynus> | test recover some s3 wiki data onto db1110 (s5) | [production] | 
            
  | 08:04 | <jynus@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1110 (duration: 00m 56s) | [production] | 
            
  | 08:04 | <marostegui> | Deploy schema change on s5 eqiad master, lag will be generated T205913 | [production] | 
            
  | 08:01 | <banyek> | converting wikidatawiki.content to TokuDB on host dbstrore1002 (T205544) | [production] | 
            
  | 07:54 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2071 (duration: 00m 55s) | [production] | 
            
  | 07:50 | <marostegui> | Deploy schema change on db2071 T205913 | [production] | 
            
  | 07:50 | <mholloway-shell@deploy1001> | Finished deploy [tilerator/deploy@6c80537] (maps1004): Disable event logging requests and remove HTTP proxy (duration: 00m 17s) | [production] | 
            
  | 07:49 | <mholloway-shell@deploy1001> | Started deploy [tilerator/deploy@6c80537] (maps1004): Disable event logging requests and remove HTTP proxy | [production] | 
            
  | 07:49 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2071 (duration: 00m 56s) | [production] | 
            
  | 07:48 | <mholloway-shell@deploy1001> | Finished deploy [kartotherian/deploy@0bf513a] (maps1004): Remove HTTP proxy (duration: 00m 16s) | [production] | 
            
  | 07:48 | <mholloway-shell@deploy1001> | Started deploy [kartotherian/deploy@0bf513a] (maps1004): Remove HTTP proxy | [production] | 
            
  | 07:42 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2088:3311 (duration: 00m 56s) | [production] | 
            
  | 07:36 | <marostegui> | Deploy schema change on db2088:3311 T205913 | [production] |