| 2017-07-19
      
      § | 
    
  | 16:27 | <robh@puppetmaster1001> | conftool action : set/pooled=inactive; selector: name=wdqs1003.eqiad.wmnet | [production] | 
            
  | 16:23 | <robh@puppetmaster1001> | conftool action : set/pooled=active; selector: name=wdqs1002.eqiad.wmnet | [production] | 
            
  | 16:16 | <marostegui> | Compressing innodb on dbstore1002 for the following wikis: viwiki ukwiki kowiki huwiki hewiki frwiktionary fawiki eswiki cawiki arwiki - T168303 | [production] | 
            
  | 16:10 | <robh@puppetmaster1001> | conftool action : set/pooled=inactive; selector: name=mw1196.eqiad.wmnet | [production] | 
            
  | 16:04 | <robh> | mw1196 has hardware failure and is being decommissioned | [production] | 
            
  | 16:04 | <robh@puppetmaster1001> | conftool action : set/pooled=no; selector: name=mw1196.eqiad.wmnet | [production] | 
            
  | 14:44 | <hashar> | Restarting Jenkins | [production] | 
            
  | 14:31 | <moritzm> | installing imagemagick security updates | [production] | 
            
  | 13:55 | <_joe_> | running clear-host-cache.js for ocg1001 decommission T170886 | [production] | 
            
  | 13:53 | <oblivian@puppetmaster1001> | conftool action : set/pooled=inactive; selector: cluster=pdf,name=ocg1001.eqiad.wmnet | [production] | 
            
  | 13:46 | <marostegui> | Compress database rowiki on dbstore1002 - T168303 | [production] | 
            
  | 13:45 | <elukey> | restart hive-server on analytics1003 - Java OOM issue due to a huge query | [production] | 
            
  | 13:34 | <hashar> | European SWAT completed | [production] | 
            
  | 13:32 | <elukey> | Limit the access to the conf* zookeeper ports via ferm rules - https://gerrit.wikimedia.org/r/366228 | [production] | 
            
  | 13:29 | <hashar> | Purged all 1685 project-logos  ( find static/images/project-logos -maxdepth 1 -type f| sed -e 's%^%https://en.wikipedia.org/%'|mwscript purgeList.php --wiki=enwiki ) | [production] | 
            
  | 13:26 | <hashar@tin> | Synchronized wmf-config/InitialiseSettings.php: Change timezone on nl.wikinews to Europe/Berlin - T170985 (duration: 00m 44s) | [production] | 
            
  | 13:24 | <marostegui> | Optimize EditConflict_8860941_15423246 and Echo_7731316 on dbstore1002 - T168303 | [production] | 
            
  | 13:17 | <hashar@tin> | Synchronized static/images/project-logos/nlwikinews.png: Change logo on nl.wikinews - T170984 (duration: 00m 47s) | [production] | 
            
  | 13:11 | <jmm@puppetmaster1001> | conftool action : set/pooled=yes; selector: nescio.wikimedia.org | [production] | 
            
  | 13:10 | <hashar@tin> | Synchronized wmf-config/InitialiseSettings.php: Update wikiversity logos to 2017 - T160491 (duration: 00m 46s) | [production] | 
            
  | 13:09 | <hashar@tin> | Synchronized static/images/project-logos: Update wikiversity logos to 2017 - T160491 (duration: 00m 48s) | [production] | 
            
  | 13:05 | <hashar@tin> | Synchronized wmf-config/throttle.php: Extend throttle rule - T170844 (duration: 00m 48s) | [production] | 
            
  | 12:56 | <moritzm> | rebooting nescio (DNS recursor) for kernel update | [production] | 
            
  | 12:55 | <jmm@puppetmaster1001> | conftool action : set/pooled=no; selector: nescio.wikimedia.org | [production] | 
            
  | 12:49 | <jmm@puppetmaster1001> | conftool action : set/pooled=yes; selector: maerlant.wikimedia.org | [production] | 
            
  | 12:40 | <Reedy> | running foreachwiki updateRestrictions.php T166184 | [production] | 
            
  | 12:34 | <moritzm> | rebooting maerlant (DNS recursor) for kernel update | [production] | 
            
  | 12:29 | <jmm@puppetmaster1001> | conftool action : set/pooled=no; selector: maerlant.wikimedia.org | [production] | 
            
  | 12:10 | <ema@neodymium> | conftool action : set/pooled=yes; selector: name=acamar.wikimedia.org | [production] | 
            
  | 12:09 | <ema@neodymium> | conftool action : set/pooled=no; selector: name=acamar.wikimedia.org | [production] | 
            
  | 12:07 | <ariel@tin> | Finished deploy [dumps/dumps@f95292e]: fix api call bug, page range query min pages (duration: 00m 03s) | [production] | 
            
  | 12:07 | <ariel@tin> | Started deploy [dumps/dumps@f95292e]: fix api call bug, page range query min pages | [production] | 
            
  | 11:40 | <kartik@tin> | Finished deploy [cxserver/deploy@1029833]: Update cxserver to d28ad0c (duration: 03m 01s) | [production] | 
            
  | 11:37 | <moritzm> | rebooting acamar (DNS recursor) for kernel update | [production] | 
            
  | 11:37 | <kartik@tin> | Started deploy [cxserver/deploy@1029833]: Update cxserver to d28ad0c | [production] | 
            
  | 10:58 | <marostegui> | Global rename of user Moros - T170941 | [production] | 
            
  | 10:09 | <ema> | ulsfo cache_text/upload: upgrade to varnish 4.1.7-1wm1 and reboot for kernel updates | [production] | 
            
  | 10:06 | <marostegui> | Deploy alter table on s1 - db1051 - T166204 | [production] | 
            
  | 10:04 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1051 - T166204 (duration: 00m 47s) | [production] | 
            
  | 10:01 | <filippo@tin> | Finished deploy [statsv/statsv@0a86be8]: (no justification provided) (duration: 00m 03s) | [production] | 
            
  | 10:01 | <filippo@tin> | Started deploy [statsv/statsv@0a86be8]: (no justification provided) | [production] | 
            
  | 09:58 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1065 - T166204 (duration: 00m 47s) | [production] | 
            
  | 09:16 | <ema> | finish up codfw cache_text/upload varnish/kernel upgrades | [production] | 
            
  | 09:05 | <oblivian@neodymium> | conftool action : set/pooled=false; selector: name=eqiad,dnsdisc=(restbase-async|citoid) | [production] | 
            
  | 09:03 | <XioNoX> | codfw repooled in dns - T170380 | [production] | 
            
  | 09:01 | <hashar> | restarting nodepool for upgrade 0.1.1-wmf7 -> 0.1.1-wmf8 | [production] | 
            
  | 08:48 | <moritzm> | uploaded nodepool 0.1.1+wmf8 to apt.wikimedia.org | [production] | 
            
  | 08:29 | <XioNoX> | asw-c-codfw back online - T170380 | [production] | 
            
  | 08:28 | <XioNoX> | asw-c-codfw restarted 8min ago for switch upgrade - T170380 | [production] | 
            
  | 08:09 | <akosiaris> | disable librenms crons on netmon1002 for a while | [production] |