| 2017-07-26
      
      § | 
    
  | 09:58 | <jmm@puppetmaster1001> | conftool action : set/pooled=no; selector: mw2119.codfw.wmnet | [production] | 
            
  | 09:58 | <jmm@puppetmaster1001> | conftool action : set/pooled=no; selector: mw2119.codfw.wmnet | [production] | 
            
  | 09:58 | <jmm@puppetmaster1001> | conftool action : set/pooled=yes; selector: mw2119.codfw.wmnet | [production] | 
            
  | 09:57 | <moritzm> | reimaging mw2152 to jessie (T145742) | [production] | 
            
  | 09:52 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1066 after maintenance (duration: 00m 46s) | [production] | 
            
  | 08:47 | <elukey> | rollout logster 0.0.10-2~jessie1 to the cache hosts | [production] | 
            
  | 08:46 | <elukey> | upload logster 0.0.10-2~jessie1 to jessie-wikimedia | [production] | 
            
  | 08:42 | <jynus> | upgrading and restarting db1066 | [production] | 
            
  | 08:26 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1066 for maintenance (duration: 00m 46s) | [production] | 
            
  | 08:26 | <moritzm> | reimaging mw2119 to jessie (T145742) | [production] | 
            
  | 08:02 | <moritzm> | installing Java security updates on jessie-based stat systems | [production] | 
            
  | 07:59 | <moritzm> | restarting cassandra-metrics-collector on maps* to pick up openjdk security update | [production] | 
            
  | 07:56 | <moritzm> | restarting cassandra-metrics-collector on restbase* to pick up openjdk security update | [production] | 
            
  | 07:53 | <jynus> | start defragmenging on pc1* hosts T167784 | [production] | 
            
  | 07:14 | <ema> | cp1008: use sdb only in varnish.service, waiting for Chris to replace sda T171028 | [production] | 
            
  | 05:53 | <_joe_> | moving all conf* servers to the future puppet parser | [production] | 
            
  | 03:01 | <l10nupdate@tin> | LocalisationUpdate failed: git pull of extensions failed | [production] | 
            
  | 01:15 | <reedy@tin> | Synchronized multiversion/: phpcs (duration: 01m 06s) | [production] | 
            
  | 01:13 | <reedy@tin> | Synchronized phpcs.xml: phpcs (duration: 00m 46s) | [production] | 
            
  | 01:12 | <reedy@tin> | Synchronized tests/multiversion/: phpcs (duration: 00m 46s) | [production] | 
            
  
    | 2017-07-25
      
      § | 
    
  | 23:13 | <reedy@tin> | Synchronized wmf-config/abusefilter.php: Allow contentadmin/sysop to configure blocking AbuseFilters (duration: 00m 46s) | [production] | 
            
  | 22:14 | <reedy@tin> | Synchronized php-1.30.0-wmf.11/includes/specials/SpecialUndelete.php: T171523 (duration: 00m 46s) | [production] | 
            
  | 22:12 | <reedy@tin> | Synchronized php-1.30.0-wmf.10/includes/specials/SpecialUndelete.php: T171523 (duration: 00m 47s) | [production] | 
            
  | 21:13 | <urandom> | Rolling restart of eqiad Cassandra instances (applying OpenJDK update) | [production] | 
            
  | 21:12 | <ejegg> | updated SmashPig from 523d6dddfb1d703213cdca12bc84b62299d8aebd to f4ca53ca303a7c9cbc4f0fd15d62116f0689b81d | [production] | 
            
  | 20:39 | <ejegg> | updated fundraising process-control to adb332586850d6d69fb83a823f6c5c0c53f229a6 | [production] | 
            
  | 19:31 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.11 | [production] | 
            
  | 18:52 | <mobrovac@tin> | Finished deploy [restbase/deploy@36ca85f]: (no justification provided) (duration: 03m 49s) | [production] | 
            
  | 18:48 | <mobrovac@tin> | Started deploy [restbase/deploy@36ca85f]: (no justification provided) | [production] | 
            
  | 18:48 | <mobrovac@tin> | Finished deploy [restbase/deploy@36ca85f]: (no justification provided) (duration: 00m 49s) | [production] | 
            
  | 18:47 | <mobrovac@tin> | Started deploy [restbase/deploy@36ca85f]: (no justification provided) | [production] | 
            
  | 18:46 | <mobrovac@tin> | Finished deploy [restbase/deploy@36ca85f]: Switch to Node v6.11 - T170548 (duration: 05m 17s) | [production] | 
            
  | 18:42 | <krinkle@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable jQuery 3 on testwikis - I37a68472cf (duration: 00m 50s) | [production] | 
            
  | 18:41 | <mobrovac@tin> | Started deploy [restbase/deploy@36ca85f]: Switch to Node v6.11 - T170548 | [production] | 
            
  | 18:28 | <mobrovac> | restbase upgrading node to v6.11 - T170548 | [production] | 
            
  | 18:14 | <demon@tin> | Finished scap: bootstrap wmf.11 (x2) (duration: 19m 23s) | [production] | 
            
  | 17:55 | <demon@tin> | Started scap: bootstrap wmf.11 (x2) | [production] | 
            
  | 17:54 | <demon@tin> | scap failed: RuntimeError scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) (duration: 16m 32s) | [production] | 
            
  | 17:53 | <demon@tin> | scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) | [production] | 
            
  | 17:49 | <urandom> | Rolling restart of codfw Cassandra instances (applying OpenJDK update) | [production] | 
            
  | 17:40 | <halfak@tin> | Finished deploy [ores/deploy@835d848]: T171505 (duration: 34m 56s) | [production] | 
            
  | 17:37 | <demon@tin> | Started scap: bootstrap wmf.11 | [production] | 
            
  | 17:33 | <jynus> | creating new database on m1 (rddmarc) T170158 | [production] | 
            
  | 17:10 | <demon@tin> | Pruned MediaWiki: 1.30.0-wmf.9 [keeping static files] (duration: 01m 39s) | [production] | 
            
  | 17:05 | <halfak@tin> | Started deploy [ores/deploy@835d848]: T171505 | [production] | 
            
  | 16:16 | <jynus> | about to delete orfphan files on einstenium T149557 | [production] | 
            
  | 15:49 | <moritzm> | installing imagemagick security updates on trusty hosts (jessie already fixed) | [production] | 
            
  | 15:31 | <cmjohnson1> | updating firmware lvs1007 T167299 | [production] | 
            
  | 15:03 | <oblivian@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=restbase1015.eqiad.wmnet | [production] | 
            
  | 14:58 | <_joe_> | enabled hyperthreading on restbase1015.eqiad.wmnet T162735, rebooting the server | [production] |