| 
      
        2016-07-07
      
      ยง
     | 
  
    
  | 19:02 | 
  <urandom> | 
  Restarting Cassandra instance restbase2001-b.codfw.wmnet : T126629 | 
  [production] | 
            
  | 18:55 | 
  <urandom> | 
  Restarting Cassandra instance restbase2001-a.codfw.wmnet : T126629 | 
  [production] | 
            
  | 18:53 | 
  <urandom> | 
  Upgrading Cassandra to 2.2.6-wmf1 on restbase2001.codfw.wmnet : T126629 | 
  [production] | 
            
  | 18:27 | 
  <urandom> | 
  Disabling Puppet on RESTBase codfw nodes : T126629 | 
  [production] | 
            
  | 18:23 | 
  <awight> | 
  Update SmashPig from 917138e159f0341e3dfbb35818c3ce479927875b to e6aa6fe6fdcaab8e961a8b0668cc742d4c443c46 | 
  [production] | 
            
  | 18:15 | 
  <urandom> | 
  Cassandra 2.2.6 upgrade of restbase1015.eqiad.wmnet instances complete : T126629 | 
  [production] | 
            
  | 18:11 | 
  <urandom> | 
  Restarting Cassandra instance restbase1015-b.eqiad.wmnet : T126629 | 
  [production] | 
            
  | 18:08 | 
  <urandom> | 
  Restarting Cassandra instance restbase1015-a.eqiad.wmnet : T126629 | 
  [production] | 
            
  | 18:07 | 
  <urandom> | 
  Upgrading Cassandra to 2.2.6-wmf1 on restbase1015.eqiad.wmnet : T126629 | 
  [production] | 
            
  | 17:12 | 
  <ostriches> | 
  gerrit: flush all caches to pick up account disable & rename | 
  [production] | 
            
  | 16:55 | 
  <urandom> | 
  Restarting Cassandra instance restbase1014-c.eqiad.wmnet : T126629 | 
  [production] | 
            
  | 16:54 | 
  <awight> | 
  update paymentwiki from 2fc573cbb94e833c4144aa9dad79de8ec374bb09 to c33ddfccf945bd075f0abff9e9de8c09f0174f89 | 
  [production] | 
            
  | 16:48 | 
  <urandom> | 
  Restarting Cassandra instance restbase1014-b.eqiad.wmnet : T126629 | 
  [production] | 
            
  | 16:47 | 
  <jynus> | 
  stopping pc2006 for hardware maintenance T139283 | 
  [production] | 
            
  | 16:39 | 
  <urandom> | 
  Restarting Cassandra instance restbase1014-a.eqiad.wmnet : T126629 | 
  [production] | 
            
  | 16:38 | 
  <urandom> | 
  Upgrading Cassandra to 2.2.6-wmf1 on restbase1014.eqiad.wmnet : T126629 | 
  [production] | 
            
  | 16:29 | 
  <urandom> | 
  Disabling Puppet on restbase101[4-5].eqiad.wmnet : T126629 | 
  [production] | 
            
  | 16:24 | 
  <gehel> | 
  starting elasticsearch and kibana upgrade on logstash cluster (T136001) | 
  [production] | 
            
  | 16:21 | 
  <awight> | 
  Taking Fundraising campaigns down for maintenance | 
  [production] | 
            
  | 15:51 | 
  <elukey> | 
  add mw1261 back into service | 
  [production] | 
            
  | 15:44 | 
  <bd808> | 
  Dropped logstash indices older than logstash-2016.07.01 in preparation for elasticsearch upgrade | 
  [production] | 
            
  | 15:25 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.9/extensions/Echo/modules/controller/mw.echo.Controller.js: SWAT: [[gerrit:297792|Correct section (alert/message/all)]] (duration: 00m 25s) | 
  [production] | 
            
  | 15:13 | 
  <_joe_> | 
  uploaded new HHVM package for jessie | 
  [production] | 
            
  | 15:11 | 
  <thcipriani@tin> | 
  Synchronized dblists/clldefault.dblist: SWAT: [[gerrit:297349|Deploy Compact Language Links as default (Stage 4) (T136677)]] PART II (duration: 00m 34s) | 
  [production] | 
            
  | 15:11 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:297349|Deploy Compact Language Links as default (Stage 4) (T136677)]] PART I (duration: 00m 55s) | 
  [production] | 
            
  | 14:43 | 
  <elukey@palladium> | 
  conftool action : set/pooled=yes; selector: aqs1001.eqiad.wmnet | 
  [production] | 
            
  | 14:38 | 
  <elukey@palladium> | 
  conftool action : set/pooled=no; selector: aqs1001.eqiad.wmnet | 
  [production] | 
            
  | 14:37 | 
  <elukey> | 
  depool aqs1001 for nodejs upgrade | 
  [production] | 
            
  | 12:05 | 
  <elukey> | 
  depooling mw1261 from service (T73487) | 
  [production] | 
            
  | 11:34 | 
  <jynus> | 
  breaking m3 replication on db1048 (depooled) to check icinga alert changes | 
  [production] | 
            
  | 10:44 | 
  <jynus> | 
  disabling all mysql lag alerts cross-fleet T122457 | 
  [production] | 
            
  | 10:29 | 
  <akosiaris> | 
  reboot fermium.wikimedia.org hassium.eqiad.wmnet install1001.wikimedia.org krypton.eqiad.wmnet meitnerium.wikimedia.org mendelevium.eqiad.wmnet T134242 | 
  [production] | 
            
  | 10:25 | 
  <akosiaris> | 
  reboot mx1001, planet1001, rutherfordium, seaborgium, ununpentium T134242 | 
  [production] | 
            
  | 10:22 | 
  <akosiaris> | 
  reboot dubnium T134242 | 
  [production] | 
            
  | 10:22 | 
  <akosiaris> | 
  reboot bromine T134242 | 
  [production] | 
            
  | 10:13 | 
  <akosiaris> | 
  reboot bohrium T134242 | 
  [production] | 
            
  | 10:11 | 
  <elukey> | 
  pooling mw1261 back to service with Apache mod-proxy-fcgi set to trace8 (T73487) | 
  [production] | 
            
  | 10:07 | 
  <akosiaris> | 
  reboot etherpad1001.eqiad.wmnet, kernel upgrade and qemu upgrade, T134242 | 
  [production] | 
            
  | 10:04 | 
  <gehel> | 
  rolling restart of elasticsearch cluster eqiad completed (T138811) | 
  [production] | 
            
  | 08:51 | 
  <_joe_> | 
  removing all old servers from the appservers pool but the canaries (T139353) | 
  [production] | 
            
  | 08:47 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Failover s4 recentchanges to db1056 (duration: 00m 38s) | 
  [production] | 
            
  | 08:33 | 
  <hoo> | 
  Updated Wikidata's property suggester with data from Monday's json dump and removed the external identifiers as a workaround for T132839 | 
  [production] | 
            
  | 07:56 | 
  <gehel> | 
  rolling restart of elasticsearch cluster codfw completed (T138811) | 
  [production] | 
            
  | 07:16 | 
  <legoktm> | 
  mysql:wikiadmin@db1041 [centralauth]> delete from localuser where lu_name ="Philippe" and lu_wiki ="scnwiki"; | 
  [production] | 
            
  | 04:37 | 
  <eileen> | 
  Update CiviCRM  from dd24368a897fd78752178ee253e7a890dd57db41 to bdf2afd417b70332c9542fd3ee4f14cb4e6f93cc | 
  [production] | 
            
  | 03:12 | 
  <eileen> | 
  CiviCRM upgrade from 5f8f7c3236e6bf12c52deea07093fbca165ef4a7 to dd24368a897fd78752178ee253e7a890dd57db41 | 
  [production] | 
            
  | 03:07 | 
  <chasemp> | 
  silence labvirt1011 flapping for 24h, we have a task, and we are attempting to move vms we can | 
  [production] | 
            
  | 02:48 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Thu Jul  7 02:48:55 UTC 2016 (duration 6m 44s) | 
  [production] | 
            
  | 02:42 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.9) (duration: 07m 07s) | 
  [production] | 
            
  | 02:26 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.8) (duration: 09m 58s) | 
  [production] |