| 
      
        2018-04-19
      
      §
     | 
  
    
  | 13:56 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2075, depool db2074 (duration: 01m 16s) | 
  [production] | 
            
  | 13:39 | 
  <marostegui> | 
  Stop atop on db1114 - T191996 | 
  [production] | 
            
  | 13:33 | 
  <marostegui> | 
  Start atop on db1114 - T191996 | 
  [production] | 
            
  | 13:30 | 
  <Trey314159> | 
  reindexing serbian wikis on elastic@eqiad (T189265) | 
  [production] | 
            
  | 13:30 | 
  <moritzm> | 
  upgrading mw1334-mw1337 (job runners) to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 13:14 | 
  <reedy@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: T192427 T189277 (duration: 01m 17s) | 
  [production] | 
            
  | 12:58 | 
  <jynus> | 
  starting reimage of db2075 | 
  [production] | 
            
  | 12:48 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Depool db2075 (duration: 01m 16s) | 
  [production] | 
            
  | 11:55 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2076 (duration: 01m 16s) | 
  [production] | 
            
  | 11:39 | 
  <moritzm> | 
  upgrading eqiad video scalers to  MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 11:24 | 
  <marostegui> | 
  Run check_private_data on labsdb - T183566 | 
  [production] | 
            
  | 11:21 | 
  <marostegui> | 
  Sanitize lfnwiki - T183566 | 
  [production] | 
            
  | 11:20 | 
  <moritzm> | 
  upgrading app servers mw1238-mw1258 to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 11:14 | 
  <akosiaris@tin> | 
  Synchronized wmf-config/ProductionServices.php: T181121 (duration: 01m 16s) | 
  [production] | 
            
  | 11:09 | 
  <akosiaris@tin> | 
  Synchronized wmf-config/ProductionServices.php: (no justification provided) (duration: 01m 17s) | 
  [production] | 
            
  | 11:05 | 
  <marostegui> | 
  Deploy schema change on db1113:3315 - T191519 T188299 T190148 | 
  [production] | 
            
  | 11:03 | 
  <jynus> | 
  starting reimage of db2076 | 
  [production] | 
            
  | 11:02 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1113:3315 for alter table (duration: 01m 16s) | 
  [production] | 
            
  | 11:01 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Depool db2076 (duration: 01m 18s) | 
  [production] | 
            
  | 10:34 | 
  <moritzm> | 
  upgrading API servers mw1221-mw1235 to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 10:27 | 
  <vgutierrez> | 
  Repool (Re-enable BGP) lvs4005 - T191897 | 
  [production] | 
            
  | 09:59 | 
  <elukey> | 
  complete migration of zookeeper on conf100[123] | 
  [production] | 
            
  | 09:55 | 
  <akosiaris> | 
  reboot ganeti VMs on row_B in codfw for cache=none setting. T181121 | 
  [production] | 
            
  | 09:54 | 
  <vgutierrez> | 
  Updating puppet compiler facts | 
  [production] | 
            
  | 09:51 | 
  <moritzm> | 
  rolling restart of Cassandra on maps completed | 
  [production] | 
            
  | 09:33 | 
  <elukey> | 
  upgrade zookeper on conf100[123] from 3.4.5 to 3.4.9 - T182924 | 
  [production] | 
            
  | 09:31 | 
  <akosiaris> | 
  start a force puppet run in all of eqiad with a batch size of 30 | 
  [production] | 
            
  | 09:29 | 
  <akosiaris> | 
  stop ircecho for a while, puppetdb1001 reboot was eventful | 
  [production] | 
            
  | 09:17 | 
  <akosiaris> | 
  reboot puppetdb1001 for cache=none setting apply. T181121 | 
  [production] | 
            
  | 09:14 | 
  <moritzm> | 
  installing Java security updates on maps* plus rolling restart of Cassandra to pick up new JRE | 
  [production] | 
            
  | 09:06 | 
  <vgutierrez> | 
  Depool and reimage lvs4005 as stretch - T191897 | 
  [production] | 
            
  | 09:03 | 
  <moritzm> | 
  upgrading API server canaries to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build (T184854) | 
  [production] | 
            
  | 08:40 | 
  <vgutierrez> | 
  Repool (Re-enable BGP) lvs4006 - T191897 | 
  [production] | 
            
  | 08:14 | 
  <ema> | 
  reboot deploy1001 and arm keyholder T175288 | 
  [production] | 
            
  | 08:14 | 
  <moritzm> | 
  upgrading app server canaries to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build (T184854) | 
  [production] | 
            
  | 07:47 | 
  <akosiaris> | 
  set cache=none for ganeti VMs in codfw cluster configuration. VM reboots to follow T181121 | 
  [production] | 
            
  | 07:32 | 
  <vgutierrez> | 
  Depool and reimage lvs4006 - T191897 | 
  [production] | 
            
  | 07:24 | 
  <akosiaris> | 
  reboot ganeti VMs on row_A in eqiad for cache=none setting. T181121 | 
  [production] | 
            
  | 07:06 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1097:3315 after alter table (duration: 01m 17s) | 
  [production] | 
            
  | 05:36 | 
  <marostegui> | 
  Kill atop on db1114 - T191996 | 
  [production] | 
            
  | 05:33 | 
  <marostegui> | 
  Revert RX buffer changes on db1114 - T191996 | 
  [production] | 
            
  | 05:27 | 
  <marostegui> | 
  Deploy schema change on db1097:3315 - T191519 T188299 T190148 | 
  [production] | 
            
  | 05:26 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1097:3315 for alter table (duration: 01m 33s) | 
  [production] | 
            
  | 03:18 | 
  <urandom> | 
  decommissioning Cassandra, restbase1010-c -- T189822 | 
  [production] | 
            
  | 02:36 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.31.0-wmf.29) (duration: 05m 52s) | 
  [production] | 
            
  | 01:15 | 
  <eileen> | 
  civicrm revision changed from 0ac27e7c0d to b1e7ccfc4d, config revision is 49f5ba45e8 | 
  [production] | 
            
  | 00:12 | 
  <Dereckson> | 
  Wikis creation done | 
  [production] | 
            
  | 00:12 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Set project namespace for hi.wikimedia (T188366) (duration: 01m 16s) | 
  [production] | 
            
  | 00:04 | 
  <Dereckson> | 
  dereckson@tin Synchronized wmf-config/InitialiseSettings.php: Fix path to hi.wikimedia.org 1x logo ([[Gerrit:427567]]) | 
  [production] |