| 
      
        2018-09-20
      
      ยง
     | 
  
    
  | 15:05 | 
  <hashar> | 
  1.32.0-wmf.22 on group2 seems fine so far \o/ | 
  [production] | 
            
  | 15:05 | 
  <moritzm> | 
  installing bind9 security updates (client-side tools and libraries) | 
  [production] | 
            
  | 14:48 | 
  <godog> | 
  repair sdn on ms-be2040 - T199198 | 
  [production] | 
            
  | 14:44 | 
  <gehel> | 
  reduce replication factor to 2 on cassandra maps eqiad - T194966 | 
  [production] | 
            
  | 14:38 | 
  <hashar@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.32.0-wmf.22 | 
  [production] | 
            
  | 14:28 | 
  <hashar@deploy1001> | 
  Synchronized typos: Dummy sync to verify list of canaries for T204907 (duration: 00m 59s) | 
  [production] | 
            
  | 14:27 | 
  <herron> | 
  maps1001:~# tune2fs -m0 /dev/mapper/maps1001--vg-data | 
  [production] | 
            
  | 14:22 | 
  <marostegui> | 
  Deploy schema change on s5 eqiad master with replication  T204006 | 
  [production] | 
            
  | 14:15 | 
  <moritzm> | 
  rebooting rdb1001 for kernel security update | 
  [production] | 
            
  | 14:15 | 
  <bblack> | 
  upgrade authdns1001 gdnsd 2.99.9 -> 2.99.42 | 
  [production] | 
            
  | 14:12 | 
  <bblack> | 
  upload gdnsd-2.99.42-beta to stretch-wikimedia | 
  [production] | 
            
  | 14:11 | 
  <herron> | 
  removing unused wiki-mail.wikimedia.org cname (gerrit 143762) | 
  [production] | 
            
  | 14:05 | 
  <moritzm> | 
  rebooting rdb1002 for kernel security update | 
  [production] | 
            
  | 13:27 | 
  <jynus@deploy1001> | 
  Synchronized wmf-config/db-codfw.php: Tune codfw database weights (duration: 00m 58s) | 
  [production] | 
            
  | 13:20 | 
  <gtirloni> | 
  T204667 reimage labtestnet2003 | 
  [production] | 
            
  | 12:14 | 
  <gehel> | 
  starting elasticsearch codfw cluster restart for new systemd unit | 
  [production] | 
            
  | 12:07 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 12:01 | 
  <moritzm> | 
  rebooting mc1036 for kernel security update | 
  [production] | 
            
  | 11:48 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:455395|Create new namespaces in zhwikiversity (T201675)]] (duration: 00m 57s) | 
  [production] | 
            
  | 11:46 | 
  <moritzm> | 
  rebooting mc1035 for kernel security update | 
  [production] | 
            
  | 11:38 | 
  <moritzm> | 
  rebooting mc1034 for kernel security update | 
  [production] | 
            
  | 11:35 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:455853|Introduce engineer user group on Czech Wikipedia (T203000)]] (duration: 00m 58s) | 
  [production] | 
            
  | 11:28 | 
  <moritzm> | 
  rebooting mc1033 for kernel security update | 
  [production] | 
            
  | 11:19 | 
  <dcausse@deploy1001> | 
  Synchronized ./docroot/search.wikimedia.org/index.php: search.wikimedia.org should properly handle multivalue separation char (0x1F) (duration: 00m 58s) | 
  [production] | 
            
  | 11:15 | 
  <jynus> | 
  rebooting es200X hosts for upgrade | 
  [production] | 
            
  | 11:07 | 
  <moritzm> | 
  rebooting mc1032 for kernel security update | 
  [production] | 
            
  | 11:03 | 
  <gehel> | 
  elasticsearch eqiad cluster restart for new systemd unit completed | 
  [production] | 
            
  | 10:58 | 
  <moritzm> | 
  rebooting mc1031 for kernel security update | 
  [production] | 
            
  | 10:43 | 
  <moritzm> | 
  rebooting mc1030 for kernel security update | 
  [production] | 
            
  | 10:30 | 
  <moritzm> | 
  rebooting mc1029 for kernel security update | 
  [production] | 
            
  | 10:16 | 
  <moritzm> | 
  rebooting mc1028 for kernel security update | 
  [production] | 
            
  | 10:05 | 
  <moritzm> | 
  rebooting mc1027 for kernel security update | 
  [production] | 
            
  | 09:50 | 
  <moritzm> | 
  rebooting mc1026 for kernel security update | 
  [production] | 
            
  | 09:37 | 
  <moritzm> | 
  rebooting mc1025 for kernel security update | 
  [production] | 
            
  | 09:26 | 
  <marostegui> | 
  Change passwords for wikiuser on dbstore1002 - T200801 | 
  [production] | 
            
  | 09:24 | 
  <moritzm> | 
  rebooting mc1024 for kernel security update | 
  [production] | 
            
  | 08:57 | 
  <moritzm> | 
  rebooting mc1023 for kernel security update | 
  [production] | 
            
  | 08:44 | 
  <moritzm> | 
  rebooting mc1022 for kernel security update | 
  [production] | 
            
  | 08:06 | 
  <banyek> | 
  replication stopped and tables being compressed for s2 on dbstore2002 | 
  [production] | 
            
  | 08:01 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-codfw.php: Repool db2066 (duration: 00m 57s) | 
  [production] | 
            
  | 07:53 | 
  <marostegui> | 
  Deploy schema change on db2066 | 
  [production] | 
            
  | 07:53 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-codfw.php: Depool db2066 (duration: 00m 57s) | 
  [production] | 
            
  | 07:46 | 
  <moritzm> | 
  upgrading intel-microcode on trusty systems to 3.20180807a | 
  [production] | 
            
  | 07:31 | 
  <godog> | 
  repair sdl on ms-be2042 - T199198 | 
  [production] | 
            
  | 07:25 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool pc1004 (duration: 00m 57s) | 
  [production] | 
            
  | 07:15 | 
  <marostegui> | 
  Stop MySQL on pc1004 for kernel upgrade | 
  [production] | 
            
  | 07:14 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool pc1004 (duration: 00m 57s) | 
  [production] | 
            
  | 06:50 | 
  <eileen> | 
  process-control config revision is 3069f85578 | 
  [production] | 
            
  | 06:42 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 00m 58s) | 
  [production] | 
            
  | 06:25 | 
  <mobrovac@deploy1001> | 
  Started restart [cpjobqueue/deploy@58f9ed3]: Reset the Kafka connections | 
  [production] |