| 
      
        2019-02-19
      
      §
     | 
  
    
  | 10:25 | 
  <marostegui> | 
  Deploy schema change on db1083 - T210713 | 
  [production] | 
            
  | 10:25 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1083 T210713 (duration: 00m 46s) | 
  [production] | 
            
  | 10:13 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Fully repool db1093 after kernel upgrade (duration: 00m 46s) | 
  [production] | 
            
  | 09:51 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: More traffic to db1093 after kernel upgrade (duration: 00m 46s) | 
  [production] | 
            
  | 09:42 | 
  <mforns@deploy1001> | 
  Finished deploy [analytics/refinery@0d7ec19]: deploying refinery to update EL sanitization whitelist (duration: 07m 49s) | 
  [production] | 
            
  | 09:34 | 
  <mforns@deploy1001> | 
  Started deploy [analytics/refinery@0d7ec19]: deploying refinery to update EL sanitization whitelist | 
  [production] | 
            
  | 09:29 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1089 T210713 (duration: 00m 45s) | 
  [production] | 
            
  | 09:23 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1093 on API after kernel upgrade (duration: 00m 46s) | 
  [production] | 
            
  | 09:08 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Slowly repool db1093 after kernel upgrade (duration: 00m 46s) | 
  [production] | 
            
  | 08:56 | 
  <_joe_> | 
  experimenting with php-fpm configuration on mwdebug1001 for T176916 | 
  [production] | 
            
  | 08:56 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1093 for kernel upgrade (duration: 00m 45s) | 
  [production] | 
            
  | 08:55 | 
  <hashar> | 
  Cleaning contint1001 / partition | 
  [production] | 
            
  | 08:50 | 
  <marostegui> | 
  Deploy schema change on db1089 - T210713 | 
  [production] | 
            
  | 08:50 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1089 T210713 (duration: 00m 46s) | 
  [production] | 
            
  | 08:05 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1106 T210713 (duration: 00m 49s) | 
  [production] | 
            
  | 07:51 | 
  <marostegui> | 
  Drop ep_* tables on s1 - T174802 | 
  [production] | 
            
  | 07:50 | 
  <moritzm> | 
  installing systemd security updates on stretch | 
  [production] | 
            
  | 07:46 | 
  <marostegui> | 
  Reboot db1106 for kernel upgrade (and remove debug from kernel) T216240 T216273 | 
  [production] | 
            
  | 07:21 | 
  <marostegui> | 
  Drop ep_* tables on s3 - T174802 | 
  [production] | 
            
  | 06:56 | 
  <marostegui> | 
  Deploy schema change on db1106 - this will generate lag on labsdb:s1 T210713 | 
  [production] | 
            
  | 06:56 | 
  <marostegui> | 
  Deploy schema change on db1106 - T210713 | 
  [production] | 
            
  | 06:56 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1106 T210713 (duration: 00m 52s) | 
  [production] | 
            
  | 05:31 | 
  <XioNoX> | 
  delete local pref for peering sessions in ulsfo - T204281 | 
  [production] | 
            
  | 05:17 | 
  <XioNoX> | 
  deleted previously deactivated BGP_community_actions terms - T204281 | 
  [production] | 
            
  | 00:01 | 
  <XioNoX> | 
  disable BGP to Zayo on cr2-codfw for intrusive testing - T215193 | 
  [production] | 
            
  
    | 
      
        2019-02-18
      
      §
     | 
  
    
  | 20:19 | 
  <gtirloni> | 
  icinga2001 ran puppet ahead of schedule (enable tools-checker-toolsdb monitor) | 
  [production] | 
            
  | 18:26 | 
  <jynus> | 
  setting clouddb1001 in read_write mode | 
  [production] | 
            
  | 18:14 | 
  <volans> | 
  upgraded to spicerack 0.0.16-1 cumin[12]001 | 
  [production] | 
            
  | 18:12 | 
  <volans> | 
  uploaded spicerack_0.0.16-1_amd64.deb to apt.wikimedia.org stretch-wikimedia | 
  [production] | 
            
  | 18:08 | 
  <jynus> | 
  killing mysql on labsdb1005 | 
  [production] | 
            
  | 18:08 | 
  <jynus> | 
  disabled puppet and edited my.cnf on labsdb1005 | 
  [production] | 
            
  | 17:56 | 
  <jynus> | 
  restarting labsdb1004 | 
  [production] | 
            
  | 17:53 | 
  <jynus> | 
  set clouddb1001 in read_only=1 | 
  [production] | 
            
  | 17:49 | 
  <jijiki> | 
  Reimaging thumbor1004 to stretch - T214597 | 
  [production] | 
            
  | 17:49 | 
  <jijiki> | 
  Reimaging thumbor1004 to stretc - T214597 | 
  [production] | 
            
  | 15:41 | 
  <jynus> | 
  performing es2 & es3 backups into es2002 | 
  [production] | 
            
  | 15:21 | 
  <jynus> | 
  move logical backups to subdirectory T210292 | 
  [production] | 
            
  | 14:29 | 
  <moritzm> | 
  rebooting mw2167 for kernel tests | 
  [production] | 
            
  | 13:59 | 
  <marostegui> | 
  Drop ep_* tables from s7 - T174802 | 
  [production] | 
            
  | 13:25 | 
  <jijiki> | 
  Depooling thumbor1004 to check if the rest of our hosts can handle the load without it - T214597 | 
  [production] | 
            
  | 12:34 | 
  <moritzm> | 
  installing brltty bugfix update from stretch point release | 
  [production] | 
            
  | 12:31 | 
  <moritzm> | 
  installing upgrading stat1005 to buster | 
  [production] | 
            
  | 12:28 | 
  <XioNoX> | 
  update clouddb_return term from cloud-in4 on cr1/2-eqiad - T216353 | 
  [production] | 
            
  | 11:53 | 
  <moritzm> | 
  installing hdparm bugfix update from stretch point release | 
  [production] | 
            
  | 11:36 | 
  <moritzm> | 
  installing uriparser security updates | 
  [production] | 
            
  | 11:11 | 
  <moritzm> | 
  installing c3p0 security updates | 
  [production] | 
            
  | 10:54 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1105:3311 T210713 (duration: 00m 46s) | 
  [production] | 
            
  | 10:54 | 
  <jijiki> | 
  Reimaging thumbor2002 to stretch - T214597 | 
  [production] | 
            
  | 10:40 | 
  <marostegui> | 
  Drop tables ep_* from s2 (cswiki nlwiki ptwiki svwiki) T174802 | 
  [production] | 
            
  | 09:50 | 
  <marostegui> | 
  Deploy schema change on db1105:3311 T210713 | 
  [production] |