| 2019-02-19
      
      § | 
    
  | 10:13 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Fully repool db1093 after kernel upgrade (duration: 00m 46s) | [production] | 
            
  | 09:51 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: More traffic to db1093 after kernel upgrade (duration: 00m 46s) | [production] | 
            
  | 09:42 | <mforns@deploy1001> | Finished deploy [analytics/refinery@0d7ec19]: deploying refinery to update EL sanitization whitelist (duration: 07m 49s) | [production] | 
            
  | 09:34 | <mforns@deploy1001> | Started deploy [analytics/refinery@0d7ec19]: deploying refinery to update EL sanitization whitelist | [production] | 
            
  | 09:29 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1089 T210713 (duration: 00m 45s) | [production] | 
            
  | 09:23 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1093 on API after kernel upgrade (duration: 00m 46s) | [production] | 
            
  | 09:08 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Slowly repool db1093 after kernel upgrade (duration: 00m 46s) | [production] | 
            
  | 08:56 | <_joe_> | experimenting with php-fpm configuration on mwdebug1001 for T176916 | [production] | 
            
  | 08:56 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1093 for kernel upgrade (duration: 00m 45s) | [production] | 
            
  | 08:55 | <hashar> | Cleaning contint1001 / partition | [production] | 
            
  | 08:50 | <marostegui> | Deploy schema change on db1089 - T210713 | [production] | 
            
  | 08:50 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1089 T210713 (duration: 00m 46s) | [production] | 
            
  | 08:05 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1106 T210713 (duration: 00m 49s) | [production] | 
            
  | 07:51 | <marostegui> | Drop ep_* tables on s1 - T174802 | [production] | 
            
  | 07:50 | <moritzm> | installing systemd security updates on stretch | [production] | 
            
  | 07:46 | <marostegui> | Reboot db1106 for kernel upgrade (and remove debug from kernel) T216240 T216273 | [production] | 
            
  | 07:21 | <marostegui> | Drop ep_* tables on s3 - T174802 | [production] | 
            
  | 06:56 | <marostegui> | Deploy schema change on db1106 - this will generate lag on labsdb:s1 T210713 | [production] | 
            
  | 06:56 | <marostegui> | Deploy schema change on db1106 - T210713 | [production] | 
            
  | 06:56 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1106 T210713 (duration: 00m 52s) | [production] | 
            
  | 05:31 | <XioNoX> | delete local pref for peering sessions in ulsfo - T204281 | [production] | 
            
  | 05:17 | <XioNoX> | deleted previously deactivated BGP_community_actions terms - T204281 | [production] | 
            
  | 00:01 | <XioNoX> | disable BGP to Zayo on cr2-codfw for intrusive testing - T215193 | [production] | 
            
  
    | 2019-02-18
      
      § | 
    
  | 20:19 | <gtirloni> | icinga2001 ran puppet ahead of schedule (enable tools-checker-toolsdb monitor) | [production] | 
            
  | 18:26 | <jynus> | setting clouddb1001 in read_write mode | [production] | 
            
  | 18:14 | <volans> | upgraded to spicerack 0.0.16-1 cumin[12]001 | [production] | 
            
  | 18:12 | <volans> | uploaded spicerack_0.0.16-1_amd64.deb to apt.wikimedia.org stretch-wikimedia | [production] | 
            
  | 18:08 | <jynus> | killing mysql on labsdb1005 | [production] | 
            
  | 18:08 | <jynus> | disabled puppet and edited my.cnf on labsdb1005 | [production] | 
            
  | 17:56 | <jynus> | restarting labsdb1004 | [production] | 
            
  | 17:53 | <jynus> | set clouddb1001 in read_only=1 | [production] | 
            
  | 17:49 | <jijiki> | Reimaging thumbor1004 to stretch - T214597 | [production] | 
            
  | 17:49 | <jijiki> | Reimaging thumbor1004 to stretc - T214597 | [production] | 
            
  | 15:41 | <jynus> | performing es2 & es3 backups into es2002 | [production] | 
            
  | 15:21 | <jynus> | move logical backups to subdirectory T210292 | [production] | 
            
  | 14:29 | <moritzm> | rebooting mw2167 for kernel tests | [production] | 
            
  | 13:59 | <marostegui> | Drop ep_* tables from s7 - T174802 | [production] | 
            
  | 13:25 | <jijiki> | Depooling thumbor1004 to check if the rest of our hosts can handle the load without it - T214597 | [production] | 
            
  | 12:34 | <moritzm> | installing brltty bugfix update from stretch point release | [production] | 
            
  | 12:31 | <moritzm> | installing upgrading stat1005 to buster | [production] | 
            
  | 12:28 | <XioNoX> | update clouddb_return term from cloud-in4 on cr1/2-eqiad - T216353 | [production] | 
            
  | 11:53 | <moritzm> | installing hdparm bugfix update from stretch point release | [production] | 
            
  | 11:36 | <moritzm> | installing uriparser security updates | [production] | 
            
  | 11:11 | <moritzm> | installing c3p0 security updates | [production] | 
            
  | 10:54 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1105:3311 T210713 (duration: 00m 46s) | [production] | 
            
  | 10:54 | <jijiki> | Reimaging thumbor2002 to stretch - T214597 | [production] | 
            
  | 10:40 | <marostegui> | Drop tables ep_* from s2 (cswiki nlwiki ptwiki svwiki) T174802 | [production] | 
            
  | 09:50 | <marostegui> | Deploy schema change on db1105:3311 T210713 | [production] | 
            
  | 09:50 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool db1105:3311 T210713 (duration: 00m 46s) | [production] | 
            
  | 09:46 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1099:3311 T210713 (duration: 00m 46s) | [production] |