| 2018-09-13
      
      § | 
    
  | 17:42 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.20/resources/src/mediawiki.widgets/mw.widgets.CheckMatrixWidget.js: I1f92479bf1, T203325 (duration: 00m 50s) | [production] | 
            
  | 17:41 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.20/includes/widget/CheckMatrixWidget.php: I1f92479bf1, T203325 (duration: 00m 49s) | [production] | 
            
  | 17:40 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.20/includes/htmlform/fields/HTMLCheckMatrix.php: I1f92479bf1, T203325 (duration: 00m 51s) | [production] | 
            
  | 17:38 | <XioNoX> | enable cr2:xe-4/0/0 (to asw-a) - T203719 | [production] | 
            
  | 17:31 | <XioNoX> | disable cr2:xe-4/0/0 (to asw-a) for optics replacement (round 2, 1st one didn't clear the errors, need to do the other side) - T203719 | [production] | 
            
  | 17:29 | <mutante> | radium - removing tor package, clearing systemd failed units, to clear Icinga alerts from this host that is to be decomed | [production] | 
            
  | 17:29 | <XioNoX> | enable cr2:xe-4/0/0 (to asw-a) for optics replacement - T203719 | [production] | 
            
  | 17:25 | <XioNoX> | disable cr2:xe-4/0/0 (to asw-a) for optics replacement - T203719 | [production] | 
            
  | 16:45 | <godog> | repair sde on ms-be1041 - T199198 | [production] | 
            
  | 15:26 | <jynus> | restarting again db1075 for proper kernel upgrade | [production] | 
            
  | 15:25 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Increase weight for db2054 - T204127 (duration: 00m 49s) | [production] | 
            
  | 15:15 | <herron> | repool mx1001 — upgrade to stretch complete T175361 | [production] | 
            
  | 15:02 | <akosiaris> | upload blubber_0.5.0-1 to apt.wikimedia.org/{stretch,jessie}-wikimedia/main T203121 | [production] | 
            
  | 14:47 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Increase weight for db2054 - T204127 (duration: 00m 50s) | [production] | 
            
  | 14:36 | <marostegui> | Reload haproxy on dbproxy1010 to depool labsdb1011 - https://phabricator.wikimedia.org/T174047 | [production] | 
            
  | 14:35 | <bstorm_> | switch labstore1004 and labstore1005 to using the cfq scheduler on the DRBD volumes | [production] | 
            
  | 14:32 | <moritzm> | rebooting dns1002 for kernel security update | [production] | 
            
  | 13:59 | <herron> | depool mx1001 and relay queued messages to mx2001 for upgrade to stretch T175361 | [production] | 
            
  | 13:51 | <moritzm> | rebooting dns1001 for kernel security update | [production] | 
            
  | 13:28 | <andrew@deploy1001> | Finished deploy [horizon/deploy@c9c7a56]: Improvements for VM creation in eqiad1, T167293 (take two) (duration: 04m 10s) | [production] | 
            
  | 13:24 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Increase weight for db2054 - T204127 (duration: 00m 49s) | [production] | 
            
  | 13:23 | <andrew@deploy1001> | Started deploy [horizon/deploy@c9c7a56]: Improvements for VM creation in eqiad1, T167293 (take two) | [production] | 
            
  | 13:20 | <andrew@deploy1001> | Finished deploy [horizon/deploy@12aa2d3]: Improvements for VM creation in eqiad1, T167293 (duration: 00m 13s) | [production] | 
            
  | 13:20 | <andrew@deploy1001> | Started deploy [horizon/deploy@12aa2d3]: Improvements for VM creation in eqiad1, T167293 | [production] | 
            
  | 13:18 | <marostegui> | Deploy schema change on s2 eqiad master (db1066) - T89737 | [production] | 
            
  | 13:07 | <marostegui> | Deploy schema change on s8 eqiad master (db1071) - T187089 | [production] | 
            
  | 13:03 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2085:3318 - T189101 (duration: 00m 49s) | [production] | 
            
  | 12:55 | <marostegui> | Stop db1071 (s8 eqiad master) and db2085:3318 in sync - T189101 | [production] | 
            
  | 12:54 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2085:3318 - T189101 (duration: 00m 49s) | [production] | 
            
  | 12:45 | <marostegui> | Reload haproxy on dbproxy1011 to repool labsdb1009 - https://phabricator.wikimedia.org/T174047 | [production] | 
            
  | 11:47 | <moritzm> | rebooting dns recursors in codfw for kernel security update | [production] | 
            
  | 11:38 | <jynus> | stopping eqiad s3 replication and shuttinf down db1075 in preparation for reimage T148507 | [production] | 
            
  | 10:49 | <banyek> | shutting down db2068 and dbstore2001:s7 for cloning | [production] | 
            
  | 10:46 | <moritzm> | uploaded debdeploy 0.0.99.5-1+deb9u1 to apt.wikimedia.org/stretch-wikimedia | [production] | 
            
  | 10:33 | <marostegui> | Deploy schema change on s4 eqiad master (db1068) - T187089 | [production] | 
            
  | 10:33 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Slowly repool db2054 - T204127 (duration: 00m 50s) | [production] | 
            
  | 09:57 | <marostegui> | Deploy schema change on s4 eqiad master (db1068) - T89737 | [production] | 
            
  | 09:54 | <godog> | repair sdd on ms-be2043 - T199198 | [production] | 
            
  | 09:44 | <marostegui> | Enable GTID on eqiad masters - T189107 | [production] | 
            
  | 09:41 | <filippo@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=wtp2020.codfw.wmnet | [production] | 
            
  | 09:34 | <moritzm> | rebooting dns recursors in esams for kernel security update | [production] | 
            
  | 09:24 | <marostegui> | Reload haproxy on dbproxy1011 to depool labsdb1009 - https://phabricator.wikimedia.org/T174047 | [production] | 
            
  | 09:08 | <marostegui> | Deploy schema change on s8 eqiad master (db1071) - T89737 | [production] | 
            
  | 08:59 | <hashar> | Bumped CI jobs based on Debian Stretch to use Chrome 69 and Firefox 60 | T203902 | [production] | 
            
  | 08:51 | <moritzm> | rebooting dns recursors in ulsfo for kernel security update | [production] | 
            
  | 08:49 | <marostegui> | Deploy schema change on s5 eqiad master (db1070) - T89737 | [production] | 
            
  | 08:41 | <moritzm> | removed labvirt1019/labvirt1020 from debmonitor (T204004) | [production] | 
            
  | 08:36 | <marostegui> | Deploy schema change on s6 eqiad master (db1061) - T89737 | [production] | 
            
  | 08:29 | <moritzm> | installing curl security updates | [production] | 
            
  | 08:21 | <marostegui> | Deploy schema change on s6 eqiad master (db1061) - T187089 | [production] |