| 2018-09-24
      
      § | 
    
  | 16:58 | <gehel@deploy1001> | Started deploy [wdqs/wdqs@195ea0e]: new version of wdqs GUI and updater  (wdqs1009 only) | [production] | 
            
  | 15:55 | <jynus@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool db1064 (duration: 00m 50s) | [production] | 
            
  | 15:52 | <moritzm> | rebooting rdb1009/rdb1010 for kernel security update | [production] | 
            
  | 15:46 | <marostegui> | Deploy schema change on s8 eqiad master with replication - might generate lag - T204006 | [production] | 
            
  | 15:23 | <jynus> | stop and upgrade db1064 (x1) | [production] | 
            
  | 15:18 | <jynus@deploy1001> | Synchronized wmf-config/db-eqiad.php: depool db1064 (duration: 00m 50s) | [production] | 
            
  | 15:04 | <moritzm> | rebooting rdb1008 for kernel security update | [production] | 
            
  | 14:51 | <moritzm> | rebooting rdb1007 for kernel security update | [production] | 
            
  | 14:30 | <bblack> | upgrade gdnsd 2.99.42 -> 2.99.1729 on authdns2001 | [production] | 
            
  | 14:28 | <moritzm> | rebooting rdb1006 for kernel security update | [production] | 
            
  | 14:28 | <bblack> | upgrade gdnsd 2.99.42 -> 2.99.1729 on authdns1001 | [production] | 
            
  | 14:20 | <moritzm> | rebooting rdb1005 for kernel security update | [production] | 
            
  | 14:18 | <jynus> | stop and upgrade es1017 (es3 eqiad master)- it may create some temporary lag on es3 | [production] | 
            
  | 13:59 | <jynus> | stop and upgrade es1015 (es2 eqiad master)- it may create some temporary lag on es2 | [production] | 
            
  | 13:52 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2089 (duration: 00m 49s) | [production] | 
            
  | 13:43 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2089 (duration: 00m 50s) | [production] | 
            
  | 13:34 | <jynus@deploy1001> | Synchronized wmf-config/db-codfw.php: Setup to db hosts for api-s5 (duration: 00m 50s) | [production] | 
            
  | 13:23 | <moritzm> | rebooting rdb1004 for kernel security update | [production] | 
            
  | 13:12 | <moritzm> | rebooting rdb1003 for kernel security update | [production] | 
            
  | 13:03 | <gehel> | start isolating maps1004 for reimage to stretch - T195285 | [production] | 
            
  | 11:39 | <elukey> | reboot an-master100[1,2] as part of the pre-checks before the hadoop master daemons swap - T203635 | [production] | 
            
  | 11:21 | <moritzm> | installing texlive-bin security updates | [production] | 
            
  | 10:44 | <jdrewniak@deploy1001> | Synchronized portals: Wikimedia Portals Update: [[gerrit:462417|Bumping portals to master (T128546)]] (duration: 00m 49s) | [production] | 
            
  | 10:43 | <jdrewniak@deploy1001> | Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:462417|Bumping portals to master (T128546)]] (duration: 00m 50s) | [production] | 
            
  | 09:42 | <jynus> | stop and upgrade db1066 (s2 eqiad master)- it may create some temporary lag on s2 | [production] | 
            
  | 09:30 | <godog> | upgrade / roll restart thumbor in eqiad / codfw - T20871 T198370 | [production] | 
            
  | 09:03 | <jynus> | stop and upgrade db1067 (s1 eqiad master)- it may create some temporary lag on s1 | [production] | 
            
  | 07:55 | <l10nupdate@deploy1001> | ResourceLoader cache refresh completed at Mon Sep 24 07:55:18 UTC 2018 (duration 10m 50s) | [production] | 
            
  | 07:44 | <bawolff@deploy1001> | scap sync-l10n completed (1.32.0-wmf.22) (duration: 07m 44s) | [production] | 
            
  | 07:31 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2062 (duration: 00m 46s) | [production] | 
            
  | 07:25 | <bawolff_> | manually running l10n update (T205238) | [production] | 
            
  | 07:09 | <moritzm> | installing libarchive-zip-perl security updates | [production] | 
            
  | 07:05 | <volans> | wiping netbox DB to re-import it cleanly from racktables - T199083 | [production] | 
            
  | 06:58 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2062 (duration: 00m 50s) | [production] | 
            
  | 06:57 | <godog> | repair sde on ms-be2041 - T199198 | [production] | 
            
  | 06:57 | <godog> | repair sdn on ms-be1041 - T199198 | [production] | 
            
  | 06:54 | <volans@deploy1001> | Finished deploy [netbox/deploy@5e70423]: Cherry pick of custom fields fix (duration: 00m 05s) | [production] | 
            
  | 06:54 | <volans@deploy1001> | Started deploy [netbox/deploy@5e70423]: Cherry pick of custom fields fix | [production] | 
            
  | 05:48 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2088:3311 (duration: 00m 50s) | [production] | 
            
  | 05:28 | <marostegui> | Deploy schema change on db2088:3311 - T203709 | [production] | 
            
  | 05:27 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2088:3311 (duration: 00m 50s) | [production] | 
            
  | 04:15 | <kartik@deploy1001> | Finished deploy [cxserver/deploy@3e2d668]: Update cxserver to d913793 (T203551, T203780, T202716, T203947) (duration: 04m 20s) | [production] | 
            
  | 04:10 | <kartik@deploy1001> | Started deploy [cxserver/deploy@3e2d668]: Update cxserver to d913793 (T203551, T203780, T202716, T203947) | [production] | 
            
  | 02:48 | <l10nupdate@deploy1001> | ResourceLoader cache refresh completed at Mon Sep 24 02:48:16 UTC 2018 (duration 10m 49s) | [production] | 
            
  | 02:37 | <l10nupdate@deploy1001> | scap sync-l10n completed (1.32.0-wmf.22) (duration: 16m 02s) | [production] |