| 2019-01-03
      
      ยง | 
    
  | 23:08 | <volans> | restarted pdfrender on scb1004 | [production] | 
            
  | 22:29 | <volans> | restarted all slaves on dbstore1002 (relayed from banyek) | [production] | 
            
  | 22:14 | <banyek> | stopping all slaves on dbstore1002 (NOT labsdb) | [production] | 
            
  | 22:14 | <banyek> | stopping all slaves on labsdb1002 | [production] | 
            
  | 20:50 | <reedy@deploy1001> | Synchronized multiversion/MWMultiVersion.php: Fix error for testcommons (duration: 00m 44s) | [production] | 
            
  | 20:46 | <reedy@deploy1001> | Synchronized dblists/group0.dblist: Add testcommonswiki to group0 (duration: 00m 43s) | [production] | 
            
  | 20:43 | <reedy@deploy1001> | Synchronized wmf-config/interwiki.php: Updating interwiki cache (duration: 02m 05s) | [production] | 
            
  | 20:24 | <reedy@deploy1001> | Synchronized wmf-config/db-codfw.php: T197616 (duration: 00m 44s) | [production] | 
            
  | 20:23 | <reedy@deploy1001> | Synchronized wmf-config/db-eqiad.php: T197616 (duration: 00m 44s) | [production] | 
            
  | 20:13 | <reedy@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T197616 (duration: 00m 44s) | [production] | 
            
  | 20:12 | <reedy@deploy1001> | Synchronized multiversion/MWMultiVersion.php: T197616 (duration: 00m 44s) | [production] | 
            
  | 20:11 | <reedy@deploy1001> | rebuilt and synchronized wikiversions files: T197616 | [production] | 
            
  | 20:09 | <reedy@deploy1001> | Synchronized dblists/: T197616 (duration: 00m 45s) | [production] | 
            
  | 18:51 | <bsitzmann@deploy1001> | Finished deploy [mobileapps/deploy@1182b3b]: Update mobileapps to f6ad0e5: Set timeout for backend /page/html requests, part 2 (duration: 05m 27s) | [production] | 
            
  | 18:46 | <bsitzmann@deploy1001> | Started deploy [mobileapps/deploy@1182b3b]: Update mobileapps to f6ad0e5: Set timeout for backend /page/html requests, part 2 | [production] | 
            
  | 18:37 | <bsitzmann@deploy1001> | Finished deploy [mobileapps/deploy@c470ed2]: Update mobileapps to f6ad0e5: Set timeout for backend /page/html requests (duration: 04m 11s) | [production] | 
            
  | 18:33 | <bsitzmann@deploy1001> | Started deploy [mobileapps/deploy@c470ed2]: Update mobileapps to f6ad0e5: Set timeout for backend /page/html requests | [production] | 
            
  | 18:21 | <volans> | restart pdfrender on scb1003 | [production] | 
            
  | 17:58 | <ariel@deploy1001> | Finished deploy [dumps/dumps@10dc8ad]: return properly if commands failed (duration: 00m 08s) | [production] | 
            
  | 17:58 | <ariel@deploy1001> | Started deploy [dumps/dumps@10dc8ad]: return properly if commands failed | [production] | 
            
  | 16:32 | <XioNoX> | remove old 10.64.22.0/24 IPs from cloud-instance-transport1-b-eqiad - T207663 | [production] | 
            
  | 16:22 | <moritzm> | rebooting kubernetes workers in eqiad for kernel security update | [production] | 
            
  | 16:02 | <arturo> | reimaging cloudvirt1013 cloudvirt1026-1028 to stretch | [production] | 
            
  | 15:48 | <moritzm> | restart parsoid on wtp1025 to pick up OpenSSL update for nodejs | [production] | 
            
  | 15:43 | <jijiki> | Enabled puppet on mw servers after merging 481796 - T197616 | [production] | 
            
  | 15:31 | <jijiki> | Disabling puppet on mw servers to test 481796 - T197616 | [production] | 
            
  | 15:14 | <ejegg> | updated Fundraising CiviCRM from b33dcd3c94 to bcb4b7a7d1 | [production] | 
            
  | 14:37 | <moritzm> | rebooting kubernetes workers in codfw for kernel security update | [production] | 
            
  | 14:37 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: repool db1101:3317 after schema change - T85757 (duration: 00m 44s) | [production] | 
            
  | 14:32 | <banyek> | repooling db1101:3317 after schema change - T85757 | [production] | 
            
  | 14:21 | <moritzm> | rebooting kubernetes masters in eqiad to pick up SSBD-enabled qemu | [production] | 
            
  | 14:14 | <moritzm> | rebooting kubernetes mastes in codfw to pick up SSBD-enabled qemu | [production] | 
            
  | 14:05 | <arturo> | T209616 reimage cloudvirt1029 as debian stretch | [production] | 
            
  | 13:43 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: depool db1101:3317 for schema change - T85757 (duration: 00m 44s) | [production] | 
            
  | 13:41 | <banyek> | depooling db1101:3317 for schema change - T85757 | [production] | 
            
  | 13:38 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: repool db1098:3317 after schema change - T85757 (duration: 00m 44s) | [production] | 
            
  | 13:34 | <banyek> | repooling db1098:3317 after schema change - T85757 | [production] | 
            
  | 13:24 | <kartik@deploy1001> | Finished deploy [cxserver/deploy@3b2ede7]: Update cxserver to 2369a18 (duration: 04m 30s) | [production] | 
            
  | 13:20 | <kartik@deploy1001> | Started deploy [cxserver/deploy@3b2ede7]: Update cxserver to 2369a18 | [production] | 
            
  | 12:58 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: depool db1098:3317 for schema change - T85757 (duration: 00m 45s) | [production] | 
            
  | 12:55 | <banyek> | depooling db1098:3317 for schema change - T85757 | [production] | 
            
  | 12:54 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: repool db1094 after schema change - T85757 (duration: 00m 45s) | [production] | 
            
  | 12:49 | <banyek> | repooling db1094 after schema change - T85757 | [production] | 
            
  | 12:41 | <arturo> | T212302 reimaging again cloudvirt1030 to test final puppet code | [production] | 
            
  | 12:33 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: depool db1094 for schema change - T85757 (duration: 00m 46s) | [production] | 
            
  | 12:28 | <banyek> | depooling db1094 for schema change - T85757 | [production] | 
            
  | 12:27 | <moritzm> | restarting tor on torrelay1001 to pick up OpenSSL security update | [production] | 
            
  | 11:02 | <_joe_> | manually reloading icinga to pick up changes to commands.cfg | [production] | 
            
  | 10:55 | <moritzm> | installing apache updates on puppetmasters | [production] | 
            
  | 10:22 | <moritzm> | installing ghostscript security updates on jessie | [production] |