| 2018-12-21
      
      § | 
    
  | 13:24 | <moritzm> | installing subversion updates from stretch point release | [production] | 
            
  | 12:34 | <reedy@deploy1001> | Synchronized wmf-config/CommonSettings.php: T211494 (duration: 00m 44s) | [production] | 
            
  | 12:31 | <reedy@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T211494 (duration: 00m 45s) | [production] | 
            
  | 12:10 | <moritzm> | rebooting url downloaders to pick up SSBD-enabled QEMU | [production] | 
            
  | 11:38 | <moritzm> | rebooting debug proxies to pick up SSBD-enabled QEMU | [production] | 
            
  | 09:58 | <ema@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=ms-fe2006.codfw.wmnet | [production] | 
            
  | 09:57 | <ema> | repool ms-fe2006 with old certs, test successful T212215#4839960 | [production] | 
            
  | 09:23 | <ema@puppetmaster1001> | conftool action : set/pooled=no; selector: name=ms-fe2006.codfw.wmnet | [production] | 
            
  | 09:22 | <ema> | depool ms-fe2006 to test new TLS certs T212215 | [production] | 
            
  | 08:45 | <moritzm> | upgrading nginx on sodium | [production] | 
            
  | 08:41 | <moritzm> | upgrading nginx on debug proxies | [production] | 
            
  | 06:55 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Remove old comments T211973 (duration: 00m 46s) | [production] | 
            
  | 00:52 | <twentyafterfour> | SWAT Finished. See you all next year! | [production] | 
            
  | 00:50 | <twentyafterfour@deploy1001> | Synchronized php-1.33.0-wmf.9/extensions/MobileFrontend/: SWAT: sync https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MobileFrontend/+/481026 (duration: 00m 48s) | [production] | 
            
  
    | 2018-12-20
      
      § | 
    
  | 22:46 | <mutante> | phab1001 / phabricator: upgraded nodejs package | [production] | 
            
  | 22:39 | <mutante> | phab1001 / phabricator: installing php5 package upgrades | [production] | 
            
  | 19:29 | <kaldari@deploy1001> | Synchronized wmf-config/Wikibase.php: syncing Wikibase for SWAT deployment (duration: 00m 45s) | [production] | 
            
  | 19:27 | <kaldari@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: syncing InitialiseSettings for SWAT deployment (duration: 00m 46s) | [production] | 
            
  | 19:03 | <elukey> | restart hdfs namenode on an-master1002 with new heap settings (currently standby, 8->12G) | [production] | 
            
  | 18:30 | <elukey> | remove hdfs journalnode config+packages from analytics10(28|35) - not used anymore - T209929 | [production] | 
            
  | 18:29 | <elukey> | restart hdfs namenode on an-master1001 with new heap settings (currently standby, 8->12G) | [production] | 
            
  | 18:06 | <mutante> | doc1001 - meged gerrit:480881 and then manually moved the entire /srv/org/wikimedia/doc/ structure into /srv/docroot/srv/org/wikimedia/ and deleted the old dirs T137890 | [production] | 
            
  | 17:53 | <arturo> | updating puppet compiler facts: `PUPPET_COMPILER=compiler1002.puppet-diffs.eqiad.wmflabs modules/puppet_compiler/files/compiler-update-facts` | [production] | 
            
  | 17:49 | <arturo> | updating puppet compiler facts: `PUPPET_COMPILER=compiler1001.puppet-diffs.eqiad.wmflabs modules/puppet_compiler/files/compiler-update-facts` | [production] | 
            
  | 17:05 | <XioNoX> | add 208.80.155.88/29 to cloud-in4 term icmp - T207663 | [production] | 
            
  | 17:02 | <XioNoX> | configure additional 208.80.155.88/29 IPs on cloud-instance-transport1-b-eqiad - T207663 | [production] | 
            
  | 16:59 | <zfilipin@deploy1001> | rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.9 | [production] | 
            
  | 16:51 | <zfilipin@deploy1001> | Synchronized php-1.33.0-wmf.9/extensions/Wikibase: SWAT: [[gerrit:480978|Revert "Fail hard if an entity namespace is not configured." (T212427)]] (duration: 01m 17s) | [production] | 
            
  | 16:31 | <elukey> | remove two journal nodes from the Analytics hadoop cluster - T209929 | [production] | 
            
  | 14:53 | <moritzm> | installing nodejs security updates on maps* (was tested via T211419) | [production] | 
            
  | 14:41 | <moritzm> | restarted etherpad for nodejs security updates | [production] | 
            
  | 14:39 | <elukey> | add two journal nodes to the Analytics Hadoop cluster - T209929 | [production] | 
            
  | 14:30 | <moritzm> | installing libdap updates from stretch point release | [production] | 
            
  | 14:26 | <moritzm> | rearmed keyholder on netmon1002 after reboot | [production] | 
            
  | 14:22 | <moritzm> | rebooting netmon1002 for kernel security update | [production] | 
            
  | 13:09 | <moritzm> | installing xapian-core updates from stretch point release | [production] | 
            
  | 12:53 | <arturo> | T209616 installing cloudvirt1030, icinga downtime for 1 day | [production] | 
            
  | 12:48 | <kartik@deploy1001> | Finished deploy [cxserver/deploy@16f65cb]: Update cxserver to 803baa4 (T210581, T211889, T144467, T209473) (duration: 04m 42s) | [production] | 
            
  | 12:47 | <moritzm> | installing libxcursor security updates | [production] | 
            
  | 12:43 | <kartik@deploy1001> | Started deploy [cxserver/deploy@16f65cb]: Update cxserver to 803baa4 (T210581, T211889, T144467, T209473) | [production] | 
            
  | 12:31 | <moritzm> | installing fuse updates from stretch point release | [production] | 
            
  | 12:09 | <jdrewniak@deploy1001> | Synchronized portals: Wikimedia Portals Update: [[gerrit:478659| Bumping portals to master (T128546, T202497)]] (duration: 00m 52s) | [production] | 
            
  | 12:08 | <jdrewniak@deploy1001> | Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:478659| Bumping portals to master (T128546, T202497)]] (duration: 00m 53s) | [production] | 
            
  | 12:02 | <moritzm> | draining restbase1016 for eventual reboot for kernel security update | [production] | 
            
  | 11:46 | <moritzm> | draining restbase1015 for eventual reboot for kernel security update | [production] | 
            
  | 11:34 | <moritzm> | powercycling restbase1014, similar EFI ASSERT error to T212305 | [production] | 
            
  | 11:34 | <moritzm> | powercycling restbase1014, similar EFI ASSEER error to T212305 | [production] | 
            
  | 11:21 | <moritzm> | draining restbase1014 for eventual reboot for kernel security update | [production] | 
            
  | 10:59 | <banyek> | executing schema change on db1068 (s4 master) - T85757 | [production] | 
            
  | 10:58 | <banyek@deploy1001> | Synchronized wmf-config/db-eqiad.php: repool db1121 after schema change - T85757 (duration: 00m 52s) | [production] |