| 2015-12-17
      
      § | 
    
  | 18:04 | <robh> | calcium is supposed to be down, reclaiming to spares, ignore any irc alerts (its in maint mode in icinga) | [production] | 
            
  | 17:55 | <gwicke> | running `nodetool cleanup` on restbase1002 | [production] | 
            
  | 17:18 | <jynus> | setting mysql db1031 as db2009's master | [production] | 
            
  | 17:05 | <jynus> | restarting and reconfiguring mysql at db2009 | [production] | 
            
  | 16:49 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.8/extensions/Math: SWAT: Make math usable without RESTbase [[gerrit:259734]] (duration: 00m 30s) | [production] | 
            
  | 16:41 | <jynus> | problems with corruption on x1-slave for cebwiki. Fixed them. Will leave db1031 depooled for a while to check they are gone. | [production] | 
            
  | 16:39 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.9/extensions/ContentTranslation/includes/Translation.php: SWAT: Fix Undefined index: targetRevisionId in ContentTranslation [[gerrit:259649]] (duration: 00m 29s) | [production] | 
            
  | 16:23 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.9/extensions/WikimediaEvents/WikimediaEventsHooks.php: SWAT: Actually define tags for cross-wiki upload A/B test [[gerrit:259729]] (duration: 00m 31s) | [production] | 
            
  | 16:17 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable cross-wiki upload A/B test in additional languages [[gerrit:259665]] (duration: 00m 30s) | [production] | 
            
  | 16:11 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.9/extensions/CirrusSearch/includes/CirrusSearch.php: SWAT: Fix array-to-string conversion [[gerrit:259633]] (duration: 00m 30s) | [production] | 
            
  | 15:57 | <moritzm> | stopping opendj LDAP servers on nembus/neptunium (read-only since about days now due to migration to openldap) | [production] | 
            
  | 15:56 | <akosiaris> | depool sca1001, playing with cxserver config | [production] | 
            
  | 15:24 | <jynus> | reinstall, reboot and reconfigure mysql at db1031 | [production] | 
            
  | 15:03 | <moritzm> | installing git security updates | [production] | 
            
  | 14:23 | <_joe_> | restarting HHVM on the first jobrunners | [production] | 
            
  | 14:11 | <jynus> | soft-rebooting mw1004, responsive to ping, but not to salt, ssh | [production] | 
            
  | 14:06 | <godog> | nodetool stop -- CLEANUP on restbase1002 | [production] | 
            
  | 13:59 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1031 (x1-slave) for maintenance (duration: 07m 30s) | [production] | 
            
  | 13:11 | <moritzm> | starting to restart hhvm on application servers (to effect security updates for libxml2, openssl and others) | [production] | 
            
  | 13:01 | <akosiaris> | moved old cruft /srv/deployment/cxserver/deploy/src/config.js out of the way | [production] | 
            
  | 12:51 | <akosiaris> | repooled sca1002 for cxserver | [production] | 
            
  | 12:51 | <akosiaris> | restarted cxserver on sca1002 | [production] | 
            
  | 12:33 | <akosiaris> | depooling sca1002 for cxserver | [production] | 
            
  | 12:33 | <akosiaris> | repooling sca1001 for cxserver | [production] | 
            
  | 12:24 | <kart_> | Updated cxserver on sca1002 | [production] | 
            
  | 12:08 | <akosiaris> | disable puppet, stop salt-minion on sca1002 | [production] | 
            
  | 12:08 | <akosiaris> | depool sca1001 from cxserver service. | [production] | 
            
  | 11:56 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1041 with low weight (duration: 00m 37s) | [production] | 
            
  | 11:33 | <jynus> | performing schema change on officewiki | [production] | 
            
  | 11:31 | <jynus> | performing schema change on x1-master flowdb | [production] | 
            
  | 11:20 | <jynus> | performing schema change on x1-master wikishared.cx_translations | [production] | 
            
  | 10:47 | <jynus> | performing schema change on s7-master metawiki.oauth_registered_consumer | [production] | 
            
  | 10:30 | <godog> | nodetool stop -- CLEANUP restbase1004 | [production] | 
            
  | 02:53 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Dec 17 02:53:11 UTC 2015 (duration 7m 12s) | [production] | 
            
  | 02:45 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.9) (duration: 08m 22s) | [production] | 
            
  | 02:27 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.8) (duration: 10m 32s) | [production] | 
            
  | 00:28 | <ori@tin> | Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: I552cf6b0420: Upgrade some ApiStashEdit logging calls to info() (duration: 00m 30s) | [production] | 
            
  | 00:05 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: Password policy for sysadmin group (duration: 00m 29s) | [production] | 
            
  
    | 2015-12-16
      
      § | 
    
  | 23:37 | <ori@tin> | Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: local hack some extra debug logging into ApiStashEdit (take 2) (duration: 00m 30s) | [production] | 
            
  | 23:32 | <ori@tin> | Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: local hack some extra debug logging into ApiStashEdit (duration: 00m 30s) | [production] | 
            
  | 23:27 | <yurik@tin> | Synchronized php-1.27.0-wmf.9/extensions/Graph/: Deployed Graph ext to master - protocol issue (duration: 00m 32s) | [production] | 
            
  | 23:10 | <legoktm> | running foreachwiki extensions/CentralAuth/maintenance/checkLocalUser.php --verbose=1 --delete=1 (T119736) | [production] | 
            
  | 23:07 | <robh> | fermium returned to normal service with new/renewed ssl cert for lists.w.o | [production] | 
            
  | 22:58 | <robh> | disabled puppet on fermium for the lists.w.o cert update | [production] | 
            
  | 22:58 | <robh> | librenms returned to normal service (puppet renabled on netmon1001) | [production] | 
            
  | 22:48 | <robh> | puppet disabled on netmon1001 for librenms certificate update | [production] | 
            
  | 21:31 | <subbu> | finished deploying parsoid 64029e12 | [production] | 
            
  | 21:29 | <bearND> | mobileapps deployed sha1 9f91ad5 | [production] | 
            
  | 21:23 | <subbu> | restarted parsoid on wtp1005 as a canary | [production] | 
            
  | 21:19 | <subbu> | starting parsoid deploy | [production] |