| 
      
        2016-09-23
      
      §
     | 
  
    
  | 02:46 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Fri Sep 23 02:46:04 UTC 2016 (duration 6m 10s) | 
  [production] | 
            
  | 02:39 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.20) (duration: 17m 04s) | 
  [production] | 
            
  | 02:13 | 
  <maxsem@tin> | 
  Synchronized php-1.28.0-wmf.20/extensions/SecurePoll/: https://gerrit.wikimedia.org/r/#/c/312450/1 (duration: 00m 51s) | 
  [production] | 
            
  | 02:10 | 
  <mutante> | 
  mw1206, mw1224 - restarted hhvm and apache  | 
  [production] | 
            
  | 01:49 | 
  <bblack> | 
  depooled mw1224 service apache2 | 
  [production] | 
            
  | 00:38 | 
  <Krenair> | 
  mw1224 apache stuck, not restarting for now in case someone wants to investigate later. possibly T89912? | 
  [production] | 
            
  | 00:17 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/312339 (duration: 00m 48s) | 
  [production] | 
            
  | 00:16 | 
  <krenair@tin> | 
  Synchronized wmf-config/mobile.php: https://gerrit.wikimedia.org/r/312339 (duration: 00m 47s) | 
  [production] | 
            
  | 00:08 | 
  <krenair@tin> | 
  Synchronized php-1.28.0-wmf.20/extensions/FlaggedRevs/business/RevisionReviewForm.php: https://gerrit.wikimedia.org/r/#/c/312423/ (duration: 00m 48s) | 
  [production] | 
            
  
    | 
      
        2016-09-22
      
      §
     | 
  
    
  | 23:46 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/310483 (duration: 00m 48s) | 
  [production] | 
            
  | 23:19 | 
  <krenair@tin> | 
  Synchronized php-1.28.0-wmf.20/resources/src/mediawiki.less/mediawiki.ui/mixins.less: https://gerrit.wikimedia.org/r/#/c/312340/ (duration: 00m 48s) | 
  [production] | 
            
  | 22:49 | 
  <aaron@tin> | 
  Synchronized php-1.28.0-wmf.20/includes/libs/rdbms/loadbalancer/LoadBalancer.php: a73a7ef9286275f797411646f9c5af60d4894c73 (duration: 01m 04s) | 
  [production] | 
            
  | 22:18 | 
  <mutante> | 
  added slaporte and zhousquared to wmf LDAP group (T146227) | 
  [production] | 
            
  | 21:24 | 
  <hasharAway> | 
  Nodepool is all back and operational.  Reduced amount of queries to the OpenStack API by more than 10% | 
  [production] | 
            
  | 21:14 | 
  <yurik> | 
  deployed tilerator https://gerrit.wikimedia.org/r/#/c/312329/ | 
  [production] | 
            
  | 21:05 | 
  <hasharAway> | 
  stopped nodepooled and restarted it with 0.1.1-wmf5 | 
  [production] | 
            
  | 21:04 | 
  <mutante> | 
  upgradede nodepool to 0.1.1-wmf5 on labnodepool1001 | 
  [production] | 
            
  | 21:04 | 
  <mutante> | 
  upgradede nodepool to 0.1.1-wmf on labnodepool1001 | 
  [production] | 
            
  | 21:04 | 
  <krinkle@tin> | 
  Synchronized php-1.28.0-wmf.20/resources/src/mediawiki/mediawiki.js: T146099 (duration: 00m 48s) | 
  [production] | 
            
  | 21:02 | 
  <mutante> | 
  imported nodepool_0.1.1-wmf5_amd64 into jessie-wikimedia (T145142) | 
  [production] | 
            
  | 20:52 | 
  <urandom> | 
  T133395: RESTBase Staging: starting dumps (3, eqiad) | 
  [production] | 
            
  | 20:47 | 
  <urandom> | 
  T133395: RESTBase Staging: altering table to set TWCS on wikipedia parsoid.html table | 
  [production] | 
            
  | 20:20 | 
  <urandom> | 
  T133395: RESTBase Staging: Restarting Cassandra to pick up TWCS jar in classpath | 
  [production] | 
            
  | 20:08 | 
  <thcipriani@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.28.0-wmf.20 | 
  [production] | 
            
  | 20:00 | 
  <thcipriani> | 
  rolling out wmf.20 to all wikis | 
  [production] | 
            
  | 19:23 | 
  <SMalyshev> | 
  Deploying new version of WDQS GUI | 
  [production] | 
            
  | 19:09 | 
  <thcipriani@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.28.0-wmf.20 | 
  [production] | 
            
  | 19:01 | 
  <thcipriani> | 
  wmf.20 to group1 will watch until 20 UTC and move forward to all wikis | 
  [production] | 
            
  | 18:56 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.18/extensions/CentralNotice: SWAT: [[gerrit:312292|Update extensions/CentralNotice submodule (T144952)]] (duration: 00m 50s) | 
  [production] | 
            
  | 18:50 | 
  <yuvipanda> | 
  enabling puppet on labcontrol1001, run on labtestcontrol2001 seems ok | 
  [production] | 
            
  | 18:47 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.20/extensions/CentralNotice: SWAT: [[gerrit:312293|Update extensions/CentralNotice submodule (T144952)]] (duration: 00m 52s) | 
  [production] | 
            
  | 18:46 | 
  <yuvipanda> | 
  disable puppet on labcontrol1001 for https://gerrit.wikimedia.org/r/#/c/312301/ | 
  [production] | 
            
  | 18:38 | 
  <aaron@tin> | 
  Synchronized php-1.28.0-wmf.20/includes/libs/rdbms/database/Database.php: 844cfd568a7c7953faa6ac69acebff1cee943b7f & 014a420b4525798b1202cc488b337acdaf09c49a (duration: 00m 49s) | 
  [production] | 
            
  | 18:31 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.19/extensions/TimedMediaHandler/MwEmbedModules: SWAT: [[gerrit:312264|Update ogv.js to 1.2.0 (T145983)]] (duration: 00m 48s) | 
  [production] | 
            
  | 18:28 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.18/extensions/TimedMediaHandler/MwEmbedModules: SWAT: [[gerrit:312263|Update ogv.js to 1.2.0 (T145983)]] (duration: 00m 51s) | 
  [production] | 
            
  | 17:24 | 
  <moritzm> | 
  rebooting ms-be1016, high load caused by XFS bug | 
  [production] | 
            
  | 17:09 | 
  <moritzm> | 
  rolling reboot of trusty swift backend servers in eqiad completed | 
  [production] | 
            
  | 16:40 | 
  <elukey> | 
  forced logrotation for /etc/logrotate.d/upstart on labvirt1014 to investigate cronspam | 
  [production] | 
            
  | 16:17 | 
  <godog> | 
  offline sdd on ms-be1004 via megacli T144499 | 
  [production] | 
            
  | 15:29 | 
  <mobrovac> | 
  restbase deploy end of d96fbc1 | 
  [production] | 
            
  | 15:10 | 
  <mobrovac> | 
  restbase deploy start of d96fbc1 | 
  [production] | 
            
  | 15:02 | 
  <bblack> | 
  upgrading openssl on cp* | 
  [production] | 
            
  | 13:22 | 
  <moritzm> | 
  resume rolling reboot of trusty swift backend servers in eqiad for kernel security update | 
  [production] | 
            
  | 13:02 | 
  <moritzm> | 
  uploaded openssl 1.0.2i for jessie-wikimedia to carbon | 
  [production] | 
            
  | 12:31 | 
  <hashar@tin> | 
  Synchronized php-1.28.0-wmf.20/extensions/Popups: Merge mw.popups.experiment into mw.popups.core T146035 (duration: 00m 49s) | 
  [production] | 
            
  | 12:27 | 
  <mobrovac> | 
  restbase deploy end of d5538ad | 
  [production] | 
            
  | 12:25 | 
  <elukey> | 
  installing varnishkafka 1.0.12 on cache:upload ulsfo and eqiad | 
  [production] | 
            
  | 12:24 | 
  <akosiaris> | 
  uploaded to apt.wikimedia.org jessie-wikimedia: apertium-es-ro_0.7.3~r57551-2+wmf1 | 
  [production] | 
            
  | 12:13 | 
  <hashar> | 
  Early SWAT for mobile team ( https://gerrit.wikimedia.org/r/#/c/311977/ ) | 
  [production] | 
            
  | 12:11 | 
  <mobrovac> | 
  restbase deploy start of d5538ad | 
  [production] |