| 
      
        2018-06-13
      
      §
     | 
  
    
  | 05:03 | 
  <marostegui> | 
  Deploy schema change on dbstore1001:s1 T191316 T192926 T89737 T195193 | 
  [production] | 
            
  | 03:14 | 
  <l10nupdate@deploy1001> | 
  ResourceLoader cache refresh completed at Wed Jun 13 03:14:53 UTC 2018 (duration 10m 19s) | 
  [production] | 
            
  | 03:04 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.8) (duration: 15m 45s) | 
  [production] | 
            
  | 02:31 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.7) (duration: 12m 21s) | 
  [production] | 
            
  | 01:12 | 
  <aaron@deploy1001> | 
  Synchronized wmf-config/mc.php: Add "memcached-mcrouter" to $wgObjectCaches as default for testwiki (duration: 00m 58s) | 
  [production] | 
            
  | 01:01 | 
  <aaron@deploy1001> | 
  Synchronized php-1.32.0-wmf.7/tests/phpunit/includes/db/LBFactoryTest.php: (no justification provided) (duration: 00m 57s) | 
  [production] | 
            
  | 01:00 | 
  <aaron@deploy1001> | 
  Synchronized php-1.32.0-wmf.7/includes/libs/rdbms/lbfactory/LBFactory.php: f83bad65fce6e6e (duration: 00m 59s) | 
  [production] | 
            
  | 00:55 | 
  <aaron@deploy1001> | 
  Synchronized php-1.32.0-wmf.8/tests/phpunit/includes/db/LBFactoryTest.php: (no justification provided) (duration: 00m 58s) | 
  [production] | 
            
  | 00:53 | 
  <aaron@deploy1001> | 
  Synchronized php-1.32.0-wmf.8/includes/libs/rdbms/lbfactory/LBFactory.php: c2df9668d13 (duration: 00m 58s) | 
  [production] | 
            
  | 00:46 | 
  <bawolff@deploy1001> | 
  Synchronized php-1.32.0-wmf.7/extensions/CentralAuth/AntiSpoof/CentralAuthAntiSpoofHooks.php: Deploy I5f25c529f5bac5c (prevent users from registering previously renamed users) (duration: 00m 57s) | 
  [production] | 
            
  | 00:44 | 
  <bawolff@deploy1001> | 
  Synchronized php-1.32.0-wmf.7/extensions/CentralAuth/extension.json: Deploy I5f25c529f5bac5c (prevent users from registering previously renamed users) (duration: 00m 58s) | 
  [production] | 
            
  | 00:40 | 
  <bawolff@deploy1001> | 
  Synchronized php-1.32.0-wmf.8/extensions/CentralAuth/AntiSpoof/CentralAuthAntiSpoofHooks.php: Deploy I5f25c529f5bac5c (prevent users from registering previously renamed users) (duration: 00m 57s) | 
  [production] | 
            
  | 00:39 | 
  <bawolff@deploy1001> | 
  Synchronized php-1.32.0-wmf.8/extensions/CentralAuth/extension.json: Deploy I5f25c529f5bac5c (prevent users from registering previously renamed users) (duration: 00m 57s) | 
  [production] | 
            
  | 00:37 | 
  <bawolff@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: Deploy I5f25c529f5bac5c (prevent users from registering previously renamed users) (duration: 00m 59s) | 
  [production] | 
            
  
    | 
      
        2018-06-12
      
      §
     | 
  
    
  | 23:47 | 
  <niharika29@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Add sites to the wgCopyUploadsDomains whitelist of Wikimedia Commons T195270, T195928 (duration: 00m 59s) | 
  [production] | 
            
  | 23:05 | 
  <tzatziki> | 
  resetting passwords for compromised accounts (T197046) | 
  [production] | 
            
  | 23:00 | 
  <bblack> | 
  cp3043 - done, reimaged, in live service for cache_upload | 
  [production] | 
            
  | 22:59 | 
  <bblack@neodymium> | 
  conftool action : set/pooled=yes; selector: name=cp3043.esams.wmnet | 
  [production] | 
            
  | 22:37 | 
  <tzatziki> | 
  (from yesterday) resetting passwords for compromised accounts (T197046) | 
  [production] | 
            
  | 22:24 | 
  <twentyafterfour> | 
  phabricator: I scheduled a 24 hour downtime in icinga for the phd service, to give me time to work on this issue. See T196840 | 
  [production] | 
            
  | 22:23 | 
  <twentyafterfour> | 
  phabricator: taking phd offline to relieve the load on the m3 database cluster | 
  [production] | 
            
  | 21:46 | 
  <bblack> | 
  cp3046 - restart varnish backend for mbox lag | 
  [production] | 
            
  | 21:40 | 
  <bblack@neodymium> | 
  conftool action : set/pooled=no; selector: name=cp3043.esams.wmnet | 
  [production] | 
            
  | 21:39 | 
  <bblack> | 
  cp3043 - starting process to move to reimage into cache_upload | 
  [production] | 
            
  | 19:16 | 
  <dduvall@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 to 1.32.0-wmf.8 | 
  [production] | 
            
  | 18:57 | 
  <herron> | 
  restarted icinga service on einsteinium | 
  [production] | 
            
  | 18:42 | 
  <dduvall@deploy1001> | 
  Finished scap: testwiki to php-1.32.0-wmf.8 and rebuild l10n cache (duration: 39m 39s) | 
  [production] | 
            
  | 18:03 | 
  <dduvall@deploy1001> | 
  Started scap: testwiki to php-1.32.0-wmf.8 and rebuild l10n cache | 
  [production] | 
            
  | 17:38 | 
  <ariel@deploy1001> | 
  Finished deploy [dumps/dumps@038c8b3]: sync after snapshot1009 install (duration: 00m 04s) | 
  [production] | 
            
  | 17:37 | 
  <ariel@deploy1001> | 
  Started deploy [dumps/dumps@038c8b3]: sync after snapshot1009 install | 
  [production] | 
            
  | 17:37 | 
  <ariel@deploy1001> | 
  Finished deploy [dumps/dumps@038c8b3]: sync after snapshot1009 install (duration: 00m 07s) | 
  [production] | 
            
  | 17:37 | 
  <ariel@deploy1001> | 
  Started deploy [dumps/dumps@038c8b3]: sync after snapshot1009 install | 
  [production] | 
            
  | 16:54 | 
  <marxarelli> | 
  starting branch cut for 1.32.0-wmf.8 | 
  [production] | 
            
  | 16:11 | 
  <volans@deploy1001> | 
  Finished deploy [debmonitor/deploy@0eca14a]: Release v0.1.3 (duration: 00m 22s) | 
  [production] | 
            
  | 16:11 | 
  <volans@deploy1001> | 
  Started deploy [debmonitor/deploy@0eca14a]: Release v0.1.3 | 
  [production] | 
            
  | 15:40 | 
  <bblack> | 
  cp3034 - nevermind, doing different approach later in the day, still pooled in text for now! | 
  [production] | 
            
  | 15:29 | 
  <bblack> | 
  cp3043 switching from text to upload shortly, downtimed in icinga for 2h - https://gerrit.wikimedia.org/r/c/operations/puppet/+/439936 | 
  [production] | 
            
  | 15:07 | 
  <ema> | 
  cp3039: restart varnish-backend | 
  [production] | 
            
  | 14:38 | 
  <addshore> | 
  file exporter importer slot done | 
  [production] | 
            
  | 14:38 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: FileImporter/Exporter [[gerrit:439876|Enable FileExporter/Importer on group0 wikis]] T195370 (duration: 00m 51s) | 
  [production] | 
            
  | 14:20 | 
  <addshore@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: FileImporter/Exporter [[gerrit:439875|Allow setting of export target for FileExporter]] T195370 (duration: 00m 50s) | 
  [production] | 
            
  | 14:09 | 
  <addshore@deploy1001> | 
  Finished scap: [[gerrit:439900|FileExporter backport]] - Pre deployment backport (extension not yet deployed) (duration: 30m 37s) | 
  [production] | 
            
  | 13:38 | 
  <addshore@deploy1001> | 
  Started scap: [[gerrit:439900|FileExporter backport]] - Pre deployment backport (extension not yet deployed) | 
  [production] | 
            
  | 13:16 | 
  <moritzm> | 
  installing openjdk-8 security updates on restbase-dev along with cassandra restarts | 
  [production] | 
            
  | 12:38 | 
  <ema> | 
  cp3035: restart varnish-be, mbox lag | 
  [production] | 
            
  | 12:34 | 
  <_joe_> | 
  repooling mw1230 after reimaging T196881 | 
  [production] | 
            
  | 12:14 | 
  <marostegui> | 
  Deploy schema change on db1099:3311 T191316 T192926 T89737 T195193 | 
  [production] | 
            
  | 12:14 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1099:3311 for alter table (duration: 00m 52s) | 
  [production] | 
            
  | 12:11 | 
  <marostegui> | 
  Deploy schema change on dbstore1002:s1 T191316 T192926 T89737 T195193 | 
  [production] | 
            
  | 12:05 | 
  <akosiaris@puppetmaster1001> | 
  conftool action : set/weight=10; selector: dc=.*,service=mathoid,cluster=kubernetes,name=.* | 
  [production] |