| 2016-07-20
      
      § | 
    
  | 19:12 | <demon@tin> | Started scap: group1 to wmf.11 | [production] | 
            
  | 17:23 | <urandom> | Stopping restbase1013-c bootstrap (pending better timeouts) : T134016 | [production] | 
            
  | 16:09 | <reedy@tin> | Synchronized wmf-config/extension-list: More to extension registration for l10n (duration: 00m 27s) | [production] | 
            
  | 16:09 | <thcipriani@tin> | Finished scap: SWAT: [[gerrit:299973|Fix spelling of RevisionSlider (T140875)]] (duration: 23m 18s) | [production] | 
            
  | 15:52 | <urandom> | Resuming failed bootstrap on restbase2003.codfw.wmnet : T134016 | [production] | 
            
  | 15:48 | <_joe_> | removed ruthenium from the list of trebuchet minions | [production] | 
            
  | 15:45 | <thcipriani@tin> | Started scap: SWAT: [[gerrit:299973|Fix spelling of RevisionSlider (T140875)]] | [production] | 
            
  | 15:41 | <thcipriani@tin> | Synchronized php-1.28.0-wmf.11/extensions/RevisionSlider/modules/ext.RevisionSlider.HelpDialog.js: SWAT: [[gerrit:299974|Open links in the "tutorial" in the new window (T140875)]] (duration: 00m 27s) | [production] | 
            
  | 15:32 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:299976|Beta: Test OresEnabledNamespaces on enwiki]] (duration: 00m 25s) | [production] | 
            
  | 15:28 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:299992|Let bureaucrats in fawiki remove sysop user group (T140810)]] (duration: 00m 25s) | [production] | 
            
  | 15:26 | <mafk> | SWAT for {{gerrit|299446}} done, which fixes T140544 | [production] | 
            
  | 15:20 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:299446|Configuration changes for he.wikinews.org]] (duration: 00m 28s) | [production] | 
            
  | 15:08 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:299751|Compact Language Links: To beta in ruwikivoyage]] (duration: 00m 33s) | [production] | 
            
  | 15:04 | <urandom> | Ammending my last to include 'Rack b' | [production] | 
            
  | 15:03 | <urandom> | RESTBase Cassandra: raising stream throughput to 25Mbit/s; lowering compaction throughput to 10MB/s : T134016 | [production] | 
            
  | 14:40 | <Jeff_Green> | running authdns-update to deploy SPF/DKIM records for wikipedia.org | [production] | 
            
  | 14:20 | <urandom> | Restarting bootstrap on restbas1013.eqiad.wmnet (duplicate resume?) : T134016 | [production] | 
            
  | 14:05 | <urandom> | Resuming failed bootstrap on restbase1013-c.eqiad.wmnet : T134016 | [production] | 
            
  | 13:29 | <urandom> | Performing rolling RESTBase restart to work-around Cassandra instance restart fallout : T138314 and T138314 | [production] | 
            
  | 13:28 | <urandom> | Restarting restbase1008-a.eqiad.wmnet to apply a (ephemeral) 7200000ms streaming timeout : T138314 | [production] | 
            
  | 13:15 | <paravoid> | cr2-eqiad: setting VRRP priority to 50 for all subnets, effectively switching the VRRP master to cr1-eqiad | [production] | 
            
  | 13:12 | <_joe_> | transitioning wtp2011-2020 | [production] | 
            
  | 12:35 | <_joe_> | transitioning wtp2002-2010 | [production] | 
            
  | 12:35 | <oblivian@palladium> | conftool action : set/pooled=yes; selector: name=wtp2001.codfw.wmnet | [production] | 
            
  | 12:26 | <_joe_> | transition ongoing on wtp2001 | [production] | 
            
  | 12:23 | <oblivian@palladium> | conftool action : set/pooled=no; selector: name=wtp2001.codfw.wmnet | [production] | 
            
  | 12:22 | <oblivian@palladium> | conftool action : set/pooled=no; selector: name= | [production] | 
            
  | 12:17 | <_joe_> | disabling puppet on all parsoid hosts for the transition to service-runner T90668 | [production] | 
            
  | 07:28 | <elukey> | restarting evenbus on kafka100[12] (T140848) | [production] | 
            
  | 06:14 | <_joe_> | updating parsoid on wtp100[12] | [production] | 
            
  | 04:55 | <mutante> | osmium package chromium-browser is missing after upgrade, refered to by jsbench | [production] | 
            
  | 04:45 | <mutante> | osmium - rsyncing /home , /srv (except /srv/mediawiki created by puppet) back from temp backup on hafnium | [production] | 
            
  | 04:38 | <mutante> | osmium result: boots into 4.4 kernel which would not work before.. lol | [production] | 
            
  | 04:35 | <mutante> | osmium edit grub config to boot second entry (3.16), update-grub, reboot | [production] | 
            
  | 03:00 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Wed Jul 20 03:00:43 UTC 2016 (duration 6m 50s) | [production] | 
            
  | 02:53 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.11) (duration: 07m 55s) | [production] | 
            
  | 02:29 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.10) (duration: 08m 41s) | [production] | 
            
  | 02:28 | <demon@tin> | Synchronized wmf-config/CommonSettings.php: delist codereview (duration: 00m 27s) | [production] | 
            
  | 01:39 | <demon@tin> | Synchronized tests/SiteConfiguration.php: for completeness (duration: 00m 24s) | [production] | 
            
  | 01:35 | <mutante> | hafnium stopping rsyncd, deleting configs | [production] | 
            
  | 01:33 | <mutante> | labstore1005, accepting salt key (reinstall 2016-06-25) | [production] | 
            
  | 01:29 | <mutante> | rhodium, new puppetmaster, add to salt | [production] | 
            
  | 01:19 | <mutante> | osmium - revoke old puppet cert, salt-key .. sign new ones | [production] | 
            
  | 01:19 | <mutante> | osmium - after reinstall with jessie, did not boot with 4.4 kernel, _does_ boot with 3.16.04.. still jessie just booted manually into the older kernel in grub | [production] | 
            
  | 01:13 | <demon@tin> | Synchronized wmf-config/abusefilter.php: Disable abusefilter profiling on commonswiki (duration: 00m 26s) | [production] | 
            
  | 00:44 | <dereckson@tin> | Finished scap: wmf-config/ upgrade: Gerrit changes 296770, 296767, 296929, 296930, 292623, 292624 (duration: 45m 42s) | [production] | 
            
  | 00:42 | <urandom> | Temporarily reducing compaction throughput to 10MB/s on restbase1013-c.eqiad.wmnet : T134016 | [production] | 
            
  | 00:32 | <mutante> | osmium - reboot into PXE, reinstall | [production] |