| 
      
        2016-07-21
      
      §
     | 
  
    
  | 10:52 | 
  <paravoid> | 
  cr2-eqiad: deactivating Transit BGP sessions | 
  [production] | 
            
  | 10:50 | 
  <paravoid> | 
  cr2-eqiad: deactivating IX BGP sessions | 
  [production] | 
            
  | 10:35 | 
  <paravoid> | 
  cr2-eqiad: increase cross-datacenter link OSPF metrics | 
  [production] | 
            
  | 09:47 | 
  <gehel> | 
  reinstalling and configuring relforge1001/1002 - T137256 | 
  [production] | 
            
  | 08:10 | 
  <_joe_> | 
  restarting apache on palladium | 
  [production] | 
            
  | 07:47 | 
  <apergos> | 
  restarted gerrit on ytterbium, it was refusing to complete git fetches for large repos (mw core, puppet...) | 
  [production] | 
            
  | 03:03 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Thu Jul 21 03:03:21 UTC 2016 (duration 7m 2s) | 
  [production] | 
            
  | 02:56 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.11) (duration: 08m 57s) | 
  [production] | 
            
  | 02:30 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.10) (duration: 09m 33s) | 
  [production] | 
            
  
    | 
      
        2016-07-20
      
      §
     | 
  
    
  | 23:43 | 
  <maxsem@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: SWAT, no-op (duration: 00m 24s) | 
  [production] | 
            
  | 23:42 | 
  <dereckson@tin> | 
  Synchronized php-1.28.0-wmf.10/extensions/EventBus/EventBus.hooks.php: Add rev_by_bot flag to revision_create event (2/2) (duration: 00m 23s) | 
  [production] | 
            
  | 23:41 | 
  <dereckson@tin> | 
  Synchronized php-1.28.0-wmf.10/extensions/EventBus/extension.json: Add rev_by_bot flag to revision_create event (1/2) (duration: 00m 26s) | 
  [production] | 
            
  | 23:40 | 
  <dereckson@tin> | 
  Synchronized php-1.28.0-wmf.11/extensions/ORES/: Let ORES extension score for some namespaces instead of all ([[Gerrit:300083]]]) (duration: 00m 30s) | 
  [production] | 
            
  | 23:38 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: ORES score edits in main and Property namespaces in wikidatawiki ([[Gerrit:300086]]) (duration: 00m 33s) | 
  [production] | 
            
  | 21:59 | 
  <mutante> | 
  new language "tcy" (Tulu) has been approved and added today - https://meta.wikimedia.org/wiki/Requests_for_new_languages/Wikipedia_Tulu | 
  [production] | 
            
  | 21:49 | 
  <mutante> | 
  DNS authdns-gen-zones on all servers to add new language tcy (bug T97051) | 
  [production] | 
            
  | 20:28 | 
  <demon@tin> | 
  Synchronized php-1.28.0-wmf.11/extensions/PagedTiffHandler/: (no message) (duration: 00m 25s) | 
  [production] | 
            
  | 20:25 | 
  <ejegg> | 
  updated civicrm from a386eb5a76ec97b3b01c46a49309dfa39bbc58b0 to 2f4805fa2d2a7c57881408be2b3a017d26d8f43e | 
  [production] | 
            
  | 20:20 | 
  <awight> | 
  update paymentswiki from 7c6fb5a3b90fffdf2229cc903fb546e0e1e47998 to f23f15656eb488f5008b45b940077abbaa779004 | 
  [production] | 
            
  | 20:06 | 
  <demon@tin> | 
  Finished scap: group1 to wmf.11 (3rd bestest try) (duration: 46m 03s) | 
  [production] | 
            
  | 19:38 | 
  <urandom> | 
  Starting Casssandra on restbase1011-b.eqiad.wmnet | 
  [production] | 
            
  | 19:20 | 
  <demon@tin> | 
  Started scap: group1 to wmf.11 (3rd bestest try) | 
  [production] | 
            
  | 19:18 | 
  <demon@tin> | 
  scap aborted: group1 to wmf.11 (2nd best try) (duration: 01m 30s) | 
  [production] | 
            
  | 19:17 | 
  <demon@tin> | 
  Started scap: group1 to wmf.11 (2nd best try) | 
  [production] | 
            
  | 19:12 | 
  <demon@tin> | 
  scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="fawiki" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.KjzXNQRvAU" ' returned non-zero exit status 1 (duration: 00m 28s) | 
  [production] | 
            
  | 19:12 | 
  <demon@tin> | 
  Started scap: group1 to wmf.11 | 
  [production] | 
            
  | 17:23 | 
  <urandom> | 
  Stopping restbase1013-c bootstrap (pending better timeouts) : T134016 | 
  [production] | 
            
  | 16:09 | 
  <reedy@tin> | 
  Synchronized wmf-config/extension-list: More to extension registration for l10n (duration: 00m 27s) | 
  [production] | 
            
  | 16:09 | 
  <thcipriani@tin> | 
  Finished scap: SWAT: [[gerrit:299973|Fix spelling of RevisionSlider (T140875)]] (duration: 23m 18s) | 
  [production] | 
            
  | 15:52 | 
  <urandom> | 
  Resuming failed bootstrap on restbase2003.codfw.wmnet : T134016 | 
  [production] | 
            
  | 15:48 | 
  <_joe_> | 
  removed ruthenium from the list of trebuchet minions | 
  [production] | 
            
  | 15:45 | 
  <thcipriani@tin> | 
  Started scap: SWAT: [[gerrit:299973|Fix spelling of RevisionSlider (T140875)]] | 
  [production] | 
            
  | 15:41 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.11/extensions/RevisionSlider/modules/ext.RevisionSlider.HelpDialog.js: SWAT: [[gerrit:299974|Open links in the "tutorial" in the new window (T140875)]] (duration: 00m 27s) | 
  [production] | 
            
  | 15:32 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:299976|Beta: Test OresEnabledNamespaces on enwiki]] (duration: 00m 25s) | 
  [production] | 
            
  | 15:28 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:299992|Let bureaucrats in fawiki remove sysop user group (T140810)]] (duration: 00m 25s) | 
  [production] | 
            
  | 15:26 | 
  <mafk> | 
  SWAT for {{gerrit|299446}} done, which fixes T140544 | 
  [production] | 
            
  | 15:20 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:299446|Configuration changes for he.wikinews.org]] (duration: 00m 28s) | 
  [production] | 
            
  | 15:08 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:299751|Compact Language Links: To beta in ruwikivoyage]] (duration: 00m 33s) | 
  [production] | 
            
  | 15:04 | 
  <urandom> | 
  Ammending my last to include 'Rack b' | 
  [production] | 
            
  | 15:03 | 
  <urandom> | 
  RESTBase Cassandra: raising stream throughput to 25Mbit/s; lowering compaction throughput to 10MB/s : T134016 | 
  [production] | 
            
  | 14:40 | 
  <Jeff_Green> | 
  running authdns-update to deploy SPF/DKIM records for wikipedia.org | 
  [production] | 
            
  | 14:20 | 
  <urandom> | 
  Restarting bootstrap on restbas1013.eqiad.wmnet (duplicate resume?) : T134016 | 
  [production] | 
            
  | 14:05 | 
  <urandom> | 
  Resuming failed bootstrap on restbase1013-c.eqiad.wmnet : T134016 | 
  [production] | 
            
  | 13:29 | 
  <urandom> | 
  Performing rolling RESTBase restart to work-around Cassandra instance restart fallout : T138314 and T138314 | 
  [production] | 
            
  | 13:28 | 
  <urandom> | 
  Restarting restbase1008-a.eqiad.wmnet to apply a (ephemeral) 7200000ms streaming timeout : T138314 | 
  [production] | 
            
  | 13:15 | 
  <paravoid> | 
  cr2-eqiad: setting VRRP priority to 50 for all subnets, effectively switching the VRRP master to cr1-eqiad | 
  [production] | 
            
  | 13:12 | 
  <_joe_> | 
  transitioning wtp2011-2020 | 
  [production] | 
            
  | 12:35 | 
  <_joe_> | 
  transitioning wtp2002-2010 | 
  [production] | 
            
  | 12:35 | 
  <oblivian@palladium> | 
  conftool action : set/pooled=yes; selector: name=wtp2001.codfw.wmnet | 
  [production] | 
            
  | 12:26 | 
  <_joe_> | 
  transition ongoing on wtp2001 | 
  [production] |