| 
      
        2016-05-12
      
      §
     | 
  
    
  | 16:30 | 
  <mattflaschen@tin> | 
  Synchronized wmf-config/CommonSettings-labs.php: Beta Cluster change (duration: 00m 29s) | 
  [production] | 
            
  | 16:30 | 
  <jynus> | 
  running "megacli -PDOffline -PhysDrv '[32:8]' -aALL" on db1023 | 
  [production] | 
            
  | 16:29 | 
  <mattflaschen@tin> | 
  Synchronized wmf-config/db-labs.php: Beta Cluster change (duration: 00m 27s) | 
  [production] | 
            
  | 16:23 | 
  <gehel> | 
  removing wdqs1001 from rotation for disk upgrade (T120714) | 
  [production] | 
            
  | 16:11 | 
  <gehel> | 
  merged and applied https://gerrit.wikimedia.org/r/#/c/288376/ (T134456) | 
  [production] | 
            
  | 16:10 | 
  <akosiaris> | 
  migrate all ganeti instances except alsafi off ganeti2006 | 
  [production] | 
            
  | 16:10 | 
  <akosiaris> | 
  evacuate ganeti2006.codfw.wmnet from ganeti secondary instances | 
  [production] | 
            
  | 16:06 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.1/extensions/Wikidata/extensions/ArticlePlaceholder/includes/SearchHookHandler.php: SWAT: Update ArticlePlaceholder - Fix notability checks and props params [[gerrit:288396]] (duration: 00m 24s) | 
  [production] | 
            
  | 16:03 | 
  <thcipriani@tin> | 
  Synchronized php-1.27.0-wmf.23/maintenance/updateCollation.php: SWAT: updateCollation.php sql changes backport (duration: 00m 26s) | 
  [production] | 
            
  | 15:52 | 
  <ema> | 
  testing file storage on misc eqiad+esams T134989 | 
  [production] | 
            
  | 15:46 | 
  <thcipriani@tin> | 
  Synchronized php-1.27.0-wmf.23/extensions/Wikidata/extensions/ArticlePlaceholder/includes/SearchHookHandler.php: SWAT: Update ArticlePlaceholder - Fix notability checks and props params [[gerrit:288396]] (duration: 00m 25s) | 
  [production] | 
            
  | 15:41 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.1/maintenance/updateCollation.php: SWAT: updateCollation.php sql updates [[gerrit:288386]] [[gerrit:288384]] [[gerrit:288385]] (duration: 00m 26s) | 
  [production] | 
            
  | 15:25 | 
  <thcipriani@tin> | 
  Synchronized php-1.27.0-wmf.23/extensions/ContentTranslation/modules/tools/ext.cx.tools.mt.js: SWAT: MT: Use custom labels instead of provider id [[gerrit:288342]] (duration: 00m 26s) | 
  [production] | 
            
  | 15:20 | 
  <thcipriani@tin> | 
  Synchronized php-1.28.0-wmf.1/extensions/ContentTranslation/modules/tools/ext.cx.tools.mt.js: SWAT: MT: Use custom labels instead of provider id [[gerrit:288171]] (duration: 00m 25s) | 
  [production] | 
            
  | 15:13 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Bump CirrusSearchRequestSet avro schema to rev 121456865906 PART II [[gerrit:287973]] (duration: 00m 25s) | 
  [production] | 
            
  | 15:12 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/event-schemas: SWAT: Bump CirrusSearchRequestSet avro schema to rev 121456865906 PART I [[gerrit:287973]] (duration: 00m 26s) | 
  [production] | 
            
  | 15:06 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/CirrusSearch-common.php: SWAT: Remove deprecated settings [[gerrit:281966]] (duration: 00m 34s) | 
  [production] | 
            
  | 14:55 | 
  <jynus> | 
  executing random CHANGE MASTERS and crashes on db200[89] trying to break them | 
  [production] | 
            
  | 14:50 | 
  <godog> | 
  downtime flapping redis replication on rdb2006 alert for 10d | 
  [production] | 
            
  | 14:45 | 
  <jynus> | 
  enabling GTID on x1 shard (except db1029) | 
  [production] | 
            
  | 13:34 | 
  <ema> | 
  upgrading misc to varnish 4.1.2-1wm4 and wiping caches (T134989) | 
  [production] | 
            
  | 13:30 | 
  <elukey> | 
  restarted memcached on mc1007 with chunk growth factor set to 1.15 - Part of a perf experiment (T129963)  | 
  [production] | 
            
  | 13:14 | 
  <jynus> | 
  deploying GTID replication on all of es3 shard | 
  [production] | 
            
  | 13:08 | 
  <volans> | 
  disabling puppet and restarting MySQL on db2040 to test change 288361 (in scheduled downtime on icinga) T133780 | 
  [production] | 
            
  | 12:40 | 
  <godog> | 
  kill duplicate ircecho daemon on neon | 
  [production] | 
            
  | 12:39 | 
  <akosiaris> | 
  upgraded nodejs on etherpad1001 to nodejs 4.3 | 
  [production] | 
            
  | 12:32 | 
  <akosiaris> | 
  etherpad-lite_1.6.0-1 uploaded on apt.wikimedia.org | 
  [production] | 
            
  | 12:05 | 
  <moritzm> | 
  added jenkins 1.651.2 for precise-wikimedia to carbon | 
  [production] | 
            
  | 10:44 | 
  <jynus> | 
  testing dual masters and r/w mode on parsercache nodes | 
  [production] | 
            
  | 10:20 | 
  <elukey> | 
  restarted oozie, hive-* daemons on analytics1003 for java upgrades | 
  [production] | 
            
  | 10:09 | 
  <godog> | 
  run puppet on graphite2001 to split cassandra metrics | 
  [production] | 
            
  | 10:06 | 
  <godog> | 
  reload carbon-c-relay on labmon1001, noop | 
  [production] | 
            
  | 09:51 | 
  <volans> | 
  Apply at runtime thread_pool_stall_limit=10 for all coredb masters (Gerrit 287394, T133333) | 
  [production] | 
            
  | 09:47 | 
  <volans> | 
  Apply at runtime thread_pool_max_threads=2000 for all coredb masters (Gerrit 287394, T133333) | 
  [production] | 
            
  | 08:33 | 
  <ema> | 
  wiping misc caches once again (T134989) | 
  [production] | 
            
  | 07:51 | 
  <ema> | 
  upgrading misc ulsfo to varnish 4.1.2-1wm3 and wiping caches (T134989) | 
  [production] | 
            
  | 07:48 | 
  <jynus> | 
  testing schema change T73563 on db1040 | 
  [production] | 
            
  | 07:47 | 
  <ema> | 
  upgrading misc codfw to 4.1.2-1wm3 and wiping caches (T134989) | 
  [production] | 
            
  | 07:05 | 
  <ema> | 
  running varnish 4.1.2-1wm3 in misc esams (T134989) | 
  [production] | 
            
  | 06:50 | 
  <ema> | 
  repooling cp3007 (ran puppet after varnish upgrade) | 
  [production] | 
            
  | 06:42 | 
  <ema> | 
  depooling cp3007 | 
  [production] | 
            
  | 04:56 | 
  <catrope@tin> | 
  Synchronized php-1.27.0-wmf.23/includes/objectcache/RedisBagOStuff.php: Fix unserialization of negative numbers (duration: 00m 32s) | 
  [production] | 
            
  | 04:37 | 
  <catrope@tin> | 
  Synchronized php-1.28.0-wmf.1/includes/objectcache/RedisBagOStuff.php: Fix unserialization of negative numbers (duration: 00m 31s) | 
  [production] | 
            
  | 03:54 | 
  <catrope@tin> | 
  Synchronized php-1.27.0-wmf.23/extensions/Flow/includes/Notifications/MentionPresentationModel.php: Fix Flow fatal (duration: 00m 26s) | 
  [production] | 
            
  | 03:05 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Thu May 12 03:05:39 UTC 2016 (duration 9m 30s) | 
  [production] | 
            
  | 02:56 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.28.0-wmf.1) (duration: 07m 23s) | 
  [production] | 
            
  | 02:30 | 
  <mwdeploy@tin> | 
  scap sync-l10n completed (1.27.0-wmf.23) (duration: 09m 28s) | 
  [production] | 
            
  | 00:05 | 
  <maxsem@tin> | 
  Synchronized php-1.27.0-wmf.23/extensions/ZeroBanner: https://gerrit.wikimedia.org/r/#/c/288236/ (duration: 00m 25s) | 
  [production] | 
            
  | 00:03 | 
  <maxsem@tin> | 
  Synchronized php-1.28.0-wmf.1/extensions/ZeroBanner: https://gerrit.wikimedia.org/r/#/c/288236/ (duration: 00m 26s) | 
  [production] |