| 2016-09-13
      
      § | 
    
  | 23:32 | <catrope@tin> | Synchronized php-1.28.0-wmf.18/resources/lib/moment/locale: T145382 (duration: 00m 47s) | [production] | 
            
  | 23:19 | <catrope@tin> | Synchronized php-1.28.0-wmf.19/resources/lib/moment/locale: T145382 (duration: 00m 49s) | [production] | 
            
  | 22:44 | <awight> | update paymentswiki config to da01ae9045af7b8b5f3d34a5f81419fde3bde1d5 | [production] | 
            
  | 22:13 | <demon@tin> | Synchronized php-1.28.0-wmf.19/includes/MediaWiki.php: Avoid stupid warnings on url parsing (duration: 00m 47s) | [production] | 
            
  | 21:45 | <demon@tin> | Synchronized php-1.28.0-wmf.19/includes/DefaultSettings.php: for lego <3 (duration: 00m 47s) | [production] | 
            
  | 20:46 | <demon@tin> | Synchronized php-1.28.0-wmf.19/extensions/Echo: For Roan <3 (duration: 00m 54s) | [production] | 
            
  | 20:23 | <Krenair> | restarted designate-api on labtestservices2001, now designate in labtest is working again | [production] | 
            
  | 20:21 | <Krenair> | restart rabbitmq-server on labtestcontrol2001 | [production] | 
            
  | 19:01 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.19 | [production] | 
            
  | 18:35 | <gehel> | starting data import on wdqs200? | [production] | 
            
  | 18:24 | <demon@tin> | Finished scap: testwiki to wmf.19 + l10n bootstrap (try 2) (duration: 47m 21s) | [production] | 
            
  | 18:08 | <gehel> | moving to scap deployed configuration for wdqs - T144380 | [production] | 
            
  | 17:58 | <mutante> | wtp2019 - powercycled, back up without the error, services started | [production] | 
            
  | 17:55 | <mutante> | wtp2019 Uncorrectable Memory Error | [production] | 
            
  | 17:55 | <mutante> | wtp2019 - down sinc a couple days. console says "Alert!  System fatal error during previous boot" | [production] | 
            
  | 17:53 | <volans> | reimaging mw2198 that failed early today | [production] | 
            
  | 17:36 | <demon@tin> | Started scap: testwiki to wmf.19 + l10n bootstrap (try 2) | [production] | 
            
  | 17:24 | <demon@tin> | scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_4282891950" --threads=4 --lang en  --quiet' returned non-zero exit status 1 (duration: 09m 23s) | [production] | 
            
  | 17:15 | <demon@tin> | Started scap: testwiki to wmf.19 + l10n bootstrap | [production] | 
            
  | 16:59 | <jynus> | beta dbs back in rw mode | [production] | 
            
  | 16:54 | <mobrovac> | restbase deploy end of d10d759 | [production] | 
            
  | 16:53 | <demon@tin> | Synchronized multiversion/updateWikiversions: unbreak myself (duration: 00m 48s) | [production] | 
            
  | 16:41 | <mobrovac> | restbase deploy start of d10d759 | [production] | 
            
  | 16:25 | <mobrovac> | change-prop deploying d701a69 | [production] | 
            
  | 16:24 | <godog> | dump hhvm backtrace on mw1162 and restart hhvm, apache gets connection refused | [production] | 
            
  | 16:17 | <urandom> | T144826: Removing compaction rate limit, increasing compactor threads (from 10 to 20), and beginning scrub of local_group_wikipedia_T_parsoid_html.data (restbase2004-b.codfw.wmnet) | [production] | 
            
  | 16:06 | <jynus> | power resetting db1082 | [production] | 
            
  | 16:02 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1082 (duration: 01m 00s) | [production] | 
            
  | 15:46 | <gehel> | test_* indices remvoed on relforge cluster, cluster restarted | [production] | 
            
  | 15:44 | <jynus> | setting deployment-db1 and deployment-db1 mysqls in read only mode | [production] | 
            
  | 15:40 | <gehel> | shutting down relforge cluster for indices cleanup | [production] | 
            
  | 02:46 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Tue Sep 13 02:46:54 UTC 2016 (duration 7m 12s) | [production] | 
            
  | 02:39 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.18) (duration: 17m 41s) | [production] | 
            
  | 01:58 | <aaron@tin> | Synchronized wmf-config/CommonSettings.php: Lower $wgMaxUserDBWriteDuration to 3 (duration: 00m 47s) | [production] | 
            
  | 01:27 | <aaron@tin> | Synchronized php-1.28.0-wmf.18/extensions/GlobalUsage: 1843a856c7c648a310a1983d53139e5b79ea6585 (duration: 00m 48s) | [production] | 
            
  | 01:24 | <bblack> | cache_upload: reverting codfw to file storage | [production] | 
            
  | 01:15 | <aaron@tin> | Synchronized php-1.28.0-wmf.18/extensions/GeoData: 2dedca3789266b53cbde5bd88913c01175926974 (duration: 00m 49s) | [production] | 
            
  | 01:14 | <aaron@tin> | Synchronized php-1.28.0-wmf.18/includes/jobqueue/utils/PurgeJobUtils.php: (no message) (duration: 00m 52s) | [production] | 
            
  | 00:50 | <bblack> | cache_upload: reverting eqiad to file storage | [production] | 
            
  | 00:50 | <bblack> | reverting eqiad to file storage | [production] | 
            
  | 00:17 | <aaron@tin> | Synchronized php-1.28.0-wmf.18/extensions/GeoData: 2dedca3789266b53cbde5bd88913c01175926974 (duration: 00m 48s) | [production] | 
            
  | 00:16 | <aaron@tin> | Synchronized php-1.28.0-wmf.18/includes/jobqueue/utils/PurgeJobUtils.php: 0419831bbae66bda1e4dca8d690b607b493b2f5f (duration: 00m 47s) | [production] |