| 2015-10-23
      
      § | 
    
  | 23:31 | <Krinkle> | mwscript deleteEqualMessages.php --wiki hewikisource | [production] | 
            
  | 23:31 | <Krinkle> | mwscript deleteEqualMessages.php --wiki ukwiki | [production] | 
            
  | 23:09 | <hoo> | Started a local rename for Stefan4 to Stefan2 on commons, per request (and attributed to) DerHexer | [production] | 
            
  | 22:23 | <robh> | powercycled nobelium | [production] | 
            
  | 21:24 | <mutante> | nobelium - powercycled, no console output | [production] | 
            
  | 20:04 | <jynus> | rename user job had created lag on almost all enwiki dbs, things should be better now | [production] | 
            
  | 19:08 | <urandom> | starting nodetool cleanup on restbase-test2001-b | [production] | 
            
  | 19:07 | <urandom> | starting nodetool cleanup on restbase-test2001-a | [production] | 
            
  | 18:16 | <godog> | bounce grafana-server on krypton | [production] | 
            
  | 17:58 | <YuviPanda> | moved mounts around on nobelium, mounted bigger disk on /var/lib/elasticsearch | [production] | 
            
  | 17:57 | <godog> | delete wikimedia-grid grafana dashboard (saved a copy first) heavy graphite queries | [production] | 
            
  | 17:53 | <YuviPanda> | stopping elasticsearch on nobelium for https://phabricator.wikimedia.org/T114856 | [production] | 
            
  | 17:13 | <akosiaris> | enable_notifications=0 in neon's icinga for a few mins while the storm dies down | [production] | 
            
  | 16:47 | <godog> | stop puppet on graphite1001 and graphite-index cron, suspected root cause | [production] | 
            
  | 16:26 | <godog> | uwsgi timing out while serving requests, bounce also carbon daemons | [production] | 
            
  | 16:20 | <godog> | bounce uwsgi for graphite-web on graphite1001 | [production] | 
            
  | 16:14 | <urandom> | starting rebuild of restbase-test2002-b | [production] | 
            
  | 15:42 | <jynus> | copied wmf-mariadb10 (10.0.16-2) .deb from trusty to jessie on apt.wikimedia.org | [production] | 
            
  | 15:42 | <urandom> | bouncing restbase-test2002-a Just To See | [production] | 
            
  | 15:06 | <urandom> | running nodetool cleanup on restbase-test2003 | [production] | 
            
  | 14:47 | <godog> | remove outdated cassandra metrics from graphite2001 | [production] | 
            
  | 14:27 | <godog> | remove  restbase-test2001 restbase-test2002 cassandra metrics | [production] | 
            
  | 14:12 | <mobrovac> | restbase rolling-restarting after applying aab840f | [production] | 
            
  | 13:55 | <jynus> | Rebooting and installing jessie on db2055-db2059 | [production] | 
            
  | 13:45 | <godog> | remove 98percentile 999percentile meanRate stddev cassandra metrics after https://gerrit.wikimedia.org/r/#/c/248313/ | [production] | 
            
  | 13:33 | <godog> | remove 15MinuteRate cassandra metrics after https://gerrit.wikimedia.org/r/#/c/248313/ | [production] | 
            
  | 11:51 | <godog> | roll-restart cassandra-metrics-collector on restbase cluster after  https://gerrit.wikimedia.org/r/248313 | [production] | 
            
  | 11:14 | <jynus> | After days of no references on the logs to the renamed user_daily_contribs, deleting delete_user_daily_contribs table on all wikis | [production] | 
            
  | 10:27 | <jynus> | retrying schema change on EventLogging (db1046) after failure | [production] | 
            
  | 10:21 | <jynus> | End of online schema change to geo_tags; all wikis on dblist have been updated | [production] | 
            
  | 09:34 | <godog> | reimage restbase-test2002.codfw.wmnet | [production] | 
            
  | 09:23 | <jynus> | restarting schema change on geo_tags for all wikis | [production] | 
            
  | 09:17 | <mobrovac> | restbase rolling-restart after config changes | [production] | 
            
  | 06:55 | <ori@tin> | Synchronized php-1.27.0-wmf.3/extensions/AbuseFilter/AbuseFilterTokenizer.php: I65d4c6064: Track tokenizer cache hits / misses (duration: 00m 18s) | [production] | 
            
  | 06:55 | <ori@tin> | Synchronized php-1.27.0-wmf.2/extensions/AbuseFilter/AbuseFilterTokenizer.php: I65d4c6064: Track tokenizer cache hits / misses (duration: 00m 17s) | [production] | 
            
  | 05:53 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Fri Oct 23 05:53:43 UTC 2015 (duration 53m 42s) | [production] | 
            
  | 03:15 | <aaron@tin> | Synchronized php-1.27.0-wmf.3/includes/profiler/TransactionProfiler.php: 5ef4a91480ea (duration: 00m 18s) | [production] | 
            
  | 02:41 | <l10nupdate@tin> | LocalisationUpdate completed (1.27.0-wmf.3) at 2015-10-23 02:41:25+00:00 | [production] | 
            
  | 02:36 | <l10nupdate@tin> | Synchronized php-1.27.0-wmf.3/cache/l10n: l10nupdate for 1.27.0-wmf.3 (duration: 08m 04s) | [production] | 
            
  | 00:10 | <krenair@tin> | Synchronized w/static/images/project-logos/anwiki.png: https://gerrit.wikimedia.org/r/#/c/247253/ (duration: 00m 16s) | [production] | 
            
  
    | 2015-10-22
      
      § | 
    
  | 23:58 | <krenair@tin> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/247850/ (duration: 00m 17s) | [production] | 
            
  | 23:49 | <krenair@tin> | Synchronized php-1.27.0-wmf.3/extensions/VisualEditor/modules/ve-mw/ui/dialogs/ve.ui.MWMediaDialog.js: https://gerrit.wikimedia.org/r/#/c/248273/ (duration: 00m 18s) | [production] | 
            
  | 23:13 | <krenair@tin> | Synchronized php-1.27.0-wmf.3/extensions/Flow/container.php: https://gerrit.wikimedia.org/r/#/c/248229/ (duration: 00m 17s) | [production] | 
            
  | 23:13 | <krenair@tin> | Synchronized php-1.27.0-wmf.3/extensions/Flow/autoload.php: https://gerrit.wikimedia.org/r/#/c/248229/ (duration: 00m 17s) | [production] | 
            
  | 23:12 | <krenair@tin> | Synchronized php-1.27.0-wmf.3/extensions/Flow/includes/SpamFilter/RateLimits.php: https://gerrit.wikimedia.org/r/#/c/248229/ (duration: 00m 17s) | [production] | 
            
  | 22:55 | <ori@tin> | Synchronized php-1.27.0-wmf.3/extensions/Cite: extensions/Cite update which fell off the SWAT train yesterday (duration: 00m 19s) | [production] | 
            
  | 22:02 | <ori@tin> | Synchronized php-1.27.0-wmf.3/extensions/WikimediaMaintenance/getJobQueueLengths.php: Ie95ec067da9: getJobQueueLengths: add '--report' option for StatsD reporting (duration: 00m 18s) | [production] | 
            
  | 22:02 | <ori@tin> | Synchronized php-1.27.0-wmf.2/extensions/WikimediaMaintenance/getJobQueueLengths.php: Ie95ec067da9: getJobQueueLengths: add '--report' option for StatsD reporting (duration: 00m 18s) | [production] | 
            
  | 21:11 | <reedy@tin> | Synchronized docroot and w: Add more dblist symlinks (duration: 00m 18s) | [production] | 
            
  | 21:10 | <aaron@tin> | Synchronized php-1.27.0-wmf.3/includes/changetags/ChangeTags.php: e7126ed331109 (duration: 00m 17s) | [production] |