| 2014-02-13
      
      § | 
    
  | 04:10 | <springle> | synchronized wmf-config/db-eqiad.php  's2 repool db1036 as slave' | [production] | 
            
  | 03:51 | <springle> | synchronized wmf-config/db-eqiad.php  's1 repool db1056 as slave' | [production] | 
            
  | 03:28 | <LocalisationUpdate> | ResourceLoader cache refresh completed at 2014-02-13 03:28:37+00:00 | [production] | 
            
  | 02:57 | <Guest92179> | DNS update - Bugzilla TTL back to 1H, migration over | [production] | 
            
  | 02:47 | <LocalisationUpdate> | completed (1.23wmf12) at 2014-02-13 02:46:53+00:00 | [production] | 
            
  | 02:33 | <springle_> | db9 mysqld stopped for decom, db1001 slave stopped | [production] | 
            
  | 02:27 | <LocalisationUpdate> | completed (1.23wmf13) at 2014-02-13 02:27:05+00:00 | [production] | 
            
  | 02:04 | <Krinkle> | Jenkins has been unresponsive to urls that retrieve build results for the past few hours (e.g. https://integration.wikimedia.org/ci/job/mediawiki-core-jslint/11129/console) | [production] | 
            
  | 01:51 | <^d> | gerrit: turned BZ plugin back on since downtime is over. No less than 5 pings to do so ;-) | [production] | 
            
  | 01:36 | <awight> | tools updated from ad90b56ab7e71142a7e98825bad3febe0823920a to 946011f1db995ee73794c6bfa2c40bf8628f960d | [production] | 
            
  | 00:57 | <mutante> | Welcome to Bugzilla 4.4.1 in eqiad, served by puppet module bugzilla on zirconium | [production] | 
            
  | 00:55 | <mutante> | DNS update - switching Bugzilla to zirconium | [production] | 
            
  | 00:50 | <springle> | xtrabackup clone db1018 to db1036 | [production] | 
            
  | 00:19 | <awight> | checkout new phpmailer and drush | [production] | 
            
  | 00:15 | <awight> | checkout twig and DonationInterface into new dirs | [production] | 
            
  
    | 2014-02-12
      
      § | 
    
  | 23:53 | <awight> | updated tools from 15fd20f80c52470033e09220a124e0d43bc42fd4 to ad90b56ab7e71142a7e98825bad3febe0823920a | [production] | 
            
  | 23:46 | <awight> | tools updated from 7c2e15b006fec25fd1881c67f6bce9e406759535 to 15fd20f80c52470033e09220a124e0d43bc42fd4 | [production] | 
            
  | 23:45 | <awight> | tools updated from 7c4114f3db978996a460d154dcd5da2bd0ffb48d to 7c2e15b006fec25fd1881c67f6bce9e406759535 | [production] | 
            
  | 23:29 | <springle> | xtrabackup clone db1049 to db1056 | [production] | 
            
  | 23:27 | <bd808> | mw1185 continues to segfault at a average rate of ~800/hr for the last 48 hours | [production] | 
            
  | 23:06 | <mutante> | checksetup.pl on zirconium doing db upgrades .. | [production] | 
            
  | 22:47 | <^d> | gerrit: disabled bz plugin during bz maintenance, spamming errors since it can't connect | [production] | 
            
  | 22:45 | <rdwrer> | deployed change to jsduck jobs that will cause them to fail more often (but in a good way) | [production] | 
            
  | 22:39 | <mutante> | Bugzilla in scheduled maintenance/upgrade period - be back in a bit | [production] | 
            
  | 22:31 | <catrope> | synchronized php-1.23wmf13/extensions/VisualEditor/  'cherry-picks' | [production] | 
            
  | 22:22 | <catrope> | synchronized wmf-config/CommonSettings.php  'plumbing for as-yet-unused TemplateDataUseGUI setting' | [production] | 
            
  | 22:20 | <catrope> | synchronized wmf-config/InitialiseSettings.php  'secondary tabs and disableforanons for eswiki' | [production] | 
            
  | 22:19 | <catrope> | synchronized visualeditor-default.dblist  'VE no longer default on eswiki' | [production] | 
            
  | 22:17 | <tstarling> | synchronized docroot/noc/conf/highlight.php | [production] | 
            
  | 22:17 | <catrope> | synchronized visualeditor.dblist  'Enable VE on frwikiversity' | [production] | 
            
  | 22:17 | <tstarling> | updated /a/common to {{Gerrit|I24ff4d72a}}: Replace easter egg by a more explaining message | [production] | 
            
  | 22:09 | <mutante> | root@wtp1019:~# service parsoid restart | [production] | 
            
  | 22:07 | <mutante> | restarting parsoids | [production] | 
            
  | 21:15 | <gwicke> | updated parsoid to deploy repo 77f4aaf2 and code repo 96c1274 | [production] | 
            
  | 21:02 | <mutante> | DNS update - switch bz.wp to cluster redirect | [production] | 
            
  | 21:00 | <csteipp> | finished scap: bug 60771 (duration: 36m 58s) | [production] | 
            
  | 20:56 | <mwalker> | updating civicrm from 99463ceb353b452a82a7647baa88b59f65e981ea to 935b6dade2c230558c12538ae61dcb3ef18b3efd to try the queueing fix again | [production] | 
            
  | 20:23 | <csteipp> | started scap: bug 60771 | [production] | 
            
  | 20:13 | <mutante|away> | DNS update - lowering bugzilla TTL | [production] | 
            
  | 19:48 | <manybubbles> | when elastic1007 came back Elasticsearch wasn't able to balance shards to my liking so I'm forcing it to shuffle them some by temporarily lowering the disk thresholds to 75/80 instead of 85/95 | [production] | 
            
  | 18:58 | <yurik> | synchronized php-1.23wmf13/extensions/ZeroRatedMobileAccess/ | [production] | 
            
  | 18:55 | <yurik> | synchronized php-1.23wmf12/extensions/ZeroRatedMobileAccess/ | [production] | 
            
  | 17:45 | <bd808> | Updated scholaships.wm.o to fbffb96; l10n updates | [production] | 
            
  | 17:26 | <ottomata> | purged oracle java 6 from analytics1012 and restarted hadoop daemons | [production] | 
            
  | 17:15 | <demon> | synchronized php-1.23wmf13/extensions/CirrusSearch  'Performance fixes' | [production] | 
            
  | 17:14 | <demon> | synchronized php-1.23wmf12/extensions/CirrusSearch  'Performance fixes' | [production] | 
            
  | 17:14 | <ottomata> | stopping hadoop daemons on an12, oracle java 6 is running there, don't know why yet, apt says it is not installed….grrr | [production] | 
            
  | 16:51 | <awight> | updated crm from 738bce68b56846ee3d493308629beaf743aa8653 to 99463ceb353b452a82a7647baa88b59f65e981ea (revert) | [production] | 
            
  | 14:59 | <manybubbles|away> | just finished in place (Cirrus) reindex of all group1 wikis except commons | [production] | 
            
  | 08:50 | <akosiaris> | restarted parsoid on wtp1004. After https://gerrit.wikimedia.org/r/112640 is was for some reason listening on 56547 and not 8000. The restart fixed it | [production] |