|
2011-07-05
§
|
| 23:53 |
<^demon> |
scratch that....ssh just seems to have been rather slow in getting its act together. ci.tesla is just fine now |
[production] |
| 23:50 |
<^demon> |
well now I've locked myself out of ci.tesla. Seems it doesn't start ssh on reboot...what a silly thing to do |
[production] |
| 23:47 |
<^demon> |
rebooting ci.tesla since it was horribly hung up on the latest build--was it really stuck for the past 24hrs? |
[production] |
| 23:35 |
<reedy> |
synchronized php-1.17/resources/mediawiki/mediawiki.js 'r91505' |
[production] |
| 21:28 |
<pdhanda> |
ran sync-common-all 'Synced to r91494 for WikiLove' |
[production] |
| 21:10 |
<pdhanda> |
synchronized php-1.17/resources/jquery.ui/themes/vector/jquery.ui.button.css 'Synced to r 91493.' |
[production] |
| 21:00 |
<pdhanda> |
synchronized php-1.17/resources/jquery.ui/themes/vector/jquery.ui.button.css 'Synced to r 91490.' |
[production] |
| 19:53 |
<reedy> |
synchronized php-1.17/includes/api/ApiQuery.php 'r91479' |
[production] |
| 19:07 |
<catrope> |
synchronized php-1.17/extensions/Vector/modules/ext.vector.collapsibleTabs.js 'r91476' |
[production] |
| 19:07 |
<catrope> |
synchronized php-1.17/extensions/Vector/modules/ext.vector.simpleSearch.js 'r91476' |
[production] |
| 16:49 |
<RoanKattouw> |
Short CPU spike on the Apaches, approx 16:45-16:50 UTC. Things seem to be recovering now |
[production] |
| 16:04 |
<mark> |
Put OSPF/OSPFv3 on csw1-sdtpa:e14/1 in active mode again; appears stable so far |
[production] |
| 15:45 |
<mark> |
Reenabled csw1-sdtpa:e14/1 with OSPF/OSPFv3 in passive mode and high metric |
[production] |
| 15:31 |
<mark> |
Shutdown csw1-sdtpa:e14/1 (l3 wave) |
[production] |
| 15:24 |
<mark> |
Disabled IP load-sharing on csw1-esams (out of next-hops), disabled e14/1 (10G wave L3), reenabled it |
[production] |
| 13:38 |
<mark> |
Rebooting streber; dpkg hung |
[production] |
| 08:34 |
<mark> |
Rerouted traffic AS14907 -> AS43821 via 2828 |
[production] |
| 07:11 |
<Tim> |
creating 256 tables on db40 for DBOC |
[production] |
| 07:08 |
<tstarling> |
synchronized php-1.17/wmf-config/CommonSettings.php 'config change for the next DBOC trial, unused at present' |
[production] |
| 07:04 |
<tstarling> |
synchronized php-1.17/includes/objectcache/SqlBagOStuff.php 'r91442: MySQL parser cache fix, ready for a new trial' |
[production] |
| 06:54 |
<root> |
synchronizing Wikimedia installation... Revision: 91441: |
[production] |
| 06:53 |
<tstarling> |
synchronized php-1.17/includes/AutoLoader.php |
[production] |
| 06:53 |
<tstarling> |
synchronized php-1.17/includes/libs/IEUrlExtension.php |
[production] |
| 06:48 |
<Tim> |
doing svn up/scap to get r91440 |
[production] |
|
2011-07-01
§
|
| 20:18 |
<hashar:> |
ci.tesla: installed exiv2 (required for some Api tests) |
[production] |
| 20:13 |
<andrewS> |
mw1-mw22 racked and drac setup |
[production] |
| 19:49 |
<RobH> |
typo in my number sequence update for the new 74 mw servers, updating dns to fix |
[production] |
| 19:09 |
<Ryan_Lane> |
Changing text backend for https in esams to textsvc IP instead of text.esams.wikimedia.org IP |
[production] |
| 18:49 |
<Ryan_Lane> |
restarting pybal on amslvs1 and amslvs3 |
[production] |
| 18:34 |
<midom> |
synchronized php-1.17/wmf-config/InitialiseSettings.php 'removing mysql from pcache, need to repartition, current scheme too costly on memory' |
[production] |
| 18:25 |
<Ryan_Lane> |
adding textsvc.esams IP to amssq and knsq, and amslvs servers |
[production] |
| 18:23 |
<hashar:> |
ci.tesla install wbritish-small package to get a word list in /usr/share/dict |
[production] |
| 18:08 |
<Ryan_Lane> |
restarting pybal on lvs2 to add textsvc service |
[production] |
| 17:40 |
<andrewS> |
Rob Halsell is a Wizard. Period. |
[production] |
| 17:39 |
<midom> |
synchronized php-1.17/wmf-config/InitialiseSettings.php |
[production] |
| 17:29 |
<midom> |
synchronized php-1.17/wmf-config/InitialiseSettings.php 'adding enwiki to the mix' |
[production] |
| 17:27 |
<midom> |
synchronized php-1.17/wmf-config/InitialiseSettings.php 'sending a bit lower cache load to db40, though I guess that was initial shock before' |
[production] |
| 17:25 |
<RobH> |
Aware that #wikipedia is down and fixed, in less than a minute, woot |
[production] |
| 17:25 |
<midom> |
synchronized php-1.17/wmf-config/InitialiseSettings.php 'oops, should have started with lower load, opentable cache..' |
[production] |
| 17:23 |
<midom> |
synchronized php-1.17/wmf-config/InitialiseSettings.php 'sending cache load again to db40' |
[production] |
| 17:16 |
<Ryan_Lane> |
adding text.svc address to pmtpa text squids and lvs2 |
[production] |
| 17:08 |
<Ryan_Lane> |
adding text.svc private address to esams and pmtpa DNS |
[production] |
| 14:56 |
<RobH> |
pdns crash after rehup on ns1.wikimedia.org, restarting pdns |
[production] |
| 14:49 |
<RobH> |
dns updated to remove the mgmt ip addresses for srv302+ and replace them with mw1-mw74 |
[production] |
| 14:33 |
<RobH> |
updating dns, removing entries for non-existent srv302+ and replacing them with mw1-mw74 |
[production] |
| 11:18 |
<andrewS> |
motion sensors are being installed today in sdtpa that are going to trip the lights when there is movement. not sure how this will affect cameras. will update once their work is done later today. |
[production] |
| 07:33 |
<andrewS> |
the 3 new racks in pmtpa have power plugs running to them now. |
[production] |