|
2011-08-02
§
|
| 10:13 |
<mark> |
Split LVS class high-traffic into high-traffic1 (text, bits) and high-traffic2 (upload), and made a separate class for https with BGP disabled |
[production] |
| 09:29 |
<Ryan_Lane> |
powercycling srv281 |
[production] |
| 08:14 |
<mark> |
Increased TTL of bits.pmtpa.wikimedia.org from 60 to 1H |
[production] |
| 08:09 |
<mark> |
Changed IP address of upload.pmtpa.wikimedia.org from .3 to 208.80.152.211 |
[production] |
| 08:03 |
<mark> |
Increased etherpad memory from 1G to 2G in /usr/local/etherpad/bin/run.sh |
[production] |
| 07:39 |
<tstarling> |
synchronized wmf-config/CommonSettings.php 'in parser cache multiwrite configuration, putting memcached first instead of mysql' |
[production] |
| 05:51 |
<mark> |
Restarted nginx on yvon and gurvin; apparently nginx needs to be restarted after ip addresses changes, as it binds each individual address instead of IN_ADDR_ANY |
[production] |
| 04:40 |
<notpeter> |
deleted the yay for fixing the nagios boxxie... |
[production] |
| 04:23 |
<notpeter> |
stopping puppet on spence temporarily |
[production] |
| 02:21 |
<LocalisationUpdate> |
completed at Tue Aug 2 02:23:56 UTC 2011 |
[production] |
| 00:26 |
<AaronSchulz> |
deployed [[rev:93695|r93695]] to fix WURFL infinite loop |
[production] |
| 00:24 |
<aaron> |
synchronized php/extensions/MobileFrontend/library/WURFL/Storage/Mwmemcache.php |
[production] |
| 00:23 |
<aaron> |
synchronized php/extensions/MobileFrontend/library/WURFL/CustomDeviceRepository.php |
[production] |
|
2011-08-01
§
|
| 23:41 |
<^demon> |
deleted another one of Roan's silly files from /h/w/c/p/ that contain less --help |
[production] |
| 23:38 |
<notpeter> |
upgrading squid and squid-frontend on sq31. sq32 is down and still needs an upgrade. |
[production] |
| 23:25 |
<aaron> |
synchronized php/extensions/MobileFrontend/library/WURFL/CustomDeviceRepository.php 'pre-OOM logging' |
[production] |
| 23:19 |
<notpeter> |
upgrading squid and squid-frontend on sq33 and sq34 |
[production] |
| 23:15 |
<notpeter> |
restarting broken torrus again again. |
[production] |
| 22:58 |
<aaron> |
synchronized live-1.5/robots.php 'commented out redundant code' |
[production] |
| 22:57 |
<notpeter> |
upgrading squid and squid-frontend on sq35 and sq36 |
[production] |
| 22:49 |
<aaron> |
synchronized multiversion/MWVersion.php 'fix for robots.php hits on bogus wikis' |
[production] |
| 22:47 |
<aaron> |
synchronized multiversion/MWVersion.php |
[production] |
| 22:26 |
<notpeter> |
upgrading squid and squid-frontend on sq37 and sq38 |
[production] |
| 22:09 |
<notpeter> |
upgrading squid and squid-frontend on sq39 and sq40 |
[production] |
| 21:51 |
<notpeter> |
restarting broken torrus again |
[production] |
| 21:18 |
<Reedy> |
srv187 although up and alive, isn't reporting any metrics to ganglia, and uptime is completely wrong |
[production] |
| 21:15 |
<notpeter> |
upgrading squid and squid-frontend on sq59 and sq60 |
[production] |
| 21:12 |
<aaron> |
synchronized multiversion/MWVersion.php |
[production] |
| 21:10 |
<aaron> |
synchronized wmf-config 'multiversion/MWVersion.php missing.php fix' |
[production] |
| 20:46 |
<aaron> |
synchronized wmf-config/CommonSettings.php 'moved two lines of code down a bit to allow usage of missing.php' |
[production] |
| 20:33 |
<aaron> |
synchronized multiversion/MWMultiVersion.php 'removed transition hack (cdb rebuilt)' |
[production] |
| 20:29 |
<AaronSchulz> |
wikiversions.cdb rebuilt |
[production] |
| 20:21 |
<aaron> |
synchronizing Wikimedia installation... Revision: 93132: |
[production] |
| 20:17 |
<aaron> |
synchronized multiversion/MWMultiVersion.php 'hardcode to 1.17 for safe transition' |
[production] |
| 20:09 |
<aaron> |
synchronized live-1.5/MWVersion.php |
[production] |
| 19:59 |
<notpeter> |
upgrading squid and squid-frontend on sq61 and sq62 |
[production] |
| 19:37 |
<notpeter> |
upgrading squid and squid-frontend on sq64 |
[production] |
| 16:15 |
<mark> |
Deployed puppet generated pybal.conf on amslvs1,2,3,4 |
[production] |
| 15:47 |
<mutante> |
amssq55: power cycle, fsck forces add. reboot, clean squid cache, dist-upgrade,.. |
[production] |
| 15:43 |
<mark> |
Deployed PyBal generated pybal.conf on amslvs3 |
[production] |
| 15:25 |
<mutante> |
amssq53: cleaned squid cache, now dist-upgrade and another reboot |
[production] |
| 15:07 |
<mark> |
Temporarily filtering traffic from 10.0.0.0/16 to puppet on sockpuppet |
[production] |
| 14:56 |
<mutante> |
power cycle amssq53 |
[production] |
| 12:36 |
<mutante> |
restarted ntpd on ms3 to fix a Nagios CRIT |
[production] |
| 12:14 |
<mark> |
Changed IP address of bits.pmtpa.wikimedia.org from .118 to 208.80.152.210 |
[production] |
| 11:03 |
<mark> |
Moved LVS services configuration out of site.pp into lvs.pp, in puppet |
[production] |
| 10:03 |
<mark> |
Started db20 |
[production] |