|
2013-03-04
§
|
| 21:50 |
<LeslieCarr> |
messing around with the network of virt2 - expect possible pmtpa labs network issues |
[production] |
| 21:50 |
<ottomata> |
staring puppet back up on oxygen, packet loss is better after disabling a filter |
[production] |
| 21:30 |
<cmjohnson1> |
authdns update |
[production] |
| 21:17 |
<logmsgbot_> |
reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki and mediawikiwiki to 1.21wmf11 |
[production] |
| 21:16 |
<logmsgbot_> |
reedy synchronized php-1.21wmf11 |
[production] |
| 21:15 |
<logmsgbot_> |
reedy synchronized php-1.21wmf10 |
[production] |
| 21:14 |
<RobH> |
shutting down professor for a fan swap, should be a short window |
[production] |
| 21:11 |
<cmjohnson1> |
authdns update |
[production] |
| 21:05 |
<ottomata> |
stopping puppet on oxygen to see if we can disable filters and reduce packet loss |
[production] |
| 20:55 |
<Ryan_Lane> |
restarted icinga |
[production] |
| 20:52 |
<logmsgbot_> |
reedy Finished syncing Wikimedia installation... : Rebuild Message Cache for 1.21wmf11 |
[production] |
| 20:52 |
<RobH> |
had to restart morebots, but it wasnt really dead, opps |
[production] |
| 20:51 |
<RobH> |
had to restart icinga |
[production] |
| 20:51 |
<RobH> |
Why arent you taking all the !log |
[production] |
| 20:50 |
<RobH> |
i hate you morebots |
[production] |
| 20:28 |
<logmsgbot_> |
reedy Started syncing Wikimedia installation... : Rebuild Message Cache for 1.21wmf11 |
[production] |
| 19:55 |
<logmsgbot_> |
reedy Started syncing Wikimedia installation... : Build 1.21wmf11 message cache |
[production] |
| 19:41 |
<logmsgbot_> |
reedy synchronized php-1.21wmf11/ |
[production] |
| 19:34 |
<logmsgbot_> |
reedy synchronized wmf-config/ |
[production] |
| 19:34 |
<logmsgbot_> |
reedy synchronized live-1.5/ |
[production] |
| 19:33 |
<logmsgbot_> |
reedy synchronized docroot |
[production] |
| 19:30 |
<hashar> |
on gallium: stopping puppet, cleaning defunct and restarting puppet |
[production] |
| 19:23 |
<logmsgbot_> |
reedy synchronized wmf-config/ |
[production] |
| 19:20 |
<mutante> |
installing package upgrades on sockpuppet |
[production] |
| 19:13 |
<sbernardin> |
taking down amaranth for hw replacement: https://rt.wikimedia.org/Ticket/Display.html?id=4497 |
[production] |
| 18:44 |
<cmjohnson1> |
authdns update |
[production] |
| 18:17 |
<K4-713> |
updated payments cluster to 059969c3d29f685c |
[production] |
| 17:44 |
<cmjohnson1> |
authdns update |
[production] |
| 16:56 |
<logmsgbot_> |
py synchronized wmf-config/db-eqiad.php 'depooling db1009 while investigating error' |
[production] |
| 16:24 |
<sbernardin> |
rebooting db70 to reset idrac: https://rt.wikimedia.org/Ticket/Display.html?id=4628 |
[production] |
| 16:14 |
<sbernardin> |
db27 being taken down for decom per: https://rt.wikimedia.org/Ticket/Display.html?id=4623 |
[production] |
| 10:12 |
<apergos> |
cleared some space on brewster by rmeoving old squid access/storage logs (2.2gb worth in 3 files!), its root partition was full, restarted squid |
[production] |
| 10:12 |
<apergos> |
reinstalled wikimedia-app-taskserver on fenari (how did it get removed?), needed for various cron jobs among other things |
[production] |
| 02:04 |
<logmsgbot_> |
LocalisationUpdate failed (1.21wmf10) at Mon Mar 4 02:04:10 UTC 2013 |
[production] |
|
2013-03-01
§
|
| 23:59 |
<cmjohnson1> |
authdns update |
[production] |
| 23:56 |
<K4-713> |
updated payments cluster to c8442435a9e2f5c |
[production] |
| 23:29 |
<notpeter> |
dropping limesurvey from db9, backup in db9:/a/backup_dumps |
[production] |
| 22:53 |
<RoanKattouw> |
Make that apache2-utils, that's the actual name of the package |
[production] |
| 22:51 |
<RoanKattouw> |
Temporarily installing apachebench on wtp1004 per Gabriel's request |
[production] |
| 22:49 |
<RoanKattouw> |
Depooled wtp1004 so Gabriel can use it for benchmarking |
[production] |
| 22:33 |
<cmjohnson1> |
authdns update |
[production] |
| 22:15 |
<cmjohnson1> |
auth-dns update |
[production] |
| 22:12 |
<mutante> |
DNS update - remove status cname |
[production] |
| 21:56 |
<cmjohnson1> |
taking wtp1001 down to replace w/new box |
[production] |
| 21:32 |
<mwalker> |
pushing 4a3ca6cc36b9d78dbb015c1bf93bbafb14eb0799 to the payments cluster (again, because the instant reviewer, me, was broken) |
[production] |