|
2010-11-23
§
|
| 21:58 |
<^demon> |
restarted wmfsvndumper with new dump location |
[production] |
| 21:44 |
<^demon> |
running svndump test in screen on formey |
[production] |
| 21:43 |
<Ryan_Lane> |
installed screen on formey |
[production] |
| 21:42 |
<Ryan_Lane> |
added demon to svnadm |
[production] |
| 21:41 |
<Ryan_Lane> |
changed ownership of svn config files from adm to svnadm |
[production] |
| 21:41 |
<Ryan_Lane> |
added an svnadm group to ldap |
[production] |
| 15:07 |
<mark> |
Finished testing, amslvs1 is primary again |
[production] |
| 14:40 |
<mark> |
Shutdown BGP session to amslvs1 on csw1-esams for testing, traffic failing over to amslvs3 |
[production] |
| 13:50 |
<mark> |
Removed redistribution of BGP LVS routes into OSPF; rely on iBGP exclusively for LVS routes |
[production] |
| 12:06 |
<catrope> |
synchronized php-1.5/includes/api/ApiDelete.php 'r77146' |
[production] |
| 02:27 |
<RobH> |
had to restart morebots, it had left all channels. also had to change admin pass for listserv |
[production] |
|
2010-11-22
§
|
| 22:09 |
<mark> |
Restarted varnish on knsq4 |
[production] |
| 22:05 |
<mark> |
Restarting varnish on knsq1 |
[production] |
| 21:44 |
<Stifle> |
Does this work? |
[production] |
| 21:44 |
<Ryan_Lane> |
err - restarting varnish on knsq2,4,5 |
[production] |
| 21:44 |
<Ryan_Lane> |
restarting varnish on knsq2,3,4 |
[production] |
| 21:43 |
<Ryan_Lane> |
restarting knsq2,3,4 |
[production] |
| 21:29 |
<mark> |
Restarted varnish on knsq2,4,5 |
[production] |
| 21:24 |
<mark> |
Restarting Varnish on knsq1 |
[production] |
| 21:09 |
<mark> |
Set static routes 91.198.174.232/31 to 91.198.174.247 (csw1-esams) on br1-knams |
[production] |
| 21:03 |
<mark> |
Started pybal on amslvs1 again |
[production] |
| 20:53 |
<mark> |
Killed PyBal on amslvs1 |
[production] |
| 20:39 |
<mark> |
Lowered cache_mem and cache_dir sizes on amssq31 for testing |
[production] |
| 20:28 |
<Ryan_Lane> |
depooling knsq24 |
[production] |
| 20:25 |
<mark> |
Restarted pybal on amslvs1 with depool-threshold = 1 (temporarily) |
[production] |
| 17:27 |
<Ryan_Lane> |
restarting nagios |
[production] |
| 17:27 |
<Ryan_Lane> |
adding warning notifications via IRC to nagios |
[production] |
| 16:40 |
<mark> |
Suppressed announcements to AS16265 on csw1-esams |
[production] |
| 16:30 |
<mark> |
Suppressed BGP announcements to AS13030 on br1-knams |
[production] |
|
2010-11-19
§
|
| 23:04 |
<Ryan_Lane> |
changed https check on payments to 1 retry in nagios, via puppet |
[production] |
| 20:59 |
<atglenn> |
restarted torrus. guess why :-P |
[production] |
| 17:52 |
<mark> |
Readded notification of Service[nagios] when changing nagios types in puppet |
[production] |
| 13:59 |
<richcole> |
set raid 10 up on DB41 |
[production] |
| 08:36 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'bug 25850 - Hide Take me Back link on all wikis' |
[production] |
| 07:32 |
<JeLuF> |
started puppet on sq42 sq41 sq47 sq45 sq46 sq44 sq50 sq52 sq53 sq51 sq48 sq55 sq54 sq43 sq56 sq58 sq64 sq62 sq61 sq65 sq66 sq60 sq63 sq72 sq75 sq71 sq73 sq74 sq76 sq78 sq77 sq81 sq80 sq79 sq83 sq84 sq82 sq85 sq86 |
[production] |
| 05:21 |
<apergos> |
on sq85 I was seeing complaints from cron about restart of puppet: unknown option -w. removed that from /etc/default/puppet and restart, but that fails: Could not parse for environment production: Could not find file /agent.pp |
[production] |
| 01:00 |
<atglenn> |
added monitoring mechanism in root's crontab on sq85 (don't need it everywhere) that will sms me when ms4 is acting up. I'd do it in puppet if someone told me how they would want it to be added there. |
[production] |
| 00:22 |
<tomasz> |
turning db9 watchdog back on. setting at 5minutes |
[production] |