|
2010-08-03
§
|
| 21:20 |
<RobH> |
grosley exim config was overwritten when converted to puppet control (default install uses simple exim setup now). defined host specifically, adding in ganglia details and removing exim control from puppet |
[production] |
| 20:21 |
<catrope> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'bug 24652: Archive namespace for officewiki' |
[production] |
| 20:10 |
<catrope> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Bump style version appendix' |
[production] |
| 20:09 |
<catrope> |
synchronized php-1.5/extensions/UsabilityInitiative/WikiEditor/WikiEditor.combined.min.js 'r70409' |
[production] |
| 16:00 |
<catrope> |
synchronized php-1.5/languages/messages/MessagesEo.php 'r70387' |
[production] |
| 14:24 |
<RobH> |
knsq7 coming down for reinstall |
[production] |
| 14:24 |
<RobH> |
kndq7 coming down for reinstall |
[production] |
| 04:57 |
<Andrew> |
scap |
[production] |
| 04:57 |
<andrew> |
synchronizing Wikimedia installation... Revision: 70064 |
[production] |
| 04:55 |
<Andrew> |
Preparing to update LiquidThreads alpha to trunk with r70106 and r70100 unmerged. |
[production] |
| 01:30 |
<mark> |
Restored exim config on williams (OTRS) |
[production] |
| 00:35 |
<RobH> |
otrs emails still bouncing, working on it. |
[production] |
| 00:31 |
<RobH> |
exim was sitting idle? on williams for otrs delivery after complaining of a failed database connection. restarted exim, it appears to be working on the delivery backlog now, will check back on it in 30 minutes or so |
[production] |
|
2010-08-02
§
|
| 23:55 |
<guillom> |
afaict, all emails sent to OTRS are being rejected with the "retry time not reached for any host after a long failure period" message. The issue seems to have started a few hours ago. |
[production] |
| 22:34 |
<tomaszf> |
upgrading civicrm to 3.1.6 |
[production] |
| 18:20 |
<RobH> |
pulling knsq6 for reinstallation and such |
[production] |
| 17:47 |
<RobH> |
sq33-sq40 reinstalled, online, serving requests |
[production] |
| 15:54 |
<RobH> |
reinstalling sq34-sq40 |
[production] |
| 14:55 |
<mark> |
Installed gmetad on streber for collecting ganglia information in local RRDs |
[production] |
| 14:34 |
<mark> |
Upgraded streber to Ubuntu 10.04 |
[production] |
| 14:24 |
<RobH> |
restarted pdns on linne |
[production] |
| 14:20 |
<RobH> |
updated dns for tesla host ci |
[production] |
| 12:46 |
<mark> |
Fixed ganglia package mess on search12-20 |
[production] |
| 12:10 |
<mark> |
Depooled all text squids from the bits.esams LVS pool |
[production] |
| 12:02 |
<mark> |
Reinstalled knsq2 and knsq5 with Ubuntu 10.04, set them up as Varnish bits caches, and pooled them in LVS |
[production] |
| 11:50 |
<mark> |
Fixed ganglia mess on kaulen |
[production] |
| 11:43 |
<mark> |
Added missing Wikimedia APT repository to kaulen. Why was it not there? Was this host installed in some nonstandard way? |
[production] |
| 11:35 |
<mark> |
Set up exim::simple-mail-sender classes for kaulen |
[production] |
| 11:35 |
<mark> |
Fixed puppet for snapshot*, base classes were not included |
[production] |
| 11:19 |
<mark> |
Turned off knsq3; broken HBA and out of warranty |
[production] |
| 10:55 |
<mark> |
Started puppetd on spence inside gdb |
[production] |
|
2010-08-01
§
|
| 17:21 |
<mark> |
Reinstalled linne.wikimedia.org with Lucid |
[production] |
| 15:41 |
<mark> |
Fixed the haproxy for puppetmaster on brewster, was broken by the upgrade |
[production] |
| 15:19 |
<mark> |
Upgraded brewster to Ubuntu 10.04 Lucid |
[production] |
| 15:18 |
<mark> |
Removed Wikimedia repository default pinning on brewster, as it's doing more harm than good |
[production] |
| 15:07 |
<mark> |
Explicitly install and deinstall gmond / ganglia-monitor packages in puppet, depending on the ubuntu version |
[production] |
| 14:53 |
<mark> |
Removed broken amanda backup client from brewster. Broken package install/dpkg state, no logs, no documentation |
[production] |