|
2010-10-17
§
|
| 22:06 |
<apergos> |
edited /etc/apache2/envvars in place and set APACHE_ULIMIT_MAX_FILES to 2000, hoping that will address the "too many open files" issue |
[production] |
| 21:29 |
<hcatlin> |
mobile3 is having file limit issues. Turned off apache so that is de-pooled. Issue description sent to Ariel for RT submission. |
[production] |
| 18:17 |
<mark> |
Restarted all varnish processes on sq67-70 |
[production] |
| 18:10 |
<mark> |
authdns-scenario normal |
[production] |
| 18:07 |
<apergos> |
stopped and restarted varnish on knsq1, 4... loads back to normal there now |
[production] |
| 18:05 |
<mark> |
Powercycled sq69 |
[production] |
| 18:00 |
<Ryan_Lane> |
restarting varnish on esams bits |
[production] |
| 17:50 |
<JeLuF> |
Setting scenario to esams-down. |
[production] |
| 17:47 |
<JeLuF> |
restarted varnish on knsq1, load average was 5149.87 |
[production] |
| 15:39 |
<jeluf> |
ran sync-common-all 'Bug 25056 - Create new wiki for Persian Wikinews' |
[production] |
| 15:15 |
<jeluf> |
ran sync-common-all 'Bug 25130 - Create Wikipedia in Komi-Permyak' |
[production] |
| 14:42 |
<jeluf> |
ran sync-common-all 'Bug 25307 - Create Wikipedia in Banjar' |
[production] |
| 14:03 |
<jeluf> |
ran sync-common-all 'Bug 25544 - Create a new Wikipedia in Hill Mari' |
[production] |
| 12:59 |
<jeluf> |
synchronized php-1.5/wmf-config/InitialiseSettings.php '25345 - outreachwiki import sources' |
[production] |
|
2010-10-14
§
|
| 20:36 |
<Ryan_Lane> |
also added rkattouw to the engineering alias |
[production] |
| 20:34 |
<Ryan_Lane> |
adding ^demon and myself to the engineering alias |
[production] |
| 20:00 |
<Ryan_Lane> |
adding ldap::client::wmf-cluster include to formey in puppet. Adding openldap client as well. |
[production] |
| 19:59 |
<Ryan_Lane> |
enabling opendj replication on nfs1 and nfs2 |
[production] |
| 18:01 |
<hcatlin> |
mobiles are acting up. keep an eye on them! |
[production] |
| 17:20 |
<mark> |
Fixed mobile by restarting apache on mobile2 |
[production] |
| 17:15 |
<hcatlin> |
LVS is down for mobile... not sure why |
[production] |
| 17:05 |
<hcatlin> |
mobile deploy is complete! |
[production] |
| 15:52 |
<hcatlin> |
deploying updates to mobile cluster |
[production] |
| 13:13 |
<mark> |
Deployed logrotate file for squid-frontend |
[production] |
| 12:34 |
<mark> |
Fixed gmond on sq61 |
[production] |
| 12:31 |
<mark> |
restarted scaling on srv222 |
[production] |
|
2010-10-13
§
|
| 23:20 |
<tfinc> |
synchronized php-1.5/wmf-config/CommonSettings.php 'White listing new Appeal pages' |
[production] |
| 22:55 |
<Ryan_Lane> |
added ldap::client::trust and ldap::client::openldap classes to nfs1 and nfs2, so that they can manage their own ldap servers |
[production] |
| 19:51 |
<Ryan_Lane> |
commented out #monitor_service { "disk space":... line in misc::nfs-server::home::monitoring class in misc-servers.pp; was already defined in base.pp. |
[production] |
| 19:36 |
<Ryan_Lane> |
installing opendj on nfs1 and nfs2 |
[production] |
| 16:20 |
<Ryan_Lane> |
reinstalling opendj and java on sanger. recreating the opendj instance. |
[production] |
| 16:12 |
<Ryan_Lane> |
fixed sanger's ip address in /etc/hosts |
[production] |
| 14:02 |
<mark> |
Killed ircd on browne - hopefully people are connecting to irc.wikimedia.org :P |
[production] |
| 14:01 |
<mark> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Move UDP IRC RC stream from browne to ekrem' |
[production] |
| 10:45 |
<mark> |
Powercycled sq68 |
[production] |
| 10:43 |
<mark> |
synchronized php-1.5/wmf-config/db.php 'Depooling db19, ECC errors' |
[production] |
| 10:42 |
<mark> |
synchronized php-1.5/wmf-config/db.php 'Repooling db15, db18, db19' |
[production] |