|
2012-07-17
§
|
| 14:52 |
<mark> |
Reinstalling lvs4 with Ubuntu Precise |
[production] |
| 14:48 |
<mark> |
Stopped PyBal on lvs4, failing over traffic to lvs3 |
[production] |
| 14:47 |
<mark> |
Reinstalling lvs1002 with Ubuntu Precise |
[production] |
| 14:46 |
<mark> |
Stopped PyBal on lvs1002, failing over traffic to lvs1005 |
[production] |
| 14:45 |
<hashar> |
Blacklisted user agents matchin /.*Googlebot.*/ |
[releng] |
| 14:44 |
<mark> |
lvs1001 is back up and serving traffic |
[production] |
| 14:11 |
<mark> |
Stopped PyBal on lvs1001, failing over traffic to lvs1004 |
[production] |
| 13:45 |
<hashar> |
Manually restarted apaches |
[releng] |
| 13:44 |
<hashar> |
Imported all.conf apache conf from production |
[releng] |
| 13:29 |
<hashar> |
err: /Stage[main]/Mediawiki::Sync/Exec[mw-sync]: Failed to call refresh: Command exceeded timeout at /etc/puppet/manifests/mediawiki.pp:24 |
[releng] |
| 13:28 |
<hashar> |
All apaches are dead :/ |
[releng] |
| 13:09 |
<maplebed> |
changed auth URL for swift to use load balancer rather than round robin DNS |
[production] |
| 13:08 |
<mark> |
lvs1003 is back up and serving traffic |
[production] |
| 13:07 |
<maplebed> |
changed order on ganglia swift view graphs to group by metric rather than host |
[production] |
| 13:02 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 12:53 |
<mark> |
Fixed boot order on lvs1003 |
[production] |
| 12:34 |
<mark> |
Reinstalling lvs1003 with Ubuntu Precise |
[production] |
| 12:28 |
<mark> |
Stopped PyBal on lvs1003, failing over traffic to lvs1006 |
[production] |
| 12:10 |
<mark> |
amslvs1 is back up and serving traffic |
[production] |
| 11:44 |
<mark> |
Reinstalling amslvs1 with Ubuntu Precise |
[production] |
| 11:40 |
<mark> |
Stopped PyBal on amslvs1, failing over traffic to amslvs3 |
[production] |
| 11:32 |
<mark> |
amslvs2 is back up and serving traffic |
[production] |
| 10:47 |
<mark> |
Reinstalling amslvs2 with Ubuntu Precise |
[production] |
| 10:36 |
<mark> |
Stopped PyBal on amslvs2, failing over traffic to amslvs4 |
[production] |
| 09:26 |
<hashar> |
Adding class role::applicationserver::jobrunner |
[releng] |
| 09:20 |
<hashar> |
sync upload6 dirs again. root@deployment-nfs-memc:$ rsync -a --progress --inplace /mnt/export/upload6 /data/project/upload6 |
[releng] |
| 09:17 |
<hashar> |
deployment-prep sync upload6 dirs again. root@deployment-nfs-memc:$ rsync -a --progress --inplace /mnt/export/upload6 /data/project/upload6 |
[production] |
| 07:30 |
<Tim> |
testing envvars/apache2.conf change on srv193 |
[production] |
| 07:09 |
<Tim> |
restarted apache on srv193 |
[production] |
| 02:58 |
<Tim> |
graceful restart of all apaches |
[production] |
| 02:52 |
<reedy> |
synchronized wmf-config/CommonSettings.php |
[production] |
| 02:47 |
<LocalisationUpdate> |
completed (1.20wmf6) at Tue Jul 17 02:47:06 UTC 2012 |
[production] |
| 02:23 |
<LocalisationUpdate> |
completed (1.20wmf7) at Tue Jul 17 02:23:41 UTC 2012 |
[production] |
| 00:44 |
<Tim> |
deploying new redirects.conf |
[production] |
| 00:28 |
<Tim> |
testing new redirects.conf on mw1 |
[production] |