|
2012-07-17
ยง
|
| 23:24 |
<Tim> |
on srv193: removed core dump files, disabled core dumping, restarted apache |
[production] |
| 21:26 |
<Platonides> |
installed python-imaging and wamerican on deployment-dbdump |
[releng] |
| 21:12 |
<beta-logmsgbot> |
petrb: updating ArticleFeedbackv5 extension |
[releng] |
| 19:52 |
<mlitn> |
Finished syncing Wikimedia installation... : |
[production] |
| 19:51 |
<hashar> |
369 languages rebuilt out of 369 |
[releng] |
| 19:45 |
<hashar> |
rebuilding l10n cache: mwscript rebuildLocalisationCache.php --wiki=aawiki --threads=2 |
[releng] |
| 19:39 |
<hashar> |
beta broken by PAGEID magic word introduced with 0a7cf03 / I11d42ca7 {{gerrit|9858}} |
[releng] |
| 19:32 |
<hashar> |
running git bisect of core 80fbb70..ef3132f |
[releng] |
| 19:26 |
<hashar> |
upgrading MediaWiki core 80fbb70..ef3132f |
[releng] |
| 19:20 |
<hashar> |
updated AFTv5: f97811f..d3bd97f |
[releng] |
| 19:09 |
<mlitn> |
Started syncing Wikimedia installation... : |
[production] |
| 19:04 |
<hashar> |
updated robots.txt to specify a user-agent. Will definitely prevents Google from killing beta :) |
[releng] |
| 18:54 |
<hashar> |
squid resumed. The swap files got corrupted somehow, needed to delete them entirely to start again. Squud storing again. |
[releng] |
| 18:40 |
<hashar> |
-squid bah doing rm -fR /data/project/squid1/* |
[releng] |
| 18:39 |
<hashar> |
installed `tree` on deployment-squid |
[releng] |
| 18:38 |
<hashar> |
removing swap files in /data/project/squid1 |
[releng] |
| 18:36 |
<hashar> |
Squid is bugged as hell : 2012/07/17 18:36:13| Store rebuilding is -0.1% complete and looping |
[releng] |
| 18:21 |
<beta-logmsgbot> |
hashar: rebooting squid glusterfs gone wild apparently |
[releng] |
| 15:18 |
<mark> |
lvs1002 is back up and idling |
[production] |
| 15:15 |
<mark> |
Fixed serial console redirection after boot to OFF on lvs1002 |
[production] |
| 15:06 |
<mark> |
lvs4 is back up and serving traffic |
[production] |
| 14:52 |
<mark> |
Reinstalling lvs4 with Ubuntu Precise |
[production] |
| 14:48 |
<mark> |
Stopped PyBal on lvs4, failing over traffic to lvs3 |
[production] |
| 14:47 |
<mark> |
Reinstalling lvs1002 with Ubuntu Precise |
[production] |
| 14:46 |
<mark> |
Stopped PyBal on lvs1002, failing over traffic to lvs1005 |
[production] |
| 14:45 |
<hashar> |
Blacklisted user agents matchin /.*Googlebot.*/ |
[releng] |
| 14:44 |
<mark> |
lvs1001 is back up and serving traffic |
[production] |
| 14:11 |
<mark> |
Stopped PyBal on lvs1001, failing over traffic to lvs1004 |
[production] |
| 13:45 |
<hashar> |
Manually restarted apaches |
[releng] |
| 13:44 |
<hashar> |
Imported all.conf apache conf from production |
[releng] |
| 13:29 |
<hashar> |
err: /Stage[main]/Mediawiki::Sync/Exec[mw-sync]: Failed to call refresh: Command exceeded timeout at /etc/puppet/manifests/mediawiki.pp:24 |
[releng] |
| 13:28 |
<hashar> |
All apaches are dead :/ |
[releng] |
| 13:09 |
<maplebed> |
changed auth URL for swift to use load balancer rather than round robin DNS |
[production] |
| 13:08 |
<mark> |
lvs1003 is back up and serving traffic |
[production] |
| 13:07 |
<maplebed> |
changed order on ganglia swift view graphs to group by metric rather than host |
[production] |
| 13:02 |
<reedy> |
synchronized wmf-config/ |
[production] |
| 12:53 |
<mark> |
Fixed boot order on lvs1003 |
[production] |
| 12:34 |
<mark> |
Reinstalling lvs1003 with Ubuntu Precise |
[production] |
| 12:28 |
<mark> |
Stopped PyBal on lvs1003, failing over traffic to lvs1006 |
[production] |
| 12:10 |
<mark> |
amslvs1 is back up and serving traffic |
[production] |
| 11:44 |
<mark> |
Reinstalling amslvs1 with Ubuntu Precise |
[production] |
| 11:40 |
<mark> |
Stopped PyBal on amslvs1, failing over traffic to amslvs3 |
[production] |
| 11:32 |
<mark> |
amslvs2 is back up and serving traffic |
[production] |
| 10:47 |
<mark> |
Reinstalling amslvs2 with Ubuntu Precise |
[production] |
| 10:36 |
<mark> |
Stopped PyBal on amslvs2, failing over traffic to amslvs4 |
[production] |
| 09:26 |
<hashar> |
Adding class role::applicationserver::jobrunner |
[releng] |
| 09:20 |
<hashar> |
sync upload6 dirs again. root@deployment-nfs-memc:$ rsync -a --progress --inplace /mnt/export/upload6 /data/project/upload6 |
[releng] |
| 09:17 |
<hashar> |
deployment-prep sync upload6 dirs again. root@deployment-nfs-memc:$ rsync -a --progress --inplace /mnt/export/upload6 /data/project/upload6 |
[production] |
| 07:30 |
<Tim> |
testing envvars/apache2.conf change on srv193 |
[production] |
| 07:09 |
<Tim> |
restarted apache on srv193 |
[production] |