2012-07-27
§
|
22:50 |
<Platonides> |
Filesystem corruption signs in deployment-bastion, most debconf backends /usr/share/perl5/Debconf/FrontEnd are zeroed files. This explains some of the earlier apt-get problems, and maybe also bug 38747 (aka. magic tail -f fix) |
[releng] |
22:49 |
<Damianz> |
Thumbnails are currently broken - thumbs folder seems empty, images are there as expected - beta quirk or need rebuilding? |
[releng] |
22:22 |
<Damianz> |
DocumentRoot got wiped, broken all the sites - fixed the broken symlink on bastion for /data/project/apache to /usr/local/apache. Also fixed common-local to common symlink. |
[releng] |
21:56 |
<Platonides> |
apt-get removed wikimedia-task-appserver in deployment-bastion :( |
[releng] |
21:42 |
<Platonides> |
removed generated captcha files |
[releng] |
21:33 |
<Platonides> |
installed in deployment-bastion the packages joe, python-imaging and wamerican |
[releng] |
20:29 |
<labs-logs-bottie> |
j: deployment-bastion: removing all deployment-nfs-memc entries from /etc/fstab |
[releng] |
13:14 |
<hashar> |
restarted job runner on jobrunner06 |
[releng] |
13:11 |
<hashar> |
removing all deployment-nfs-memc entries from /etc/fstab |
[releng] |
09:51 |
<hashar> |
{{bug|38749}} jobrunner06 : removed /usr/local/apache and /mnt/upload6 empty dir. Downgraded PHP manually. Rerunning puppet. |
[releng] |
08:46 |
<hashar> |
migrating jobrunner06 to use the /data/project for uploads |
[releng] |
2012-07-26
§
|
19:27 |
<labs-logs-bottie> |
j: delete broken deployment-video01 instance |
[releng] |
18:42 |
<labs-logs-bottie> |
j: new instance deployment-video02 with videoscaler with access to gluster instead of nfs |
[releng] |
16:15 |
<hashar> |
applying nfs::apache::labs on -dbdump to get /usr/local/apache from /data/project |
[releng] |
16:12 |
<hashar> |
-dbdump umounted deployment-nfs-memc:/mnt/export/apache on /usr/local/apache |
[releng] |
13:56 |
<hashar> |
archiving -nfs-memc:/mnt/export : root@deployment-nfs-memc:/mnt# mv export /data/project/deployment-nfs-memc_mnt-export_backup |
[releng] |
13:46 |
<hashar> |
seems to work fine with /data/project now. |
[releng] |
13:46 |
<hashar> |
rsync finished for both apache and upload6. Remounting and restarting apaches |
[releng] |
13:35 |
<hashar> |
rsync from nfs-memc:/mnt/export/upload6 to /data/project/upload6 completed. YEAHHH |
[releng] |
13:32 |
<hashar> |
on -dbdump, unmounted /mnt/upload (from nfs-memc). Please use the /mnt/upload6 -> /data/project/upload6 |
[releng] |
13:29 |
<hashar> |
root@deployment-nfs-memc:/mnt/export# rsync -a --progress --delete --inplace /mnt/export/apache /data/project |
[releng] |
13:22 |
<hashar> |
applying nfs::upload::labs to -dbdump so it uses /data/project/upload6 at /mnt/upload6 |
[releng] |
13:15 |
<hashar> |
manually umount /mnt/upload6 on apaches |
[releng] |
13:14 |
<hashar> |
stopping apache backends |
[releng] |
13:10 |
<hashar> |
Manually running puppet for {{gerrit|15545}} which should fix {{bug|38084}} uses /data/project instead of NFS instance |
[releng] |
09:35 |
<hashar> |
Regenerating captcha with the new shared key fixed {{bug|38699}} |
[releng] |
09:26 |
<hashar> |
Deleted all captchas in /mnt/upload6/private/captcha and regenerate them using 1000 and dirs=3 |
[releng] |
09:22 |
<hashar> |
Set $wmgCaptchaSecret in the local file common/wmf-config/PrivateSettings.php and used that value in captcha.py |
[releng] |
2012-07-25
§
|
17:00 |
<labs-logs-bottie> |
j: clear messages after updating localization(cache/l10n) to get new messages in TMH: php MWScript.php ../php/extensions/WikimediaMaintenance/clearMessageBlobs.php --wiki=aawiki |
[releng] |
14:43 |
<hashar> |
rebooting bits02 |
[releng] |
14:39 |
<hashar> |
dist-upgrade on cache-bits02, will reboot after that (bits.beta.wmflabs.org will be disabled while it reboot) |
[releng] |
14:24 |
<hashar> |
fixed puppet on cache-bits02 : ln -s /var/lib/git/operations/puppet/modules /etc/puppet/modules . That was an empty directory, thus prevented puppet to find the modules and made it breaking when trying to install ntp::client |
[releng] |
14:19 |
<hashar> |
root@deployment-cache-bits02:/var/lib/git/operations/puppet(git:13304/10)# git fetch anonymous refs/changes/04/13304/12 && git checkout -b 13304/12 FETCH_HEAD |
[releng] |
13:48 |
<hashar> |
rebooting apache33 so it can fsck /dev/vdb |
[releng] |
00:02 |
<labs-logs-bottie> |
j: run mwscript rebuildLocalisationCache.php --wiki=commonswiki --threads=2 |
[releng] |