|
2012-05-25
§
|
| 15:18 |
<beta-logmsgbot> |
hashar: deleted jobrunner06 (precise), we just need one precise instance which will be jobrunner05 for now |
[releng] |
| 14:45 |
<beta-logmsgbot> |
hashar: installing applicationserver::homeless and applicationserver::jobrunner on jobrunner06 |
[releng] |
| 14:42 |
<beta-logmsgbot> |
hashar: installing applicationserver::homeless and applicationserver::jobrunner on jobrunner05 |
[releng] |
| 14:27 |
<beta-logmsgbot> |
hashar: installed jobrunner05 and 06 using Ubuntu precise. Should let get a 0.27 ffmpeg installation for {{bug|37043}} |
[releng] |
| 08:38 |
<beta-logmsgbot> |
root: on dbdump, deleted /etc/logrotate.d/mw-udp2log . Most probably in conflict with the one from deployment-feed which host the udp2log process |
[releng] |
| 08:35 |
<beta-logmsgbot> |
root: gzipped /home/wikipedia/logs/archive/*20120525 see {{bug|37012}} :-( |
[releng] |
| 08:23 |
<beta-logmsgbot> |
hashar: killed stuck jobs on jobrunner 02 and 03. Restarted loop. |
[releng] |
|
2012-05-23
§
|
| 19:54 |
<hashar> |
rebooting apache20 following installation of imagescaler puppet class |
[releng] |
| 19:49 |
<hashar> |
rebooting apache23 following installation of imagescaler puppet class |
[releng] |
| 19:46 |
<hashar> |
rebooting apache22 following installation of imagescaler puppet class |
[releng] |
| 19:15 |
<hashar> |
rebooting apache21 following installation of imagescaler puppet class |
[releng] |
| 18:48 |
<hashar> |
Adding puppet class 'imagescaler' on all deployment-apacheXX instances in an attempt to fix thumbnails |
[releng] |
| 18:36 |
<labs-logs-bottie> |
hashar: relocalisation cache done 367/367 languages rebuilt |
[releng] |
| 18:28 |
<labs-logs-bottie> |
hashar: running `mwscript rebuildLocalisationCache.php --wiki=aawiki` for {{bug|36806}} |
[releng] |
| 16:22 |
<labs-logs-bottie> |
hashar: delete all 3 webVideoTranscode jobs from enwiki database |
[releng] |
| 16:10 |
<hashar> |
rebooted jobrunner03 to check everything works fine there |
[releng] |
| 15:41 |
<hashar> |
deleting jobrunner01, it is crashed beyond repair. Will create a new one named jobrunner03 |
[releng] |
| 15:27 |
<hashar> |
rebooting jobrunner01 to see how it goes |
[releng] |
| 14:56 |
<hashar> |
stopped job runner on jobrunner01, amounted /mnt/upload6 and /mnt/ |
[releng] |
| 14:37 |
<hashar> |
running puppet on job runner to check change 8584 & 8585 worked |
[releng] |
|
2012-05-22
§
|
| 13:49 |
<hashar> |
Deleting jobrunner03 and 04, not going to need them afterall |
[releng] |
| 13:24 |
<hashar> |
deleting refreshLinks2 jobs from enwiki database |
[releng] |
| 12:52 |
<hashar> |
deleting deployment-jobrunner{3,4} installation failed I got permission denied. Will recreate them using same hostname |
[releng] |
| 11:55 |
<hashar> |
create two more job runner instances |
[releng] |
| 10:09 |
<hashar> |
Remove deployment-webs instance which was meant to emulate the HTTPS access. Hacky and low priority for now, we will need to setup a nginx proxy one day to properly replicate the production infrastructure. |
[releng] |
| 09:39 |
<labs-logs-bottie> |
hashar: rebooting jobrunner02 just to be sure it is properly loaded up |
[releng] |
| 09:30 |
<labs-logs-bottie> |
hashar: jobrunner logs are available in /home/wikipedia/logs/runJobs.log now |
[releng] |
| 09:25 |
<hashar> |
Fixed udp2log not able to add new log files in /home/wikipedia/log , that dir need to be writable by udp2log user! See https://gerrit.wikimedia.org/r/8442 | https://bugzilla.wikimedia.org/37014 |
[releng] |
| 08:54 |
<hashar> |
purged all logs from /home/wikipedia/logs/archive/ just to be safe |
[releng] |
| 08:41 |
<hashar> |
restarted upd2log on -feed (again) |
[releng] |
| 08:23 |
<hashar> |
started job loop on deployment-job-runner02 |
[releng] |
| 07:49 |
<hashar> |
installing jobrunner2 |
[releng] |
| 05:11 |
<hashar> |
creating a second job runner instance deployment-jobrunner02 . Will apply puppet classes later on. |
[releng] |
| 03:47 |
<labs-logs-bottie> |
hashar: (Bug 36870) deleting deployment-web{,3,4,5} |
[releng] |
|
2012-05-21
§
|
| 21:11 |
<hashar> |
deployment-nfs-memc : fix user right for upload6 : chown apache /mnt/export/upload6 |
[releng] |
| 21:11 |
<hashar> |
On deployment-nfs-memc : added apache (uid 48) entry in /etc/passwd |
[releng] |
| 21:11 |
<hashar> |
Adding Faidon and Platonides to default CC list of "depoyment prep (beta)" component |
[releng] |
| 21:11 |
<hashar> |
In Bugzilla, I have removed Petr Bena as a default assignee of bugs opened for "deployment-prep (beta)" component. Default is now "Nobody", Petr is on CC. That will makes bug triage a bit easier. |
[releng] |
| 08:58 |
<hashar> |
rerebooting deployment-feed |
[releng] |
| 08:53 |
<hashar> |
Looks like -feed is dead : EXT3-fs: INFO: recovery required on readonly filesystem. |
[releng] |
| 08:49 |
<hashar> |
rebooting deployment-feed |
[releng] |
| 08:22 |
<hashar> |
installing iotop on deployment-nfs-memc |
[releng] |