|
2014-06-16
§
|
| 21:16 |
<bd808> |
Jenkins beta-scap-eqiad job broken because of missing puppet config on deployment-jobrunner01; needs role::beta::scap_target |
[releng] |
| 20:36 |
<bd808> |
Enabled puppet on deployment-jobrunner01 and forced a run |
[releng] |
| 20:34 |
<bd808> |
Puppet disabled on deployment-jobrunner01 since 2014-06-03; No SAL logs explaining why |
[releng] |
| 20:19 |
<bd808> |
Updated scap to 5adce72; trebuchet reported i-00000237 (deployment-videoscaler01) as not updating, but manual check shows it did sync properly |
[releng] |
| 20:00 |
<bd808> |
Deleted /var/lib/puppet/state/agent_catalog_run.lock on deployment-bastion after verifying that no puppet processes were running |
[releng] |
| 19:55 |
<bd808> |
Truncated /var/log/diamond/diamond.log and restarted diamond on deployment-bastion |
[releng] |
| 19:36 |
<bd808> |
/var/log/diamond is 787M of 1.2G total logs |
[releng] |
| 19:29 |
<bd808> |
/var 0% free on deployment-bastion; looking for things to clean-up |
[releng] |
|
2014-06-06
§
|
| 19:26 |
<bblack> |
- synced labs/private on deployment-salt again |
[releng] |
| 16:30 |
<bd808> |
Rebooted deployment-salt |
[releng] |
| 16:27 |
<bd808> |
Made /var/log a symlink to /srv/var-log on deployment-salt |
[releng] |
| 16:26 |
<bblack> |
Updated labs/private.git on puppetmaster. brings in updated zero+netmapper password for beta |
[releng] |
| 16:18 |
<bd808> |
Changed from role::labs::lvm::biglogs to role::labs::lvm::srv on deployment-salt and made /var/lib a symlink to /srv/var-lib |
[releng] |
| 15:45 |
<bd808> |
/var on deployment-salt still at 97% full after moving logs; /var/lib is our problem |
[releng] |
| 15:43 |
<bd808> |
Archived deployment-salt:/var/log to /data/project/deployment-salt |
[releng] |
| 15:40 |
<bd808> |
Disabled puppet on deployment-salt to work on disk space issues |
[releng] |
| 12:44 |
<hashar> |
Updated labs/private.git on puppetmaster. Brings Brandon Black change "add labs copy of zerofetcher auth file" {{gerrit|137918}} |
[releng] |
| 02:48 |
<mwalker> |
added role::labs::lvm::biglogs to deployment-salt because it is out of room on /var and I don't know what I can delete |
[releng] |
| 01:25 |
<bd808> |
Live hacked /etc/apache2/wmf/hhvm.conf on apaches to allow them to start |
[releng] |
| 00:30 |
<bd808> |
`git stash`ed dirty dblist files found in /a/common on deployment-bastion |
[releng] |
|
2014-06-04
§
|
| 20:46 |
<bd808> |
Fixed file ownership on /data/project/apache/uncommon for beta-recompile-math-texvc-eqiad job |
[releng] |
| 19:27 |
<manybubbles> |
sorry, can't do that yet, |
[releng] |
| 19:27 |
<manybubbles> |
plugins deployed to beta - time to restart Elasticsearch in beta - should cause not interruption of service |
[releng] |
| 19:01 |
<manybubbles> |
deploying Elasticsearch 1.2.1 and some updated plugins to beta |
[releng] |
| 17:11 |
<bd808> |
Unwedged the jenkins jobs to updating beta by stopping the stuck db update job |
[releng] |
| 16:27 |
<bd808> |
Changed uid/git for files owned by l10nupdate user |
[releng] |
| 09:50 |
<mwalker> |
Reset salt caches by running `salt '*' state.clear_cache` from deployment-salt -- deployment-pdf01 now no longer reports errors when returning status for deployment |
[releng] |