301-350 of 1761 results (4ms)
2014-07-02 §
07:49 <hashar> cxserver being configured! {{gerrit|140723}} by Kartik and Niklas \\O/ [releng]
2014-07-01 §
15:46 <bd808> Fixed git rebase conflict in operations/puppet on deployment-salt [releng]
13:29 <manybubbles> rebuilding Cirrus search index in beta to pick up new configuration and cache warmers [releng]
11:20 <hashar> Added Filippo Giunchedi to the project as an admin (WMF ops) [releng]
2014-06-30 §
20:47 <bd808> The state of puppet for beta is badly broken. I have hacked things to get puppet to apply on deployment-apache0[12] but puppet won't apply on deployment-bastion in part due to the same hacks. [releng]
18:48 <bd808> Created symlink /apache -> /usr/local/apache on deployment-apache0[12] to fix docroot symlinks [releng]
18:09 <bd808> Beta apaches are broken with latest puppet config applied. Working to correct. [releng]
18:08 <bd808> Manually added symlink for /etc/apache/wmf on deployment-apache0[12] [releng]
2014-06-26 §
12:48 <YuviPanda> cherry picked https://gerrit.wikimedia.org/r/#/c/142228/ to puppetmaster, sending events to charcoal.wmflabs.org now with projectname \\o/ [releng]
09:46 <YuviPanda> cherry-picked https://gerrit.wikimedia.org/r/#/c/142210/ on to puppetmaster [releng]
09:38 <hashar> Granting sudo to YuviPanda [releng]
2014-06-25 §
20:58 <bd808> Fixed rebase conflict in operations/puppet.git on deployment-salt caused by cherry-picked vcl patch left over from varnish submodule usage [releng]
2014-06-24 §
19:29 <bd808> Manually updated operations/puppet checkout on deployment-salt to deal with varnish submodule change [releng]
2014-06-19 §
22:47 <bd808> Updated scap to 792a572 [releng]
22:46 <bd808> Trebuchet runs on deployment-videoscaler01 are succeeding but not showing up in the `git deploy report` output [releng]
22:40 <bd808> Deleted /var/log/diamond/diamond.log on deployment-jobrunner01 because /var was full [releng]
2014-06-18 §
16:55 <bd808> Setup hourly cron as user bd808 on deployment-salt to test automatic update of puppet repo using ~bd808/git-sync-upstream script [releng]
2014-06-17 §
20:36 <bd808> Upgraded elasticsearch to version 1.2.1 on deployment-logstash1 [releng]
2014-06-16 §
21:16 <bd808> Jenkins beta-scap-eqiad job broken because of missing puppet config on deployment-jobrunner01; needs role::beta::scap_target [releng]
20:36 <bd808> Enabled puppet on deployment-jobrunner01 and forced a run [releng]
20:34 <bd808> Puppet disabled on deployment-jobrunner01 since 2014-06-03; No SAL logs explaining why [releng]
20:19 <bd808> Updated scap to 5adce72; trebuchet reported i-00000237 (deployment-videoscaler01) as not updating, but manual check shows it did sync properly [releng]
20:00 <bd808> Deleted /var/lib/puppet/state/agent_catalog_run.lock on deployment-bastion after verifying that no puppet processes were running [releng]
19:55 <bd808> Truncated /var/log/diamond/diamond.log and restarted diamond on deployment-bastion [releng]
19:36 <bd808> /var/log/diamond is 787M of 1.2G total logs [releng]
19:29 <bd808> /var 0% free on deployment-bastion; looking for things to clean-up [releng]
2014-06-09 §
15:19 <andrewbogott> doing a 'rebase origin' on deployment-salt, because it needs it. [releng]
15:10 <andrewbogott> updating all instances to puppet 3 via a cherry-pick�� of https://gerrit.wikimedia.org/r/#/c/137898/ on deployment-salt [releng]
2014-06-07 §
02:44 <bd808> Restarted logstash on deployment-logstash1; last even logged at 2014-06-06T22:11:04 [releng]
2014-06-06 §
19:26 <bblack> - synced labs/private on deployment-salt again [releng]
16:30 <bd808> Rebooted deployment-salt [releng]
16:27 <bd808> Made /var/log a symlink to /srv/var-log on deployment-salt [releng]
16:26 <bblack> Updated labs/private.git on puppetmaster. brings in updated zero+netmapper password for beta [releng]
16:18 <bd808> Changed from role::labs::lvm::biglogs to role::labs::lvm::srv on deployment-salt and made /var/lib a symlink to /srv/var-lib [releng]
15:45 <bd808> /var on deployment-salt still at 97% full after moving logs; /var/lib is our problem [releng]
15:43 <bd808> Archived deployment-salt:/var/log to /data/project/deployment-salt [releng]
15:40 <bd808> Disabled puppet on deployment-salt to work on disk space issues [releng]
12:44 <hashar> Updated labs/private.git on puppetmaster. Brings Brandon Black change "add labs copy of zerofetcher auth file" {{gerrit|137918}} [releng]
02:48 <mwalker> added role::labs::lvm::biglogs to deployment-salt because it is out of room on /var and I don't know what I can delete [releng]
01:25 <bd808> Live hacked /etc/apache2/wmf/hhvm.conf on apaches to allow them to start [releng]
00:30 <bd808> `git stash`ed dirty dblist files found in /a/common on deployment-bastion [releng]
2014-06-05 §
14:16 <manybubbles> rebuild beta's jawiki's search index without kuromoji - it didn't help much anyway [releng]
14:01 <manybubbles> elasticsearch cluster got really angry in beta when I restarted some node - its like they aren't talking to eachother properly - trying to recover. once that is done I'll upgrade to 1.2.1 and that might fix it [releng]
13:59 <hashar> deployment-elastic01 puppet was broken due to {{bug|63322}} i.e. having some HTML garbage as ec2id which would be used as puppet certname [releng]
13:47 <manybubbles> rolling restart of elasticsearch nodes in beta to pick up new kernel [releng]
2014-06-04 §
20:46 <bd808> Fixed file ownership on /data/project/apache/uncommon for beta-recompile-math-texvc-eqiad job [releng]
19:27 <manybubbles> sorry, can't do that yet, [releng]
19:27 <manybubbles> plugins deployed to beta - time to restart Elasticsearch in beta - should cause not interruption of service [releng]
19:01 <manybubbles> deploying Elasticsearch 1.2.1 and some updated plugins to beta [releng]
17:11 <bd808> Unwedged the jenkins jobs to updating beta by stopping the stuck db update job [releng]