2401-2450 of 4520 results (17ms)
2014-12-19 §
18:23 <bd808> redis input to logstash stuck; restarted service [releng]
18:16 <bd808> ran `apt-get dist-upgrade` on logstash01 [releng]
18:02 <bd808> removed local mwdeploy user & group from videoscaler01 [releng]
18:01 <bd808> deployment-videoscaler01 has mysteriously aquired a local mwdeploy user instead of the ldap one [releng]
17:58 <bd808> forcing puppet run on deploymnet-videoscaler01 [releng]
07:24 <Krinkle> Restarting Gearman connection to Jenkins [releng]
07:24 <Krinkle> Attempt #5 at re-creating integration-slave1001. Completed provisioning per Setup instructions. Pooled. [releng]
05:33 <Krinkle> Rebasing integration-puppetmaster with latest operations/puppet upstream (5 patches) [releng]
00:06 <bd808> restored local commit with ssh keys for scap to deployment-salt [releng]
2014-12-18 §
23:57 <bd808> temporarily disabled jenkins scap job [releng]
23:56 <bd808> killed some ancient screen sessions on deployment-bastion [releng]
23:53 <bd808> Restarted udp2log-mw on deployment-bastion [releng]
23:53 <bd808> Restarted salt-minion on deployement-bastion [releng]
23:47 <bd808> Updated scap to latest HEAD version [releng]
21:57 <Krinkle> integration-slave1005 is not ready. It's incompletely setup due to https://phabricator.wikimedia.org/T84917 [releng]
19:29 <marxarelli> restarted puppetmaster on deployment-salt [releng]
19:29 <marxarelli> seeing "Could not evaluate: getaddrinfo: Temporary failure in name resolution" in the deployment-* puppet logs [releng]
14:17 <hashar> deleting instance deployment-parsoid04 and removing it from Jenkins [releng]
14:08 <hashar> restarted varnish backend on parsoidcache02 [releng]
14:00 <hashar> parsoid05 seems happy: curl http://localhost:8000/_version: <tt>{"name":"parsoid","version":"0.2.0-git","sha":"d16dd2db6b3ca56e73439e169d52258214f0aeb2"}</tt> [releng]
14:00 <hashar> parsoid05 seems happy: curl http://localhost:8000/_version<br/> [releng]
13:56 <hashar> applying latest changes of Parsoid on parsoid05 via: <tt>zuul enqueue --trigger gerrit --pipeline postmerge --project mediawiki/services/parsoid --change 180671,2</tt> [releng]
13:56 <hashar> parsoid05: disabling puppet, stopping parsoid, rm -fR /srv/deployment/parsoid ; rerunning the Jenkins beta-parsoid-update-eqiad to hopefully recreate everything properly [releng]
13:52 <hashar> making parsoid05 a Jenkins slave to replace parsoid04 [releng]
13:24 <hashar> apt-get upgrade on parsoidcache02 and parsoid04 [releng]
13:23 <hashar> updated labs/private on puppet master to fix a puppet dependency cycle with sudo-ldap [releng]
13:19 <hashar> rebased puppetmaster repo [releng]
12:52 <hashar> deleting the workspace for the beta-parsoid-update-eqiad jenkins job on deployment-parsoid04 . Some file belong to root which prevent the job from processing [releng]
09:13 <hashar> enabled MediaWiki core 'structure' PHPUnit tests for all extensions. Will require folks to fix their incorrect AutoLoader and RessourceLoader entries. {{gerrit|180496}} {{bug|T78798}} [releng]
2014-12-17 §
21:02 <hashar> cancelled all browser tests,suspecting them to deadlock Jenkins somehow :( [releng]
2014-12-16 §
17:17 <bd808> git-sync-upstream runs cleanly on deployment-salt again! [releng]
17:16 <bd808> removed cherry pick of Ib2a0401a7aa5632fb79a5b17c0d0cef8955cf990 (-2 by _joe_; replaced by Ibcad98a95413044fd6c5e9bd3c0a6fb486bd5fe9) [releng]
17:15 <bd808> removed cherry pick of I3b6e37a2b6b9389c1a03bd572f422f898970c5b4 (modified in gerrit by bd808 and not repicked; merged) [releng]
17:15 <bd808> removed cherry pick of I08c24578596506a1a8baedb7f4a42c2c78be295a (-2 by _joe_ in gerrit; replaced by Iba742c94aa3df7497fbff52a856d7ba16cf22cc7) [releng]
17:13 <bd808> removed cherry pick of I6084f49e97c855286b86dbbd6ce8e80e94069492 (merged by Ori with a change) [releng]
17:09 <bd808> trying to fix it without using important changes [releng]
17:08 <bd808> deployment-salt:/var/lib/git/operations/puppet is a rebase hell of cherry-picks that don't apply [releng]
16:52 <manybubbles> elasticsearch restart finished [releng]
16:48 <mutante> deployment-db2 is down [releng]
16:48 <manybubbles> restarting beta's elasticsearch servers to pick up a new version of a plugin. won't interfere with current downtime. [releng]
13:51 <hashar> deleting integration-slave1001 and recreating it. It is blocked on boot and we can't console on it https://phabricator.wikimedia.org/T76250 [releng]
2014-12-15 §
23:24 <Krinkle> integration-slave1001 isn't coming back (T76250), building integration-slave1005 as its replacement. [releng]
12:53 <YuviPanda> manually restarted diamond on all betalabs host, to see if that is why metrics aren’t being sent anymore [releng]
09:41 <hashar> deleted hhvm core files in /var/tmp/core from both mediawiki01 and mediawiki02 {{T1259}} and {{T71979}} [releng]
2014-12-13 §
18:51 <bd808> Running chmod -R g+s /data/project/upload7 on deploymnet-mediawiki02 [releng]
18:25 <bd808> Running chmod -R u=rwX,g=rwX,o=rX /data/project/upload7 from deployment-mediawiki02 [releng]
18:16 <bd808> chown done for /data/project/upload7 [releng]
17:51 <bd808> Running chown -R apache:apache on /data/project/upload7 from deployment-mediawiki02 [releng]
17:11 <bd808> Labs DNS seems to be flaking out badly and causing random scap and puppet failures [releng]
17:10 <bd808> Many strange puppet and scap failures in beta that look to be related to DNS failures [releng]