4251-4300 of 10000 results (30ms)
2017-07-20 §
09:03 <hashar> Restoring castorby updating all jobs to point to castor02 ( https://gerrit.wikimedia.org/r/366524 ) Starts with a cold cache :( - T171148 [releng]
08:53 <hashar> Created castor02.integration.eqiad.wmflabs with puppet role role::ci::castor::server and adding it to Jenkins. Will then update the Jenkins jobs to point to it - T171148 [releng]
08:00 <hashar> Disabled castor entirely via https://gerrit.wikimedia.org/r/366520 . The instance is broken - T171148 [releng]
07:55 <hashar> Refreshing all Jenkins jobs defined in JJB in order to then disable castor entirely for T171148 [releng]
07:09 <_joe_> rebooting castor, jobs are failing, and no one seems able to login [releng]
07:05 <_joe_> adding myself to projectadmins for integration, trying to troubleshoot castor [releng]
01:38 <thcipriani> scap on beta was failing because during the ldap downtime puppet created a shadow mwdeploy user, fixed using vipw and vigr [releng]
2017-07-19 §
14:43 <hashar> Jenkins: uploaded a patched android-emulator plugin for T150623 and restarting Jenkins [releng]
13:55 <hashar> Jenkins: added JDK "Debian - OpenJdk 7" with JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 [releng]
12:54 <hashar> Gerrit: created repo integration/jenkinsci/android-emulator-plugin.git owned by access group integration-jenkinsci-android-emulator-plugin which has Mholloway - T170904 [releng]
2017-07-18 §
16:26 <halfak> manually restarted uwsgi-ores and celery-ores-worker on deployment-sca03 [releng]
16:19 <halfak> manually installed "aspell-el" on deployment-sca03 (work around for ongoing puppet issues) [releng]
09:04 <hashar> deleted integration-slave-trusty-1006 [releng]
03:57 <twentyafterfour> Fixed deployment-imagescaler01 by cherry-picking https://gerrit.wikimedia.org/r/#/c/365891/ on deployment-puppetmaster02 [releng]
2017-07-17 §
18:20 <thcipriani> reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/365198 [releng]
2017-07-14 §
20:16 <Amir1> cpan[1]> install LWP::UserAgent on tin [releng]
2017-07-13 §
17:04 <thcipriani> restarting jenkins for updates [releng]
2017-07-12 §
20:07 <bearND> Update mobileapps to d30dae2 [releng]
18:19 <greg-g> where "things" == nodepool instance delete/creation [releng]
18:18 <greg-g> things are back to a bad state, chase etc investigating [releng]
17:52 <greg-g> nodepool is back to making instances and running jobs, thanks Cloud team [releng]
17:22 <greg-g> CI is backed up, only one nodepoll instance running for the last long while, many in building [releng]
00:35 <legoktm> deploying https://gerrit.wikimedia.org/r/364628 [releng]
2017-07-11 §
21:30 <legoktm> deploying https://gerrit.wikimedia.org/r/364601 [releng]
2017-07-09 §
01:15 <Amir1> ladsgroup@deployment-tin:~$ mwscript extensions/ORES/maintenance/CheckModelVersions.php --wiki=enwiki (T170026, T165716) [releng]
2017-07-07 §
14:53 <hashar> deployment-prep: change webproxy http://recommendation-api-beta.wmflabs.org/ to deployment-sca02 (has the proper security rule) - T148129 [releng]
14:53 <hashar> deployment-prep: add port 9632 to security group "sca" https://horizon.wikimedia.org/project/access_and_security/security_groups/593/ - T148129 [releng]
14:03 <hashar> Image snapshot-ci-trusty-1499435837 in wmflabs-eqiad is ready [releng]
13:57 <hashar> Nodepool: updating snapshot-ci-trusty [releng]
13:56 <hashar> Nodepool: uploaded new Ubuntu Trusty image [releng]
2017-07-06 §
17:28 <thcipriani> committed changes to modules/kafkatee on deployment-puppetmaster02 since having them uncommitted broke git-sync-upstream [releng]
16:20 <hashar> Deleting Nodepool snapshot snapshot-ci-jessie-1499350442 - faulty php7.0-sqlite package that breaks phan jobs - T169904 [releng]
15:29 <hashar> deployment-cache-upload04 manually ran apt-get upgrade to downgrade ldap-utils and libldap-2.4-2 (caused puppet failure) [releng]
14:14 <hashar> regenerating mediawiki-core-qunit-selenium-jessie jenkins job [releng]
12:05 <hashar> deployment-prep created Web proxy for recommendation-api-beta.wmflabs.org -> http://10.68.20.183:9632 (deployment-sca01) for schana [releng]
02:38 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/363519 [releng]
2017-07-04 §
14:10 <hashar> manually upgraded apache2 on deployment-puppetmaster02 see T159254 [releng]
13:33 <hashar> beta cluster puppet is broken: Error: Could not send report: Connection refused - connect(2) for "deployment-puppetmaster02.deployment-prep.eqiad.wmflabs" port 8140 [releng]
09:28 <hashar> gerrit: marking read-only mediawiki/extensions/Nonlinear - T169519 [releng]
2017-07-03 §
11:34 <hashar> jenkins: refreshing all jobs and updating the castor-save bit ( https://gerrit.wikimedia.org/r/#/c/361843/ ) [releng]
2017-06-30 §
08:16 <hashar> Gerrit: changing repos to read-only: analytics/kraken analytics/kraken/deploy analytics/vagrant/kraken - T169303 [releng]
2017-06-29 §
23:17 <legoktm> deploying https://gerrit.wikimedia.org/r/362314 [releng]
2017-06-28 §
15:55 <hashar> beta: git gc mediawiki repos in /srv/mediawiki-staging [releng]
15:47 <hashar> beta: git -C /srv/deployment/ores/deploy/submodules/editquality gc (saving 380MBytes) [releng]
15:33 <hashar> running git gc under /srv/mediawiki-staging [releng]
14:43 <hashar> pypi.python.org is back again - T169091 [releng]
14:33 <elukey> running alter tables on the EL database in deployment-eventlogging03.deployment-prep.eqiad.wmflabs [releng]
14:06 <hashar> pypi.python.org has an issue with its CDN . That would affect any CI jobs relying on tox/python - See https://status.python.org for updates and T169091 [releng]
14:04 <hashar> pypi.python.org has an issue with its CDN . That would affect any CI jobs relying on tox/python - See https://status.python.org for updates [releng]
10:06 <hashar> Unblocked beta cluster jenkins job. Have been stalled for a while [releng]