|
2016-05-11
§
|
| 16:43 |
<hashar> |
Reduced number of executors on Trusty instances from 3 to 2. Memory get exhausted causing the tmpfs to drop files and thus MW jobs to fail randomly. |
[releng] |
| 13:33 |
<hashar> |
Added contint::packages::php to Nodepool images T119139 |
[releng] |
| 12:59 |
<hashar> |
Dropping texlive and its dependencies from gallium. |
[releng] |
| 12:52 |
<hashar> |
deleted integration-dev |
[releng] |
| 12:51 |
<hashar> |
creating integration-dev instance to hopefully have Shinken clean itself |
[releng] |
| 11:42 |
<hashar> |
rebooting deployment-aqs01 via wikitech T134981 |
[releng] |
| 10:46 |
<hashar> |
beta/ci puppetmaster : deleting old tags in /var/lib/git/operations/puppet and repacking the repos |
[releng] |
| 08:49 |
<hashar> |
Deleting instances deployment-memc02 and deployment-memc03 (Precise instances, migrated to Jessie) #T134974 |
[releng] |
| 08:43 |
<hashar> |
Beta: switching memcached to new Jessie servers by cherry picking https://gerrit.wikimedia.org/r/#/c/288156/ and running puppet on mw app servers #T134974 |
[releng] |
| 08:20 |
<hashar> |
Creating deployment-memc04 and deployment-memc05 to switch beta cluster memcached to Jessie. m1.medium with security policy "cache" T13497 |
[releng] |
| 01:44 |
<matt_flaschen> |
Created Flow-specific External Store tables (blobs_flow1) on all wiki databases on Beta Cluster: T128417 |
[releng] |
|
2016-05-10
§
|
| 19:17 |
<hashar> |
beta / CI purging old Linux kernels: salt -v '*' cmd.run 'dpkg -l|grep ^rc|awk "{ print \$2 }"|grep linux-image|xargs dpkg --purge' |
[releng] |
| 17:34 |
<cscott> |
updated OCG to version b0c57a1c6890e9fa1f2c3743fc14cb6a7f244fc3 |
[releng] |
| 16:44 |
<bd808> |
Cleaned up 8.5G of pbuilder tmp output on integration-slave-jessie-1001 with `sudo find /mnt/pbuilder/build -maxdepth 1 -type d -mtime +1 -exec rm -r {} \+` |
[releng] |
| 16:35 |
<bd808> |
https://integration.wikimedia.org/ci/job/debian-glue failure on integration-slave-jessie-1001 due to /mnt being 100$ full |
[releng] |
| 14:20 |
<hashar> |
deployment-puppetmaster mass cleaned packages/service/users etc T134881 |
[releng] |
| 13:54 |
<moritzm> |
restarted zuul-merger on scandium for openssl update |
[releng] |
| 13:52 |
<moritzm> |
restarting zuul on gallium for openssl update |
[releng] |
| 13:51 |
<moritzm> |
restarted apache and zuul-merger on gallium for openssl update |
[releng] |
| 13:48 |
<hashar> |
deployment-puppetmaster : dropping role::ci::jenkins_access role::ci::slave::labs and role::ci::slave::labs::common T134881 |
[releng] |
| 13:46 |
<hashar> |
Deleting Jenkins slave deployment-puppetmaster T134881 |
[releng] |
| 13:45 |
<hashar> |
Change https://integration.wikimedia.org/ci/job/beta-build-deb/ job to use label selector "DebianGlue && DebianJessie" instead of "BetaDebianRepo" T134881 |
[releng] |
| 13:33 |
<hashar> |
Migrating all debian glue jobs to Jessie permanent slaves T95545 |
[releng] |
| 13:30 |
<hashar> |
Adding integration-slave-jessie-1002 in Jenkins. it is all puppet compliant |
[releng] |
| 12:59 |
<thcipriani|afk> |
triggering puppet run on scap targets in beta for https://gerrit.wikimedia.org/r/#/c/287918/ cherry pick |
[releng] |
| 09:07 |
<hashar> |
fixed puppet.conf on deployment-cache-text04 |
[releng] |
|
2016-05-09
§
|
| 20:57 |
<hashar> |
Unbroke puppet on integration-raita.integration.eqiad.wmflabs . Puppet was blocked because role::ci::raita was no more. Fixed by rebasing https://gerrit.wikimedia.org/r/#/c/208024 T115330 |
[releng] |
| 20:13 |
<hashar> |
beta: salt -v '*' cmd.run 'dpkg --purge libganglia1 ganglia-monitor; rm -fR /etc/ganglia' # T134808 |
[releng] |
| 20:06 |
<hashar> |
CI, removing ganglia configuration entirely via: salt -v '*' cmd.run 'rm -fRv /etc/ganglia' # T134808 |
[releng] |
| 20:04 |
<hashar> |
CI, removing ganglia configuration entirely via: salt -v '*' cmd.run 'dpkg --purge ganglia-monitor' # T134808 |
[releng] |
| 16:32 |
<jzerebecki> |
reloading zuul for 3e2ab56..d663fd0 |
[releng] |
| 15:39 |
<andrewbogott> |
migrating deployment-flourine to labvirt1009 |
[releng] |
| 15:39 |
<hashar> |
Adding label contintLabsSlave to integration-slave-jessie1001 and integration-slave-jessie1002 |
[releng] |
| 15:26 |
<hashar> |
Creating integration-slave-jessie-1001 T95545 |
[releng] |
|
2016-05-04
§
|
| 21:28 |
<cscott> |
deployed puppet FQDN domain patch for OCG: https://gerrit.wikimedia.org/r/286068 and restarted ocg on deployment-pdf0[12] |
[releng] |
| 15:03 |
<hashar> |
beta-scap: deployment-tin.deployment-prep.eqiad.wmflabs Name or service not known |
[releng] |
| 15:03 |
<hashar> |
beta-scap: deployment-tin.deployment-prep.eqiad.wmflabs |
[releng] |
| 12:24 |
<hashar> |
deleting Jenkins job mediawiki-core-phpcs , replaced by Nodepool version mediawiki-core-phpcs-trusty T133976 |
[releng] |
| 12:11 |
<hashar> |
beta: restarted nginx on varnish caches ( systemctl restart nginx.service ) since they were not listening on port 443 #T134362 |
[releng] |
| 11:07 |
<hashar> |
restarted CI puppetmaster (out of memory leak) |
[releng] |
| 10:57 |
<hashar> |
CI: mass upgrading deb packages |
[releng] |
| 10:53 |
<hashar> |
beta: clearing out leftover apt conf that points to unreachable web proxy : salt -v '*' cmd.run "find /etc/apt -name '*-proxy' -delete" |
[releng] |
| 10:48 |
<hashar> |
Manually fixing nginx upgrade on deployment-cache-text04 and deployment-cache-upload04 see T134362 for details |
[releng] |
| 09:27 |
<hashar> |
deployment-cache-text04 systemctl stop varnish-frontend.service . To clear out all the stuck CLOSE_WAIT connections T134346 |
[releng] |
| 08:33 |
<hashar> |
fixed puppet on deployment-cache-text04 (race condition generating puppet.conf ) |
[releng] |