351-400 of 3961 results (15ms)
2015-10-17 §
03:45 <bd808> Freed diskspace on deployment-bastion with `sudo /etc/cron.daily/acct; sudo rm /var/log/account/pacct.?*` [releng]
2015-10-16 §
20:51 <hashar> Restarting Jenkins to remove potential dead locks before the week-end [releng]
20:34 <hashar> cancelled a bunch of https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-publish/ jobs. We keep rebuilding over and over REL* merged changes [releng]
20:24 <hashar> disconnected / reconnected a bunch of trusty slaves. Seems some node executors were disabled/deadlocked [releng]
12:56 <hashar> Added Cards and RelatedArticles to the shared jobs mediawiki-extensions-*' https://gerrit.wikimedia.org/r/#/c/246818/ [releng]
2015-10-15 §
22:27 <legoktm> deploying https://gerrit.wikimedia.org/r/246785 [releng]
20:33 <SMalyshev> cherry-picked https://gerrit.wikimedia.org/r/#/c/240888/1 to deployment-puppetmaster.eqiad.wmflabs to test portal deployment [releng]
04:34 <bd808> freed another 258M on deployment-bastion by forcing an early rotation of /var/log/account/pacct and deleting the archived copy [releng]
04:19 <bd808> Freed 290M on deployment-bastion:/var by deleting old pacct files [releng]
2015-10-14 §
18:02 <legoktm> deploying https://gerrit.wikimedia.org/r/246276 [releng]
16:22 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/246256 [releng]
13:02 <hashar> Adjusting Jenkins SMTP server from polonium.wikimedia.org to mx1001.wikimedia.org [releng]
2015-10-13 §
03:21 <bd808> Updated scap to 13c2af4 (Fix sync-dblist to go with dblist moves to folder) [releng]
2015-10-12 §
08:54 <hashar> zuul-merger process leaked file descriptors and ended up unable to open any more files. Fixed by restarting the service on gallium. https://phabricator.wikimedia.org/T115243 [releng]
08:44 <hashar> Zuul CI in trouble. zuul-merger can't not apply patches anymore https://phabricator.wikimedia.org/T115243 [releng]
02:43 <Krenair> fixed puppet on deployment-conf03 by running several manual apt-get commands [releng]
00:38 <bd808> fixed puppet on deploymnet-restbase01 by running several manual apt-get and dpkg commands; had to downgrade zsh from 5.1.1-1 (unstable) to 5.0.7-5 (stable) [releng]
00:27 <bd808> puppet failing on deployment-restbase01 due to corrupt apt config state [releng]
2015-10-11 §
14:57 <legoktm> deploying https://gerrit.wikimedia.org/r/244192 [releng]
2015-10-09 §
21:57 <greg-g> 21:51 < ori> !log deployment-prep Accidentally clobbered /etc/init.d/mysql on deployment-db1, causing deployment-prep failures. Restored now [releng]
21:55 <twentyafterfour> deployment-db1 has a running mysqld again, shinken reports recovery. [releng]
21:41 <twentyafterfour> ori broke mariadb on deployment-db1 :-P [releng]
20:28 <hashar> beta cluster parsoid now runs from /parsoid.git && npm install (was from /deploy.git previously). In case of troubles poke subbu and see revert instructions on https://phabricator.wikimedia.org/T92871 [releng]
20:16 <hashar> Parsoid on beta is broken. Busy installing npm dependencies [releng]
20:09 <hashar> switching Parsoid on beta to install dependencies with npm (instead of /deploy) https://phabricator.wikimedia.org/T92871 for subbu [releng]
14:54 <hashar> added Geodata as a dependency to the wikibase jobs ( https://gerrit.wikimedia.org/r/#/c/244489/ ) [releng]
2015-10-07 §
15:49 <bd808> Updated cherry-pick of https://gerrit.wikimedia.org/r/#/c/241984 on deployment-puppetmaster and forced puppet run on deployment-logstash2 [releng]
15:48 <bd808> Made [LOCAL HACK] commit on deployment-puppetmaster for l10nupdate cache location change [releng]
15:45 <bd808> Uncommited local change to /var/lib/git/operations/puppet/modules/scap/files/l10nupdate-1 on deployment-puppetmaster. Looks like something Krenair was working on. [releng]
13:57 <hashar> fixing up apt on deployment-cache-parsoid04 as well as salt install https://phabricator.wikimedia.org/T114755 [releng]
13:52 <hashar> Upgrading packages oon deployment-cache-{text04,mobile04,upload04} , downgrading salt-common and salt-minion in the process ( https://phabricator.wikimedia.org/T114755 ) [releng]
13:43 <hashar> Cleaning up <tt>http://debian.saltstack.com/debian/ jessie-saltstack/main</tt> from some beta-cluster instances | https://phabricator.wikimedia.org/T114755 [releng]
10:09 <hashar> deleted angry-caching-proxy.integration.eqiad.wmflabs | was an experiment no more use for it [releng]
00:16 <Krinkle> Restored normal mysql replication on deployment-beta [releng]
2015-10-06 §
23:39 <Krinkle> Messing with mysql slave replication on deployment-db2 to investigate CentralAuth bug with AaronSchulz [releng]
13:14 <hashar> mass apt-get upgrade on beta cluster ( salt '*' pkg.upgrade ) [releng]
13:09 <hashar> Upgrades are now done on a daily basis thanks to unattended upgrade ( https://gerrit.wikimedia.org/r/243925 ) [releng]
13:08 <hashar> Mass dist-upgrade of all jenkins slaves [releng]
04:04 <legoktm> deploying https://gerrit.wikimedia.org/r/243828 [releng]
2015-10-05 §
22:20 <Krenair> deployment-prep deleted deployment-bastion:/var/log/account files to free up some space on /var [releng]
22:10 <Krenair> deployment-prep Renamed deployment-bastion:/srv/l10nupdate/mediawiki/extensions/BlueSpiceExtensions to OldBlueSpiceExtensions, was not in a clean state for l10nupdate [releng]
22:00 <Krenair> deployment-prep Doing a manual run of l10nupdate [releng]
21:58 <Krenair> deployment-prep Fixed l10nupdate - I broke it with https://gerrit.wikimedia.org/r/#/c/228126/ - created https://wikitech.wikimedia.org/wiki/Hiera:Deployment-prep/host/deployment-bastion [releng]
20:25 <hashar> Image ci-jessie-wikimedia-1444076296 in wmflabs-eqiad is ready. It provides bundler/ruby [releng]
20:18 <hashar> nodepool image-update wmflabs-eqiad ci-jessie-wikimedia [releng]
15:28 <hashar> integration-slave-jessie-1001 ssh reacheable [releng]
15:28 <hashar> integration-slave-jessie-1001 pass puppet again. Caused by a conflict with the 'zip' package, solved by cherry picking https://gerrit.wikimedia.org/r/#/c/243674/ [releng]
15:21 <jzerebecki> reloading zuul for d944bbb..1272657 [releng]
14:48 <hashar> integration-slave-jessie-1001 fails ssh auth with: error: AuthorizedKeysCommand /usr/sbin/ssh-key-ldap-lookup returned status 1 [releng]
14:45 <hashar> rebooting integration-slave-jessie-1001.integration.eqiad.wmflabs , does not reply to ssh [releng]