1151-1200 of 4541 results (12ms)
2015-08-26 §
07:38 <hashar_> enabling puppet agent on deployment-puppetmaster. It is disable with no reason given [releng]
07:24 <hashar_> resetted beta cluster puppet master to origin/production . We have lost any cherry pick that might have existed [releng]
07:16 <hashar_> started puppetmaster on deployment-puppetmaster [releng]
07:11 <hashar> puppet fails on most beta cluster instances :-( [releng]
2015-08-25 §
23:48 <thcipriani> stopping puppetmaster and disabling puppet runs on deployment-puppetmaster until we get a change to diagnose/rebuild (tomorrow) [releng]
23:47 <thcipriani> deployment-puppetmaster showing signs of a corrupt disk "error: object file .git/objects/cc/026ba0cdc872490ef6a616b2bac4bb829639cd is empty" shutting it off for now. [releng]
23:43 <thcipriani> reboot deployment-puppetmaster unreachable from other vms (labvirt1007 thing, probably) [releng]
15:10 <hashar> unpooling and deleting integration-slave-trusty-1014 integration-slave-trusty-1017 and integration-slave-precise-1014 . They are most probably corrupted ( https://phabricator.wikimedia.org/T110052#1571184 ) [releng]
15:04 <hashar> soft rebooting integration-slave-trusty-1014 (ssh dead) [releng]
15:02 <hashar> tashing workspaces on integration-slave-trusty-1014 and integration-slave-trusty-1017 ( https://phabricator.wikimedia.org/T110052#1571184 ) [releng]
14:55 <hashar> dropping all workspaces from integration-slave-precise-1014 . Some .git repos in workspaces might be corrupted [releng]
11:47 <hashar> Upgraded a bunch of Jenkins plugins [releng]
10:02 <hashar> deleted job https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce/ Was disabled and no more in our JJB config [releng]
08:52 <hashar> pooling back integration-slave-precise-1014 , integration-slave-trusty-1014 and integration-slave-trusty-1017 . labvirt1007 missed disk space ( https://phabricator.wikimedia.org/T110052 ) [releng]
00:58 <bd808> Updated tin:/srv/deployment/integration/slave-scripts to b287e93 (Revert "Run mw-install-mysql.sh with statement tracing") and synced via trebuchet [releng]
2015-08-24 §
23:36 <bd808> Updated tin:/srv/deployment/integration/slave-scripts to a2cdf4f (Run mw-install-mysql.sh with statement tracing) and synced via trebuchet [releng]
21:51 <tgr> migrating OAuth to metawiki on beta - T108648 [releng]
20:51 <bd808> Restarted puppetmaster on integration-puppetmaster [releng]
20:47 <bd808> Ran sudo apt-get install hhvm hhvm-dev on integration-slave-trusty-1016 to get hhvm 3.6.5+dfsg1-1+wm3 [releng]
18:34 <matt_flaschen> Completed on Beta Cluster: foreachwiki populateContentModel.php --ns=all --table=page | tee populateContentModel_page_table_all_namespaces_all_wikis_2015-08-24.log [releng]
18:24 <Krinkle> Reloading Zuul to deploy https://gerrit.wikimedia.org/r/233480 [releng]
18:23 <matt_flaschen> Running on Beta Cluster: foreachwiki populateContentModel.php --ns=all --table=page | tee populateContentModel_page_table_all_namespaces_all_wikis_2015-08-24.log [releng]
15:23 <hashar> hard rebooting integration-slave-trusty-1017 [releng]
15:19 <hashar> hard rebooting integration-slave-trusty-1014 [releng]
15:16 <hashar> apt-get upgrade all Trusty slaves [releng]
15:14 <hashar> apt-get upgrade on gallium [releng]
15:12 <hashar> integration-slave-precise-1011 and integration-slave-precise-1014 went offline due to elasticsearch not starting properly. https://phabricator.wikimedia.org/T109497 [releng]
15:06 <hashar> upgrading packages on precise slaves [releng]
09:17 <hashar> recreating Jenkins job mwext-PronunciationRecording-jslint (still triggered by Zuul somehow) [releng]
2015-08-23 §
00:37 <legoktm> deploying https://gerrit.wikimedia.org/r/233178 [releng]
2015-08-22 §
00:08 <marxarelli> Reloading Zuul to deploy I86fcd065f7a77e293a8882fa3ad2c20fffe4b092 [releng]
2015-08-21 §
18:31 <marxarelli> Deployed I4975a99f1c7d22c2f99f4557de4d7a081a5300f4 to integration slaves [releng]
17:59 <marxarelli> Updated 29 Jenkins jobs with minor changes to bundler execution and MW LocalSettings.php (closing php tag) (I2f32701dd9478857ed5a2fb1bfbe13e134d7b27c) [releng]
17:55 <marxarelli> Running `jenkins-jobs update` to deploy I2f32701dd9478857ed5a2fb1bfbe13e134d7b27c [releng]
2015-08-20 §
20:42 <legoktm> deploying https://gerrit.wikimedia.org/r/232839 [releng]
20:30 <legoktm> deploying https://gerrit.wikimedia.org/r/232837 [releng]
18:27 <legoktm> deploying https://gerrit.wikimedia.org/r/232749 [releng]
14:07 <jzerebecki> reloading zuul for 838af5a..0c601ae [releng]
2015-08-19 §
16:35 <marxarelli> Verified that elasticsearch and mysql are now running after the reinstall and a manual puppet run [releng]
16:32 <marxarelli> Running apt-get install --reinstall elasticsearch to re-create missing /var/run/elasticsearch (and possibly others) directory [releng]
16:29 <marxarelli> Investigating stopped mysql on integration-slave-precise-1014 [releng]
16:21 <marxarelli> Reloading Zuul to deploy I6815cd66169ee8f6fbb5ea394e3a10ce6b6e7609 [releng]
16:17 <marxarelli> Reloading Zuul to deploy I48ab39e330ebc71266b72cae8449cc2f6da495fe [releng]
14:51 <jzerebecki> reload zuul for 700f380..0384ff5 [releng]
2015-08-18 §
18:33 <jzerebecki> (mysql wasn't started as puppet never got to that point) [releng]
18:32 <jzerebecki> /etc/init.d/elasticsearch start was looping endlessly because /var/run/elasticsearch/ did not exist even though it is part of the debian package elasticsearch which was installed. fixed the issue on this instance by: integration-slave-precise-1013:~# apt-get install --reinstall elasticsearch [releng]
16:32 <jzerebecki> offlined integration-slave-precise-1013 : Fails to connect to mysl. /etc/init.d/mysql start fails. [releng]
16:00 <jzerebecki> reloading zuul for 6486889..700f380 [releng]
2015-08-17 §
22:18 <legoktm> running schema change for [[gerrit:202344]] on beta [releng]
19:19 <legoktm> freeing up disk space on 1012 [releng]