|
2015-08-26
§
|
| 23:32 |
<Krenair> |
Re-armed keyholder on deployment-bastion |
[releng] |
| 21:51 |
<matt_flaschen> |
To fix https://gerrit.wikimedia.org/r/#/c/233952/1 on Beta, manually ran: while read line; do echo "Starting $line\n"; echo 'ALTER TABLE flow_wiki_ref DROP COLUMN ref_src_wiki;' | sql --write "$line"; echo "Finished $line\n"; done < /srv/mediawiki/all-labs.dblist |
[releng] |
| 16:39 |
<bd808> |
marked https://integration.wikimedia.org/ci/computer/integration-slave-precise-1014/ offline for git clone problems |
[releng] |
| 16:18 |
<marxarelli> |
deleted udp2log.log and restarted service. so far nothing out of `tail -fn0 udp2log.log` |
[releng] |
| 16:16 |
<marxarelli> |
stopping udp2log on deployment-flourine |
[releng] |
| 16:14 |
<marxarelli> |
udp2log is mostly "egrep: writing output: Broken pipe" |
[releng] |
| 16:10 |
<marxarelli> |
disk space at 97% on deployment-flourine, mainly due to 15G /var/log/udp2log/udp2log.log |
[releng] |
| 16:01 |
<bd808> |
sudo rm -rf integration-slave-precise-1014:/mnt/jenkins-workspace/workspace/mediawiki-core-phplint/.git |
[releng] |
| 09:57 |
<hashar> |
Bumping our JJB mirror a3aef64..f01628c Required for the Android Emulator plugin support ( https://phabricator.wikimedia.org/T110307 ) |
[releng] |
| 07:39 |
<hashar_> |
puppet is back in action on beta cluster |
[releng] |
| 07:38 |
<hashar_> |
enabling puppet agent on deployment-puppetmaster. It is disable with no reason given |
[releng] |
| 07:24 |
<hashar_> |
resetted beta cluster puppet master to origin/production . We have lost any cherry pick that might have existed |
[releng] |
| 07:16 |
<hashar_> |
started puppetmaster on deployment-puppetmaster |
[releng] |
| 07:11 |
<hashar> |
puppet fails on most beta cluster instances :-( |
[releng] |
|
2015-08-25
§
|
| 23:48 |
<thcipriani> |
stopping puppetmaster and disabling puppet runs on deployment-puppetmaster until we get a change to diagnose/rebuild (tomorrow) |
[releng] |
| 23:47 |
<thcipriani> |
deployment-puppetmaster showing signs of a corrupt disk "error: object file .git/objects/cc/026ba0cdc872490ef6a616b2bac4bb829639cd is empty" shutting it off for now. |
[releng] |
| 23:43 |
<thcipriani> |
reboot deployment-puppetmaster unreachable from other vms (labvirt1007 thing, probably) |
[releng] |
| 15:10 |
<hashar> |
unpooling and deleting integration-slave-trusty-1014 integration-slave-trusty-1017 and integration-slave-precise-1014 . They are most probably corrupted ( https://phabricator.wikimedia.org/T110052#1571184 ) |
[releng] |
| 15:04 |
<hashar> |
soft rebooting integration-slave-trusty-1014 (ssh dead) |
[releng] |
| 15:02 |
<hashar> |
tashing workspaces on integration-slave-trusty-1014 and integration-slave-trusty-1017 ( https://phabricator.wikimedia.org/T110052#1571184 ) |
[releng] |
| 14:55 |
<hashar> |
dropping all workspaces from integration-slave-precise-1014 . Some .git repos in workspaces might be corrupted |
[releng] |
| 11:47 |
<hashar> |
Upgraded a bunch of Jenkins plugins |
[releng] |
| 10:02 |
<hashar> |
deleted job https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-8-sauce/ Was disabled and no more in our JJB config |
[releng] |
| 08:52 |
<hashar> |
pooling back integration-slave-precise-1014 , integration-slave-trusty-1014 and integration-slave-trusty-1017 . labvirt1007 missed disk space ( https://phabricator.wikimedia.org/T110052 ) |
[releng] |
| 00:58 |
<bd808> |
Updated tin:/srv/deployment/integration/slave-scripts to b287e93 (Revert "Run mw-install-mysql.sh with statement tracing") and synced via trebuchet |
[releng] |
|
2015-08-24
§
|
| 23:36 |
<bd808> |
Updated tin:/srv/deployment/integration/slave-scripts to a2cdf4f (Run mw-install-mysql.sh with statement tracing) and synced via trebuchet |
[releng] |
| 21:51 |
<tgr> |
migrating OAuth to metawiki on beta - T108648 |
[releng] |
| 20:51 |
<bd808> |
Restarted puppetmaster on integration-puppetmaster |
[releng] |
| 20:47 |
<bd808> |
Ran sudo apt-get install hhvm hhvm-dev on integration-slave-trusty-1016 to get hhvm 3.6.5+dfsg1-1+wm3 |
[releng] |
| 18:34 |
<matt_flaschen> |
Completed on Beta Cluster: foreachwiki populateContentModel.php --ns=all --table=page | tee populateContentModel_page_table_all_namespaces_all_wikis_2015-08-24.log |
[releng] |
| 18:24 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/233480 |
[releng] |
| 18:23 |
<matt_flaschen> |
Running on Beta Cluster: foreachwiki populateContentModel.php --ns=all --table=page | tee populateContentModel_page_table_all_namespaces_all_wikis_2015-08-24.log |
[releng] |
| 15:23 |
<hashar> |
hard rebooting integration-slave-trusty-1017 |
[releng] |
| 15:19 |
<hashar> |
hard rebooting integration-slave-trusty-1014 |
[releng] |
| 15:16 |
<hashar> |
apt-get upgrade all Trusty slaves |
[releng] |
| 15:14 |
<hashar> |
apt-get upgrade on gallium |
[releng] |
| 15:12 |
<hashar> |
integration-slave-precise-1011 and integration-slave-precise-1014 went offline due to elasticsearch not starting properly. https://phabricator.wikimedia.org/T109497 |
[releng] |
| 15:06 |
<hashar> |
upgrading packages on precise slaves |
[releng] |
| 09:17 |
<hashar> |
recreating Jenkins job mwext-PronunciationRecording-jslint (still triggered by Zuul somehow) |
[releng] |