|
2015-04-23
§
|
| 04:35 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/206044 and https://gerrit.wikimedia.org/r/206072 |
[releng] |
| 02:49 |
<LocalisationUpdate> |
completed (1.26wmf3) at 2015-04-23 02:48:40+00:00 |
[production] |
| 02:46 |
<l10nupdate> |
Synchronized php-1.26wmf3/cache/l10n: (no message) (duration: 03m 46s) |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.26wmf2) at 2015-04-23 02:27:39+00:00 |
[production] |
| 02:24 |
<l10nupdate> |
Synchronized php-1.26wmf2/cache/l10n: (no message) (duration: 05m 46s) |
[production] |
| 00:29 |
<bd808> |
cherry-picked and applied https://gerrit.wikimedia.org/r/#/c/205969/ (logstash: Convert $::realm switches to hiera) |
[releng] |
| 00:17 |
<bd808> |
beta cluster fatal monitor full of "Bad file descriptor: AH00646: Error writing to /data/project/logs/apache-access.log" |
[releng] |
| 00:15 |
<kaldari> |
Synchronized wmf-config/InitialiseSettings.php: Turning on WikiGrok on English Wikipedia (for 2 week test) (duration: 00m 11s) |
[production] |
| 00:07 |
<krenair> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/206024 (duration: 00m 14s) |
[production] |
| 00:05 |
<krenair> |
Synchronized php-1.26wmf2/extensions/ZeroBanner/includes/ZeroSpecialPage.php: https://gerrit.wikimedia.org/r/#/c/206023/ (duration: 00m 13s) |
[production] |
| 00:03 |
<bd808> |
cleaned up redis leftovers on deployment-logstash1 |
[releng] |
|
2015-04-22
§
|
| 23:57 |
<bd808> |
cherry-picked and applied https://gerrit.wikimedia.org/r/#/c/205968 (remove redis from logstash) |
[releng] |
| 23:47 |
<krenair> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/206015 (duration: 00m 12s) |
[production] |
| 23:44 |
<krenair> |
Synchronized php-1.26wmf3/extensions/ZeroBanner/includes/ZeroSpecialPage.php: https://gerrit.wikimedia.org/r/#/c/206017/ (duration: 00m 13s) |
[production] |
| 23:33 |
<bd808> |
reset deployment-salt:/var/lib/git/operations/puppet HEAD to production; forced update with upstream; re-cherry-picked I46e422825af2cf6f972b64e6d50040220ab08995 |
[releng] |
| 23:29 |
<bd808> |
deployment-salt:/var/lib/git/operations/puppet in detached HEAD state; looks to be for cherry pick of I46e422825af2cf6f972b64e6d50040220ab08995 ? |
[releng] |
| 23:26 |
<krenair> |
Synchronized php-1.26wmf3/extensions/Flow: https://gerrit.wikimedia.org/r/#/c/206008/ (duration: 00m 13s) |
[production] |
| 23:17 |
<krenair> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/205889/ (duration: 00m 12s) |
[production] |
| 23:16 |
<krenair> |
Synchronized php-1.26wmf2/extensions/OpenStackManager/nova/OpenStackNovaUser.php: https://gerrit.wikimedia.org/r/#/c/205887/ (duration: 00m 12s) |
[production] |
| 22:55 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.26wmf3 |
[production] |
| 22:52 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.26wmf2 |
[production] |
| 22:47 |
<twentyafterfour> |
Finished scap: testwiki to php-1.26wmf3 and rebuild l10n cache (duration: 37m 11s) |
[production] |
| 22:44 |
<hoo> |
Killed demon's "sudo -u www-data php /srv/mediawiki-staging/multiversion/MWScript.php refreshLinks.php --wiki=ptwiki" on terbium, sending the box into swap |
[production] |
| 22:10 |
<twentyafterfour> |
Started scap: testwiki to php-1.26wmf3 and rebuild l10n cache |
[production] |
| 21:40 |
<thcipriani> |
restarted mariadb on deployment-db{1,2} |
[releng] |
| 21:31 |
<Coren> |
reboot round of deployment-prep done |
[production] |
| 21:05 |
<Coren> |
Starting deployment-prep rolling reboots |
[production] |
| 20:20 |
<thcipriani> |
gzipped /var/log/pacct.0 on deployment-bastion |
[releng] |
| 20:10 |
<twentyafterfour> |
Started scap: testwiki to php-1.26wmf3 and rebuild l10n cache |
[production] |
| 20:08 |
<subbu> |
deployed parsoid version 3311936a |
[production] |
| 19:51 |
<hashar> |
Zuul / Jenkins back up and processing the 1+ hour backlog of changes. Will take a while. Multiple causes: Zuul gearmand being stalled on a socket that has no more data to emit and Jenkins being deadlocked due to an IRC plugin |
[production] |
| 19:50 |
<hashar> |
zuul/jenkins are back up (blame Jenkins) |
[releng] |
| 19:44 |
<hashar> |
Killing Jenkins cause .... we know |
[production] |
| 19:40 |
<hashar> |
reenabling Jenkins gearman client |
[releng] |
| 19:30 |
<hashar> |
Gearman went back. Reenabling Jenkins as a Gearman client |
[releng] |
| 19:27 |
<hashar> |
Zuul gearman is stalled. Disabling Jenkins gearman client to free up connections |
[releng] |
| 19:27 |
<hashar> |
zuul gearman server is stalled |
[production] |
| 17:58 |
<Krinkle> |
Creating integration-slave-trusty-1021 per T96629 (using ci1.medium type) |
[releng] |
| 15:30 |
<gwicke> |
stopped restbase on restbase1002 in preparation for cmjohnson1 checking the hardware |
[production] |
| 15:30 |
<demon> |
Finished scap: 1.26wmf2 was tracking master. should be fixed, being paranoid and doing full sync + i18n rebuild (duration: 08m 11s) |
[production] |
| 15:21 |
<demon> |
Started scap: 1.26wmf2 was tracking master. should be fixed, being paranoid and doing full sync + i18n rebuild |
[production] |
| 15:19 |
<demon> |
Synchronized php-1.26wmf2/extensions/VisualEditor/: (no message) (duration: 00m 12s) |
[production] |
| 15:19 |
<demon> |
Synchronized php-1.26wmf2/extensions/WikiEditor/: (no message) (duration: 00m 11s) |
[production] |
| 15:12 |
<demon> |
Synchronized php-1.26wmf1/extensions/WikiEditor/: (no message) (duration: 00m 13s) |
[production] |
| 14:34 |
<hashar> |
beta: failures on instances are due to them being moved on different openstack compute nodes (virt***) |
[releng] |
| 13:51 |
<jzerebecki> |
integration-slave-trusty-1015:~$ sudo -u jenkins-deploy rm -rf /mnt/jenkins-workspace/workspace/mwext-Wikibase-qunit/src/node_modules |
[releng] |
| 13:37 |
<godog> |
ms-be101[678] weight to 2820 |
[production] |
| 13:25 |
<paravoid> |
switched eqiad<->ulsfo link to Giglinx |
[production] |
| 12:48 |
<andrewbogott> |
migrating to new labvirt nodes |
[releng] |
| 12:48 |
<hashar> |
beta: Andrew B. starting to migrate beta cluster instances on new virt servers |
[releng] |