|
2015-04-23
§
|
| 18:05 |
<andrewbogott> |
rebooting labvirt1006 |
[production] |
| 17:51 |
<kartik> |
Synchronized php-1.26wmf2/extensions/ContentTranslation: (no message) (duration: 00m 15s) |
[production] |
| 17:29 |
<kartik> |
Finished scap: Update ContentTranslation (duration: 26m 11s) |
[production] |
| 17:28 |
<ori> |
scap stuck on snapshot1004; not accepting mwdeploy key |
[production] |
| 17:19 |
<andrewbogott> |
rebooting deployment-parsoidcache02 because it seems troubled |
[releng] |
| 17:03 |
<kartik> |
Started scap: Update ContentTranslation |
[production] |
| 16:53 |
<aaron> |
Synchronized php-1.26wmf2/includes/jobqueue/JobRunner.php: d23777e6832f660984ce4445ab04f98b7ff0d25f (duration: 00m 12s) |
[production] |
| 16:33 |
<andrewbogott> |
rebooting labvirt1005 |
[production] |
| 15:03 |
<manybubbles> |
Synchronized wmf-config/CommonSettings.php: swat: Re-enable Special:SupportedLanguages (duration: 00m 11s) |
[production] |
| 12:29 |
<godog> |
investigating icinga UNKNOWN for hhvm queue/threads |
[production] |
| 09:15 |
<godog> |
restart carbon on graphite1001, replace with carbon-c-relay |
[production] |
| 08:31 |
<godog> |
restart carbon on labmon1001, replace with carbon-c-relay |
[production] |
| 06:11 |
<Krinkle> |
Running git-cache-update inside screen on integration-slave-trusty-1021 at /mnt/git |
[releng] |
| 06:11 |
<Krinkle> |
integration-slave-trusty-1021 stays depooled (see T96629 and T96706) |
[releng] |
| 05:22 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Apr 23 05:21:17 UTC 2015 (duration 21m 16s) |
[production] |
| 04:35 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/206044 and https://gerrit.wikimedia.org/r/206072 |
[releng] |
| 02:49 |
<LocalisationUpdate> |
completed (1.26wmf3) at 2015-04-23 02:48:40+00:00 |
[production] |
| 02:46 |
<l10nupdate> |
Synchronized php-1.26wmf3/cache/l10n: (no message) (duration: 03m 46s) |
[production] |
| 02:28 |
<LocalisationUpdate> |
completed (1.26wmf2) at 2015-04-23 02:27:39+00:00 |
[production] |
| 02:24 |
<l10nupdate> |
Synchronized php-1.26wmf2/cache/l10n: (no message) (duration: 05m 46s) |
[production] |
| 00:29 |
<bd808> |
cherry-picked and applied https://gerrit.wikimedia.org/r/#/c/205969/ (logstash: Convert $::realm switches to hiera) |
[releng] |
| 00:17 |
<bd808> |
beta cluster fatal monitor full of "Bad file descriptor: AH00646: Error writing to /data/project/logs/apache-access.log" |
[releng] |
| 00:15 |
<kaldari> |
Synchronized wmf-config/InitialiseSettings.php: Turning on WikiGrok on English Wikipedia (for 2 week test) (duration: 00m 11s) |
[production] |
| 00:07 |
<krenair> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/206024 (duration: 00m 14s) |
[production] |
| 00:05 |
<krenair> |
Synchronized php-1.26wmf2/extensions/ZeroBanner/includes/ZeroSpecialPage.php: https://gerrit.wikimedia.org/r/#/c/206023/ (duration: 00m 13s) |
[production] |
| 00:03 |
<bd808> |
cleaned up redis leftovers on deployment-logstash1 |
[releng] |
|
2015-04-22
§
|
| 23:57 |
<bd808> |
cherry-picked and applied https://gerrit.wikimedia.org/r/#/c/205968 (remove redis from logstash) |
[releng] |
| 23:47 |
<krenair> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/206015 (duration: 00m 12s) |
[production] |
| 23:44 |
<krenair> |
Synchronized php-1.26wmf3/extensions/ZeroBanner/includes/ZeroSpecialPage.php: https://gerrit.wikimedia.org/r/#/c/206017/ (duration: 00m 13s) |
[production] |
| 23:33 |
<bd808> |
reset deployment-salt:/var/lib/git/operations/puppet HEAD to production; forced update with upstream; re-cherry-picked I46e422825af2cf6f972b64e6d50040220ab08995 |
[releng] |
| 23:29 |
<bd808> |
deployment-salt:/var/lib/git/operations/puppet in detached HEAD state; looks to be for cherry pick of I46e422825af2cf6f972b64e6d50040220ab08995 ? |
[releng] |
| 23:26 |
<krenair> |
Synchronized php-1.26wmf3/extensions/Flow: https://gerrit.wikimedia.org/r/#/c/206008/ (duration: 00m 13s) |
[production] |
| 23:17 |
<krenair> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/205889/ (duration: 00m 12s) |
[production] |
| 23:16 |
<krenair> |
Synchronized php-1.26wmf2/extensions/OpenStackManager/nova/OpenStackNovaUser.php: https://gerrit.wikimedia.org/r/#/c/205887/ (duration: 00m 12s) |
[production] |
| 22:55 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.26wmf3 |
[production] |
| 22:52 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.26wmf2 |
[production] |
| 22:47 |
<twentyafterfour> |
Finished scap: testwiki to php-1.26wmf3 and rebuild l10n cache (duration: 37m 11s) |
[production] |
| 22:44 |
<hoo> |
Killed demon's "sudo -u www-data php /srv/mediawiki-staging/multiversion/MWScript.php refreshLinks.php --wiki=ptwiki" on terbium, sending the box into swap |
[production] |
| 22:10 |
<twentyafterfour> |
Started scap: testwiki to php-1.26wmf3 and rebuild l10n cache |
[production] |
| 21:40 |
<thcipriani> |
restarted mariadb on deployment-db{1,2} |
[releng] |
| 21:31 |
<Coren> |
reboot round of deployment-prep done |
[production] |
| 21:05 |
<Coren> |
Starting deployment-prep rolling reboots |
[production] |
| 20:20 |
<thcipriani> |
gzipped /var/log/pacct.0 on deployment-bastion |
[releng] |
| 20:10 |
<twentyafterfour> |
Started scap: testwiki to php-1.26wmf3 and rebuild l10n cache |
[production] |
| 20:08 |
<subbu> |
deployed parsoid version 3311936a |
[production] |
| 19:51 |
<hashar> |
Zuul / Jenkins back up and processing the 1+ hour backlog of changes. Will take a while. Multiple causes: Zuul gearmand being stalled on a socket that has no more data to emit and Jenkins being deadlocked due to an IRC plugin |
[production] |
| 19:50 |
<hashar> |
zuul/jenkins are back up (blame Jenkins) |
[releng] |
| 19:44 |
<hashar> |
Killing Jenkins cause .... we know |
[production] |
| 19:40 |
<hashar> |
reenabling Jenkins gearman client |
[releng] |
| 19:30 |
<hashar> |
Gearman went back. Reenabling Jenkins as a Gearman client |
[releng] |