|
2015-10-28
§
|
| 10:39 |
<moritzm> |
updated kernel on restbase1001 to latest 3.19 |
[production] |
| 10:37 |
<_joe_> |
manually removed crontab from mw1152, erroneously created by puppet |
[production] |
| 10:11 |
<_joe_> |
reimaging mw1152 |
[production] |
| 10:01 |
<hashar> |
applying role::cache::parsoid to deployment-cache-parsoid05 |
[releng] |
| 09:58 |
<hashar> |
Deleting deployment-parsoidcache02 (Trusty) 10.68.16.145 to be replaced with deployment-cache-parsoid05 10.68.20.102 (Jessie) |
[releng] |
| 09:54 |
<hashar> |
beta: deleting deployment-cache-parsoid04 not enough disk space for /srv/ ( https://phabricator.wikimedia.org/T103660 ) |
[releng] |
| 09:15 |
<_joe_> |
preparing to reimage mw1152, disabling puppet, scheduling downtime. |
[production] |
| 06:22 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Oct 28 06:22:06 UTC 2015 (duration 22m 5s) |
[production] |
| 05:26 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/249349 |
[releng] |
| 05:15 |
<legoktm> |
ran mwscript updateSpecialPages.php --wiki=testwiki --only=GadgetUsage on terbium |
[production] |
| 03:43 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/249341 ( |
[releng] |
| 03:03 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.4) at 2015-10-28 03:03:34+00:00 |
[production] |
| 03:00 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.4/cache/l10n: l10nupdate for 1.27.0-wmf.4 (duration: 06m 00s) |
[production] |
| 02:39 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.3) at 2015-10-28 02:39:23+00:00 |
[production] |
| 02:34 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.3/cache/l10n: l10nupdate for 1.27.0-wmf.3 (duration: 08m 33s) |
[production] |
| 00:55 |
<yurik> |
deployed graphoid - https://gerrit.wikimedia.org/r/#/c/249324/ |
[production] |
| 00:37 |
<tgr@tin> |
Finished scap: Updating MediaViewer with r246112 (duration: 43m 18s) |
[production] |
| 00:35 |
<mutante> |
mw1135 temp. unresponsive - OOM killer killing hhvm |
[production] |
| 00:27 |
<mutante> |
powercycling unresponsive mw1127 |
[production] |
|
2015-10-27
§
|
| 23:53 |
<tgr@tin> |
Started scap: Updating MediaViewer with r246112 |
[production] |
| 23:13 |
<ori> |
running sync-common on mw2050, mw2128, mw2187 and mw2209 (cf I324134438955c7) |
[production] |
| 23:12 |
<gwicke> |
ran `sudo mdadm --readwrite md1` on restbase1007 to resolve `pending` state |
[production] |
| 23:11 |
<gwicke> |
rebooted restbase1007 to rule out a funky hardware state causing elevated read latencies |
[production] |
| 23:05 |
<krenair@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/248639/ (duration: 00m 17s) |
[production] |
| 23:05 |
<krenair@tin> |
Synchronized wikiversions-labs.json: https://gerrit.wikimedia.org/r/#/c/248639/ (duration: 00m 17s) |
[production] |
| 23:05 |
<krenair@tin> |
Synchronized dblists/all-labs.dblist: https://gerrit.wikimedia.org/r/#/c/248639/ (duration: 00m 18s) |
[production] |
| 23:01 |
<legoktm@tin> |
Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/249306, no-op (duration: 00m 18s) |
[production] |
| 22:39 |
<mutante> |
powercycling unresponsive analytics1039, here's what i saw on mgmt https://phabricator.wikimedia.org/P2248 |
[production] |
| 21:50 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.4/includes/MovePage.php: 7a7c7b27d6c (duration: 00m 17s) |
[production] |
| 21:50 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.3/extensions/Flow/: Fix for cache key bug (duration: 00m 20s) |
[production] |
| 21:29 |
<twentyafterfour@tin> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 wikis to 1.27.0-wmf.4 |
[production] |
| 21:29 |
<ori@tin> |
Synchronized php-1.27.0-wmf.3/includes/debug/logger/LoggerFactory.php: I437bcb532: LoggerFactory: Only check for Psr\Log\LoggerInterface once (duration: 00m 18s) |
[production] |
| 20:42 |
<hashar> |
Nuking Nodepool instances using the previous snapshot ( ci-jessie-wikimedia-1445955240 ) |
[releng] |
| 20:40 |
<hashar> |
Nodepool snapshot ci-jessie-wikimedia-1445977928 generated. Includes /usr/bin/rake ( puppet: https://gerrit.wikimedia.org/r/#/c/249219/ ) |
[releng] |
| 20:31 |
<hashar> |
Generating new Nodepool snapshot ( https://wikitech.wikimedia.org/wiki/Nodepool#Manually_generate_a_new_snapshot ) to have 'rake' included ( puppet: https://gerrit.wikimedia.org/r/#/c/249219/ ) |
[releng] |
| 19:46 |
<twentyafterfour@tin> |
Finished scap: sync everything for 1.27.0-wmf.4 and point testwiki to the new branch (duration: 31m 32s) |
[production] |
| 19:44 |
<bd808> |
Ran sync-common on mw2187 to rebuild l10n caches |
[production] |
| 19:37 |
<YuviPanda> |
upgraded quota for cores from 40 to 56 per milimetric's request |
[analytics] |
| 19:15 |
<twentyafterfour@tin> |
Started scap: sync everything for 1.27.0-wmf.4 and point testwiki to the new branch |
[production] |
| 19:06 |
<ottomata> |
deploying aqs |
[analytics] |
| 18:24 |
<joal> |
deploying refinery |
[analytics] |
| 17:56 |
<paravoid> |
updating jessie debian-installer to 20150422+deb8u2 |
[production] |
| 17:50 |
<Jeff_Green> |
add fundraising-banner-logger hosts to icinga/nsca |
[production] |
| 17:40 |
<krinkle@tin> |
Synchronized php-1.27.0-wmf.3/extensions/NavigationTiming/modules/ext.navigationTiming.js: I95db9deefe363a65 (duration: 00m 17s) |
[production] |
| 17:34 |
<ori@tin> |
Synchronized php-1.27.0-wmf.3/resources/src/mediawiki.ui: I54c195541: Get rid of CSS transitions on form elements in mediawiki.ui (duration: 00m 17s) |
[production] |
| 16:46 |
<joal> |
Releasing refinery-source v0.0.21 |
[analytics] |
| 16:07 |
<Jeff_Green> |
round of fundraising OS updates, occasional icinga noise is expected |
[production] |
| 16:02 |
<thcipriani@tin> |
Synchronized php-1.27.0-wmf.3/resources/src/mediawiki/mediawiki.ForeignStructuredUpload.BookletLayout.js: SWAT: mw.ForeignStructuredUpload: Mark description as being in source wikis content language [[gerrit:249081]] (duration: 00m 17s) |
[production] |
| 15:41 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable VisualEditor in the "Projet" namespace on the French Wikipedia [[gerrit:248910]] (duration: 00m 17s) |
[production] |
| 15:15 |
<godog> |
reenable puppet on graphite1001 |
[production] |