|
2015-04-03
§
|
| 18:40 |
<ori> |
Restarted nutcracker on HHVM and mw1147 and repooled |
[production] |
| 18:35 |
<ori> |
Depooled mw1147. Spamming fluorine:/a/mw-log/memcache-serious.log. Some nutcracker issue most likely. |
[production] |
| 17:49 |
<ori> |
Synchronized wmf-config/CommonSettings.php: I9c4de264: Send server side eventlogging logs to eventlog1001 instead of vanadium (duration: 00m 11s) |
[production] |
| 15:14 |
<kartik> |
Synchronized php-1.25wmf24/extensions/ContentTranslation: (no message) (duration: 00m 14s) |
[production] |
| 15:14 |
<kartik> |
Synchronized php-1.25wmf23/extensions/ContentTranslation: (no message) (duration: 00m 17s) |
[production] |
| 11:14 |
<springle> |
Synchronized wmf-config/db-eqiad.php: db1049 to normal load (duration: 00m 11s) |
[production] |
| 10:37 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1049, warm up (duration: 00m 12s) |
[production] |
| 10:32 |
<paravoid> |
depooled mw1234 |
[production] |
| 10:20 |
<paravoid> |
staggered restart of the API cluster (sans mw1234, left for further debugging) |
[production] |
| 09:32 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1049 (duration: 00m 20s) |
[production] |
| 09:24 |
<springle> |
mw1114 critical, no ssh, no console, powercycle |
[production] |
| 09:19 |
<springle> |
tin sync-file: mw1114.eqiad.wmnet returned [-15] |
[production] |
| 09:18 |
<springle> |
Synchronized wmf-config/db-eqiad.php: reduce db1049 load (duration: 06m 26s) |
[production] |
| 04:56 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Fri Apr 3 04:55:08 UTC 2015 (duration 55m 7s) |
[production] |
| 03:31 |
<mattflaschen> |
Synchronized php-1.25wmf24/includes/libs/normal/UtfNormalUtil.php: Fix UtfNormal shim so account creations work (duration: 00m 12s) |
[production] |
| 03:29 |
<mattflaschen> |
Synchronized php-1.25wmf24/includes/libs/normal/UtfNormalUtil.php: Fix UtfNormal shim so account creations work (duration: 00m 12s) |
[production] |
| 03:03 |
<LocalisationUpdate> |
completed (1.25wmf24) at 2015-04-03 03:02:30+00:00 |
[production] |
| 02:59 |
<l10nupdate> |
Synchronized php-1.25wmf24/cache/l10n: (no message) (duration: 06m 00s) |
[production] |
| 02:38 |
<LocalisationUpdate> |
completed (1.25wmf23) at 2015-04-03 02:37:21+00:00 |
[production] |
| 02:31 |
<l10nupdate> |
Synchronized php-1.25wmf23/cache/l10n: (no message) (duration: 09m 08s) |
[production] |
| 02:03 |
<YuviPanda> |
restarted hhvm on mw1209 |
[production] |
| 02:02 |
<YuviPanda> |
restarted hhvm on mw1249 and mw1065 |
[production] |
| 01:54 |
<catrope> |
Synchronized w: (no message) (duration: 00m 12s) |
[production] |
| 01:30 |
<ori> |
Synchronized php-1.25wmf24/extensions/ConfirmEdit: 7cb7ef4e6f: Update ConfirmEdit for Id4798364d (duration: 00m 12s) |
[production] |
| 01:11 |
<catrope> |
Synchronized php-1.25wmf24/extensions/ContentTranslation/modules/campaigns/ext.cx.campaigns.contributionsmenu.js: touch (duration: 00m 13s) |
[production] |
| 01:11 |
<catrope> |
Synchronized php-1.25wmf23/extensions/ContentTranslation/modules/campaigns/ext.cx.campaigns.contributionsmenu.js: touch (duration: 00m 12s) |
[production] |
| 01:08 |
<catrope> |
Synchronized php-1.25wmf23/includes: SWAT (duration: 00m 15s) |
[production] |
| 01:06 |
<catrope> |
Synchronized php-1.25wmf23/autoload.php: SWAT (duration: 00m 12s) |
[production] |
| 01:04 |
<catrope> |
Synchronized php-1.25wmf24/includes: SWAT (duration: 00m 15s) |
[production] |
| 01:03 |
<catrope> |
Synchronized php-1.25wmf24/autoload.php: (no message) (duration: 00m 11s) |
[production] |
| 01:01 |
<catrope> |
Synchronized php-1.25wmf24/extensions/Gather: SWAT (duration: 00m 13s) |
[production] |
| 01:00 |
<ori> |
restart HHVM on mw1120 |
[production] |
| 00:48 |
<catrope> |
Synchronized php-1.25wmf24/extensions/VisualEditor: SWAT (duration: 00m 12s) |
[production] |
| 00:47 |
<catrope> |
Synchronized php-1.25wmf24/extensions/Flow: SWAT (duration: 00m 14s) |
[production] |
| 00:47 |
<catrope> |
Synchronized php-1.25wmf24/extensions/ConfirmEdit: SWAT (duration: 00m 13s) |
[production] |
| 00:47 |
<catrope> |
Synchronized php-1.25wmf23/extensions/Flow: SWAT (duration: 00m 12s) |
[production] |
| 00:47 |
<catrope> |
Synchronized php-1.25wmf23/extensions/ConfirmEdit: SWAT (duration: 00m 11s) |
[production] |
| 00:44 |
<catrope> |
Synchronized php-1.25wmf23/extensions/Gather: SWAT (duration: 00m 11s) |
[production] |
|
2015-04-02
§
|
| 23:11 |
<mutante> |
temp. disabling puppet on restbase servers |
[production] |
| 22:50 |
<bd808> |
lots of SYSTEM ERROR responses from nutcracker on mw1147 |
[production] |
| 22:13 |
<greg-g> |
Account creation is broken/not working for either iOS or Android WP apps, investigation in -mobile |
[production] |
| 19:37 |
<ori> |
Synchronized wmf-config/InitialiseSettings.php: I3bbf2418d: Set $wgLogoHD for enwiki (duration: 00m 12s) |
[production] |
| 18:52 |
<mutante> |
running puppet on mw2095 - proxy error |
[production] |
| 18:43 |
<RoanKattouw> |
Running clearMessageBlobs.php |
[production] |
| 18:37 |
<demon> |
Synchronized wmf-config/CirrusSearch-labs.php: cleanups for labs, no-op (duration: 00m 12s) |
[production] |
| 18:37 |
<demon> |
Synchronized wmf-config/CirrusSearch-common.php: turn off "yay new search!!" msg. old news now (duration: 00m 11s) |
[production] |
| 17:15 |
<kartik> |
Synchronized php-1.25wmf23/extensions/ContentTranslation/modules/campaigns/ext.cx.campaigns.contributionsmenu.js: (no message) (duration: 00m 15s) |
[production] |
| 16:52 |
<kartik> |
Finished scap: Update ContentTranslation (duration: 40m 01s) |
[production] |
| 16:25 |
<godog> |
reload uwsgi on graphite1001 |
[production] |
| 16:12 |
<kartik> |
Started scap: Update ContentTranslation |
[production] |