|
2014-11-30
§
|
| 23:17 |
<qchris> |
Updated EventLogging to 19c23698bc03694017d764af33307d6f035fc224 (on 22:51) and restarted it |
[production] |
| 20:51 |
<qchris> |
restarted eventlogging mysql-m2-master consumer. It seems it could no longer write to the database. |
[production] |
| 19:17 |
<Krinkle> |
Disabling and relauching Gearman connection from Jenkins. |
[production] |
| 10:28 |
<oblivian> |
Synchronized wmf-config/jobqueue-eqiad.php: reverting to rdb1001 (duration: 00m 05s) |
[production] |
| 10:13 |
<mark> |
Rebooted asw-c4-eqiad |
[production] |
| 09:13 |
<_joe_> |
jobsqueues work again |
[production] |
| 09:08 |
<oblivian> |
Synchronized wmf-config/jobqueue-eqiad.php: changing the aggregator address as well (duration: 00m 05s) |
[production] |
| 07:27 |
<_joe_> |
restarted the jobrunner service on all jobrunners |
[production] |
| 07:15 |
<oblivian> |
Synchronized wmf-config/jobqueue-eqiad.php: (no message) (duration: 00m 05s) |
[production] |
| 05:50 |
<ori> |
3:50 UTC: switch asw-c-eqiad lost connectivity with cabinet C4. Impact: phabricator down; gap in web request logs and some perf monitoring. Job queue and Recent Changes stream OK b/c redundant servers are up. |
[production] |
| 03:41 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Sun Nov 30 03:41:03 UTC 2014 (duration 41m 2s) |
[production] |
| 02:18 |
<LocalisationUpdate> |
completed (1.25wmf10) at 2014-11-30 02:18:22+00:00 |
[production] |
| 02:18 |
<l10nupdate> |
Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 02s) |
[production] |
| 02:11 |
<LocalisationUpdate> |
completed (1.25wmf9) at 2014-11-30 02:11:01+00:00 |
[production] |
| 02:11 |
<l10nupdate> |
Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
|
2014-11-27
§
|
| 17:55 |
<_joe_> |
restarted hhvm on mw1224, the alarm may have been lost in the puppet failure shower earlier |
[production] |
| 17:24 |
<godog> |
removed /var/lib/carbon/whisper/archived/jenkins from tungsten |
[production] |
| 17:09 |
<godog> |
upload txstatsd 0.7.0~bzr30-0ubuntu0+14 to precise-wikimedia on carbon |
[production] |
| 16:49 |
<godog> |
upload missing txstatsd 1.0.0-1 _source package_ to carbon |
[production] |
| 16:48 |
<godog> |
upload missing txstatsd 1.0.0-1 to carbon |
[production] |
| 15:48 |
<hoo> |
Synchronized php-1.25wmf10/extensions/Wikidata/: Fixing a data model bug + enable Statements on Properties for testwikidata (duration: 00m 12s) |
[production] |
| 15:34 |
<hoo> |
Synchronized wmf-config/Wikibase.php: Set "displayStatementsOnProperties" for wikidata/testwikidata (duration: 00m 06s) |
[production] |
| 14:48 |
<akosiaris> |
upgrading librsvg throughout the fleet |
[production] |
| 04:35 |
<springle> |
restarted squid3 on carbon, but glitches seem to be upstream |
[production] |
| 04:23 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu Nov 27 04:23:45 UTC 2014 (duration 23m 44s) |
[production] |
| 03:45 |
<springle> |
puppet failures everywhere; transient apt timeout |
[production] |
| 02:33 |
<LocalisationUpdate> |
completed (1.25wmf10) at 2014-11-27 02:33:36+00:00 |
[production] |
| 02:33 |
<l10nupdate> |
Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 01s) |
[production] |
| 02:20 |
<LocalisationUpdate> |
completed (1.25wmf9) at 2014-11-27 02:20:52+00:00 |
[production] |
| 02:20 |
<l10nupdate> |
Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 02s) |
[production] |
| 00:30 |
<catrope> |
Synchronized php-1.25wmf10/extensions/VisualEditor: SWAT (duration: 00m 06s) |
[production] |