|
2014-05-14
§
|
| 15:44 |
<chasemp> |
disabling puppet on tungsten to try tweaking carbon settings to affect queue drops (for the better) |
[production] |
| 14:28 |
<cmjohnson1> |
mw1053 going down for disk replacement |
[production] |
| 13:27 |
<bblack> |
restarting pybals on lvs300x |
[production] |
| 12:30 |
<_joe_> |
restarted uwsgi on tungsten |
[production] |
| 09:46 |
<mark> |
Started PyBal on lvs300* and established BGP sessions with the routers |
[production] |
| 09:43 |
<mark> |
Setup BGP configuration for lvs300* on cr1-esams and cr2-knams, with elevated MEDs to keep them as last resorts |
[production] |
| 04:18 |
<Tim> |
deploying apache configuration change with fixes |
[production] |
| 03:11 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed May 14 03:10:36 UTC 2014 (duration 10m 35s) |
[production] |
| 03:01 |
<Tim> |
reverting apache change |
[production] |
| 02:53 |
<Tim> |
deploying apache configuration change https://gerrit.wikimedia.org/r/106109 |
[production] |
| 02:26 |
<LocalisationUpdate> |
completed (1.24wmf4) at 2014-05-14 02:25:08+00:00 |
[production] |
| 02:21 |
<springle> |
upgrade db1043, rebuild as m3 master |
[production] |
| 02:14 |
<LocalisationUpdate> |
completed (1.24wmf3) at 2014-05-14 02:13:23+00:00 |
[production] |
| 00:18 |
<manybubbles> |
bouncing elasticsearch on elastic1015 to pick up gc logging configuration. it might warn but shouldn't cause any service disrubtion. |
[production] |
|
2014-05-13
§
|
| 23:27 |
<maxsem> |
synchronized php-1.24wmf4/includes/exception/MWException.php 'https://gerrit.wikimedia.org/r/#/c/133184/' |
[production] |
| 23:25 |
<maxsem> |
synchronized php-1.24wmf3/includes/exception/MWException.php 'https://gerrit.wikimedia.org/r/#/c/133183/' |
[production] |
| 23:21 |
<MaxSem> |
Ran namespaceDupes after adding new namespaces to zhwikisource - no problems found |
[production] |
| 23:18 |
<maxsem> |
synchronized wmf-config/InitialiseSettings.php 'https://gerrit.wikimedia.org/r/127584' |
[production] |
| 22:43 |
<chasemp> |
ms-be1009 rebooted as it had locked up, swift seems to have recoverd |
[production] |
| 22:35 |
<mutante> |
created new gerrit projects for phabricator,arcanist and libphutil |
[production] |
| 22:23 |
<ori> |
restarting tungsten to verify fix for gdash/graphite initialization |
[production] |
| 21:33 |
<ori> |
gdash and graphite currently down; chase & ori debugging |
[production] |
| 21:29 |
<manybubbles> |
I caused elasticsearch1015 to drop out of the Elasticsearch cluster by tring to take a heap dump on it. don't do that. It stops the application for many seconds. |
[production] |
| 19:07 |
<yurik> |
synchronized php-1.24wmf3/extensions/ZeroRatedMobileAccess/ |
[production] |
| 19:02 |
<yurik> |
synchronized php-1.24wmf4/extensions/ZeroRatedMobileAccess/ |
[production] |
| 18:49 |
<cmjohnson1> |
replacing failed disk dataset1001 |
[production] |
| 18:25 |
<reedy> |
synchronized wmf-config/InitialiseSettings.php 'touch for I1681addaed690b652822c0296b7a3e9b84de93b6' |
[production] |
| 18:22 |
<reedy> |
rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.24wmf4 |
[production] |
| 15:52 |
<anomie> |
synchronized php-1.24wmf4/resources/Resources.php 'SWAT: Deploy jQuery Migrate to 1.24wmf4' |
[production] |
| 15:51 |
<anomie> |
synchronized php-1.24wmf4/resources/lib/jquery/jquery.migrate.js 'SWAT: Deploy jQuery Migrate to 1.24wmf4' |
[production] |
| 04:09 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue May 13 04:07:53 UTC 2014 (duration 7m 52s) |
[production] |
| 03:01 |
<LocalisationUpdate> |
completed (1.24wmf4) at 2014-05-13 03:00:07+00:00 |
[production] |
| 02:31 |
<LocalisationUpdate> |
completed (1.24wmf3) at 2014-05-13 02:30:33+00:00 |
[production] |
|
2014-05-12
§
|
| 23:28 |
<yurik> |
synchronized php-1.24wmf3/extensions/ZeroRatedMobileAccess/ |
[production] |
| 23:25 |
<yurik> |
synchronized php-1.24wmf4/extensions/ZeroRatedMobileAccess/ |
[production] |
| 20:11 |
<gwicke> |
deployed Parsoid d1c778ea3 |
[production] |
| 17:00 |
<mutante> |
re-enabled mw1186 in pybal |
[production] |
| 16:59 |
<manybubbles> |
Finished scap: scapping again to get ms1186 synced up (duration: 00m 56s) |
[production] |
| 16:58 |
<manybubbles> |
Started scap: scapping again to get ms1186 synced up |
[production] |
| 16:49 |
<mutante> |
disabled mw1186 in pybal |
[production] |
| 16:46 |
<mutante> |
mw1186 - down, powercycling |
[production] |
| 16:39 |
<manybubbles> |
Finished scap: fix php symlink (duration: 04m 25s) |
[production] |
| 16:35 |
<manybubbles> |
Started scap: fix php symlink |
[production] |
| 16:09 |
<manybubbles> |
synchronized wmf-config/InitialiseSettings.php |
[production] |
| 16:07 |
<manybubbles> |
Finished scap: update visual editor for swat deploy (duration: 28m 58s) |
[production] |
| 15:38 |
<manybubbles> |
Started scap: update visual editor for swat deploy |
[production] |
| 11:01 |
<springle> |
killed bunch of slow Flow\\Formatter\\ContributionsQuery::queryRevisions queries on flowdb |
[production] |
| 03:14 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Mon May 12 03:13:40 UTC 2014 (duration 13m 39s) |
[production] |
| 02:27 |
<LocalisationUpdate> |
completed (1.24wmf4) at 2014-05-12 02:25:54+00:00 |
[production] |
| 02:15 |
<LocalisationUpdate> |
completed (1.24wmf3) at 2014-05-12 02:14:23+00:00 |
[production] |