|
2016-07-14
§
|
| 14:56 |
<bblack> |
cache_misc: manually raised default_ttl to 3600 (to match https://gerrit.wikimedia.org/r/#/c/298970/ without restarts) |
[production] |
| 14:48 |
<aude@tin> |
Started scap: Update i18n for RevisionSlider |
[production] |
| 14:48 |
<jynus> |
shutting down es2011, es2012, es2013 for hardware maintenance T139714 |
[production] |
| 14:47 |
<aude@tin> |
scap aborted: (no message) (duration: 00m 02s) |
[production] |
| 14:47 |
<aude@tin> |
Started scap: (no message) |
[production] |
| 14:45 |
<aude@tin> |
Synchronized wmf-config/CommonSettings.php: Enable RevisionSlider on test wikis (duration: 00m 28s) |
[production] |
| 14:44 |
<aude@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable RevisionSlider on test wikis (duration: 00m 27s) |
[production] |
| 14:38 |
<aude@tin> |
Synchronized wmf-config/extension-list: Add RevisionSlider to extension-list (duration: 00m 42s) |
[production] |
| 09:58 |
<godog> |
powercycle ms-be1012, adding back replaced disk |
[production] |
| 08:40 |
<Amir1> |
for ores |
[production] |
| 08:40 |
<Amir1> |
deploying 0e9555f to scb nodes |
[production] |
| 08:39 |
<elukey> |
restarted hhvm on mw1289 mw1280 mw1288 mw1284 mw1287 |
[production] |
| 08:18 |
<jynus> |
running "megacli -PDOffline -PhysDrv '[32:6]' -aALL" on dbstore1002 to debug issue T140337 |
[production] |
| 08:06 |
<elukey> |
upgrading cache misc to varnishkafka 1.0.11-1 |
[production] |
| 08:03 |
<_joe_> |
removing appservers mw1018-25 from service via conftool for decommissioning (T139353) |
[production] |
| 08:01 |
<elukey> |
removing api servers mw111[4-9] from service via conftool as first decom step (T139353) |
[production] |
| 07:55 |
<elukey> |
removing api servers mw112[0-9] from service via conftool as first decom step (T139353) |
[production] |
| 06:45 |
<moritzm> |
restarted hhvm on mw1170 |
[production] |
| 06:07 |
<moritzm> |
upgrading hhvm in codfw |
[production] |
| 04:22 |
<twentyafterfour> |
Phabricator hotfix: applied patch to disable task graph on tasks with > 100 related tasks. |
[production] |
| 03:14 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Jul 14 03:14:57 UTC 2016 (duration 7m 16s) |
[production] |
| 03:07 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.10) (duration: 15m 50s) |
[production] |
| 02:37 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.8) (duration: 15m 17s) |
[production] |
| 00:36 |
<matt_flaschen> |
Ran backfillReadBundles on labtestwiki |
[production] |
| 00:35 |
<matt_flaschen> |
Started backfillReadBundles on labswiki |
[production] |
| 00:24 |
<matt_flaschen> |
Started backfillUnreadWikis --rebuild and backfillReadBundles for all group 0 and group 1 wikis earlier |
[production] |
| 00:06 |
<twentyafterfour> |
Phabricator maintenance completed. Service restored |
[production] |
| 00:01 |
<twentyafterfour> |
preparing to take Phabricator offline momentarily for scheduled maintenance / upgrade. Service should be restored within a couple of minutes. |
[production] |
|
2016-07-13
§
|
| 23:31 |
<eileen> |
update CiviCRM from 0898bb9360fe4a5ddea1a41d4e3f3e9823afee27 to cea316cc57c511c645a92a003028c95e19cac877 |
[production] |
| 23:27 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/298822 (duration: 00m 26s) |
[production] |
| 23:25 |
<krenair@tin> |
Synchronized static/images/project-logos: update for https://gerrit.wikimedia.org/r/298819 and https://gerrit.wikimedia.org/r/298822 (duration: 00m 24s) |
[production] |
| 23:17 |
<krenair@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/225509 + https://gerrit.wikimedia.org/r/298899 - create https://meta.wikimedia.org/wiki/Special:Contact/stewards (duration: 00m 26s) |
[production] |
| 23:17 |
<ori@tin> |
Synchronized php-1.28.0-wmf.10/includes/resourceloader/ResourceLoaderStartUpModule.php: I882bf7075: ResourceLoader: Update expected length of module version hash (duration: 00m 25s) |
[production] |
| 21:55 |
<mutante> |
ytterbium - puppet enabled again, fix deployed |
[production] |
| 21:48 |
<mutante> |
ytterbium, disabled puppet, started apache, needs fix |
[production] |
| 20:40 |
<demon@tin> |
Synchronized README: no-op to bring co-masters in sync (duration: 00m 28s) |
[production] |
| 20:27 |
<bearND> |
deployed mobileapps d1eb1da |
[production] |
| 20:20 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 to wmf.10 |
[production] |
| 20:19 |
<bearND> |
starting mobileapps deploy |
[production] |
| 20:03 |
<ebernhard|lunch> |
Update codfw elasticsearch cluster sttings with cluster.routing.allocation.disk.watermark.low: 70% to match eqiad and reduce free space icinga warnings |
[production] |
| 19:28 |
<ostriches> |
gerrit: restarting, puppet back on, issue fixed. |
[production] |
| 19:23 |
<anomie@tin> |
Synchronized php-1.28.0-wmf.10/includes/auth/AuthManager.php: Add timing data logging for T119736 (duration: 00m 28s) |
[production] |
| 19:23 |
<anomie@tin> |
Synchronized php-1.28.0-wmf.8/includes/auth/AuthManager.php: Add timing data logging for T119736 (duration: 00m 27s) |
[production] |
| 19:14 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.8/includes/api/ApiQueryRecentChanges.php: API: Remove index forcing in ApiQueryRecentChanges - T140108 (duration: 00m 26s) |
[production] |
| 19:04 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all group0 to wmf.10 |
[production] |
| 18:41 |
<ostriches> |
gerrit/ytterbium: flapped for a minute because of incompat 2.12/2.8 config. Working, puppet disabled pending real fix. |
[production] |
| 18:40 |
<mutante> |
gerrit has a temp problem. maintenance going on |
[production] |
| 18:34 |
<krenair@tin> |
Synchronized wmf-config: labs-only change, should be a noop here: https://gerrit.wikimedia.org/r/298812 (duration: 00m 27s) |
[production] |
| 18:32 |
<mutante> |
gerrit will restart shortly for a config change. expect a very short downtime |
[production] |
| 18:22 |
<legoktm> |
checkLocalUser.php finished, starting run #2 now |
[production] |