|
2015-03-17
§
|
| 21:09 |
<robh> |
updating apche redirects, disabling puppet on mw hosts |
[production] |
| 21:09 |
<bd808> |
Updated scap to include I6301816 (Check for content before <?php) and I61dcf7a (Run rebuildLocalisationCache.php as www-data) |
[production] |
| 21:07 |
<bd808> |
trebuchet checkout errors from mw1104, mw1113, mw1222. No response from mw2003 |
[production] |
| 21:05 |
<bd808> |
mw1222.eqiad.wmnet and mw2003.codfw.wmnet not responding to trebuchet fetch for scap |
[production] |
| 20:04 |
<yuvipanda> |
Synchronized private: (no message) (duration: 00m 06s) |
[production] |
| 20:00 |
<YuviPanda> |
stashing TimStarling’s changes to scap, re-enabling puppet on tin |
[production] |
| 19:22 |
<bblack> |
temporarily disabling puppet on all cp* |
[production] |
| 18:57 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 to 1.25wmf21 |
[production] |
| 18:39 |
<twentyafterfour> |
Deploying 1.25wmf21 to group1 wikis. |
[production] |
| 18:04 |
<_joe_> |
updating libavcodec53 and libavformat53 on the imagescalers and videoscalers |
[production] |
| 17:21 |
<godog> |
repool restbase1006 |
[production] |
| 17:14 |
<legoktm> |
started sendForceRenameNotification.php (CentralAuth/SULF) for all wikis |
[production] |
| 17:07 |
<legoktm> |
migrateAccount.php --auto finished |
[production] |
| 15:50 |
<demon> |
Synchronized php-1.25wmf21/includes/jobqueue/: (no message) (duration: 00m 09s) |
[production] |
| 15:43 |
<demon> |
Synchronized php-1.25wmf20/includes/jobqueue/: (no message) (duration: 00m 07s) |
[production] |
| 15:42 |
<godog> |
restart carbon/relay on graphite1001 |
[production] |
| 15:41 |
<demon> |
Synchronized php-1.25wmf21/extensions/VisualEditor/: (no message) (duration: 00m 06s) |
[production] |
| 15:34 |
<demon> |
Synchronized php-1.25wmf21/extensions/Citoid: (no message) (duration: 00m 07s) |
[production] |
| 15:20 |
<demon> |
Synchronized php-1.25wmf20/includes/specials/SpecialUploadStash.php: (no message) (duration: 00m 07s) |
[production] |
| 15:20 |
<demon> |
Synchronized php-1.25wmf21/includes/specials/SpecialUploadStash.php: (no message) (duration: 00m 07s) |
[production] |
| 15:18 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: restbase off for wikipedias (duration: 00m 06s) |
[production] |
| 15:17 |
<gwicke> |
restarted cassandra on restbase1006 after clearing the data & removing it from its own seeds |
[production] |
| 15:09 |
<demon> |
Synchronized wmf-config/abusefilter.php: (no message) (duration: 00m 07s) |
[production] |
| 15:04 |
<demon> |
Synchronized wmf-config/InitialiseSettings.php: draft namespace for kowiki (duration: 00m 05s) |
[production] |
| 14:59 |
<bblack> |
rebooting cp1072-4, cp3030-49 (none in production) |
[production] |
| 12:57 |
<_joe_> |
updating sudo across all production |
[production] |
| 12:05 |
<_joe_> |
upgraded libicu48 and mediawiki-math-texvc across the cluster |
[production] |
| 11:16 |
<YuviPanda> |
ran chown -R gitpuppet:gitpuppet /var/lib/git/operations/puppet on palladium, fix permission issues |
[production] |
| 11:14 |
<YuviPanda> |
ran chown -R gitpuppet:gitpuppet /var/lib/git/operations/puppet on strontium, fix permission issues |
[production] |
| 11:10 |
<akosiaris> |
chown gitpuppet:gitpuppet /var/lib/git/operations/puppet/.git/logs/refs/remotes/origin/production on strontium, palladium. Somehow it was owned by root |
[production] |
| 10:59 |
<godog> |
depool restbase1006, provisioning |
[production] |
| 07:40 |
<_joe_> |
powercycled mw2027, went down with an unresponsive console |
[production] |
| 07:07 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue Mar 17 07:06:36 UTC 2015 (duration 6m 35s) |
[production] |
| 04:46 |
<ejegg> |
updated tools from 84442d51a841af4265ff103827cda83d5dd9dc54 to 9fd0a885e84074f215082aad689649a0684660f9 |
[production] |
| 02:36 |
<LocalisationUpdate> |
completed (1.25wmf21) at 2015-03-17 02:35:00+00:00 |
[production] |
| 02:34 |
<l10nupdate> |
Synchronized php-1.25wmf21/cache/l10n: (no message) (duration: 00m 04s) |
[production] |
| 02:22 |
<LocalisationUpdate> |
completed (1.25wmf20) at 2015-03-17 02:20:58+00:00 |
[production] |
| 02:20 |
<l10nupdate> |
Synchronized php-1.25wmf20/cache/l10n: (no message) (duration: 00m 03s) |
[production] |
| 02:03 |
<ori> |
applied I98d383a629 locally on mw1017 |
[production] |
| 01:45 |
<ori> |
scap failed: CalledProcessError Command 'cp -r "/tmp/scap_l10n_0.713383032704/*" "/srv/mediawiki-staging/php-1.25wmf20/cache/l10n"' returned non-zero exit status 1 (duration: 00m 10s) |
[production] |
| 01:45 |
<ori> |
Started scap: (no message) |
[production] |
| 01:44 |
<ori> |
scap failed: TypeError temp_dir() takes exactly 2 arguments (1 given) (duration: 00m 10s) |
[production] |
| 01:44 |
<ori> |
Started scap: (no message) |
[production] |
| 01:31 |
<tstarling> |
Finished scap: (no message) (duration: 30m 19s) |
[production] |
| 01:21 |
<ori> |
restarted HHVM on mw1139; bt in https://phabricator.wikimedia.org/P406 |
[production] |
| 01:04 |
<mutante> |
disabled contacts.wikimedia.org - if you are an (unexpected) user please contact me or T90679 |
[production] |
| 01:01 |
<tstarling> |
Started scap: (no message) |
[production] |
| 00:58 |
<ori> |
Increased memory limit on HHVM app servers from 300M to 500M in an attempt to reduce the rate at which T89918 occurs |
[production] |
| 00:57 |
<tstarling> |
scap failed: CalledProcessError Command 'cp -r "/tmp/scap_l10n_2149279197/*" "/srv/mediawiki-staging/php-1.25wmf20/cache/l10n"' returned non-zero exit status 1 (duration: 04m 27s) |
[production] |
| 00:53 |
<tstarling> |
Started scap: (no message) |
[production] |