|
2016-01-15
§
|
| 21:29 |
<mutante> |
protactinium - shut down, unused system with outdated software |
[production] |
| 21:28 |
<milimetric> |
importing db data to new prod server |
[wikimetrics] |
| 21:28 |
<milimetric> |
set up symlinks for static files on new prod server |
[wikimetrics] |
| 21:27 |
<milimetric> |
stopping old prod queue and scheduler |
[wikimetrics] |
| 21:09 |
<mutante> |
(ganglia for ulsfo will be affected, brb) |
[production] |
| 21:07 |
<mutante> |
bast4001 - reinstalling with jessie |
[production] |
| 18:54 |
<ori> |
disabled gzip in apache for javascript mime types and did an apache config reload |
[production] |
| 18:34 |
<chasemp> |
tools-mail-01 is locked up I am rebooting |
[tools] |
| 18:04 |
<ori@tin> |
ori@tin Synchronized docroot and w: Ie60638b0: Mirror homepage.js from 15.wikipedia.org (duration: 00m 42s) |
[production] |
| 16:01 |
<godog> |
bounce hhvm on mw1129 / mw1204 |
[production] |
| 15:41 |
<godog> |
reimage ms-be3001 with trusty |
[production] |
| 14:54 |
<godog> |
reimage ms-fe3002 with trusty |
[production] |
| 14:13 |
<mark> |
Temporarily paused md126 RAID check on labstore1001 (sync_action idle) |
[production] |
| 14:09 |
<chasemp> |
phab restart phd (reports as not running in phab itself) seems ok now |
[production] |
| 14:03 |
<mark> |
set sync_speed_min to 5000 for md126 on labstore1001 |
[production] |
| 13:28 |
<demon@tin> |
demon@tin Synchronized wmf-config/InitialiseSettings.php: w:he as import source for commonswiki (duration: 00m 49s) |
[production] |
| 12:17 |
<hashar> |
restarting Jenkins for plugins updates |
[production] |
| 12:17 |
<hashar> |
restarting Jenkins for plugins updates |
[releng] |
| 11:07 |
<_joe_> |
re-enabled puppet on mw1013, restarted HHVM to make it pick up our latest changes |
[production] |
| 10:01 |
<moritzm> |
installed ganeti security updates |
[production] |
| 09:18 |
<moritzm> |
installed git security updates on all jessie systems |
[production] |
| 03:10 |
<l10nupdate@tin> |
l10nupdate@tin ResourceLoader cache refresh completed at Fri Jan 15 03:10:09 UTC 2016 (duration 6m 48s) |
[production] |
| 03:03 |
<mwdeploy@tin> |
mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 16m 02s) |
[production] |
| 02:49 |
<bd808> |
Trying to fix submodules in deployment-bastion:/srv/mediawiki-staging/php-master/extensions for T123701 |
[releng] |
| 02:30 |
<krenair@tin> |
krenair@tin Synchronized php-1.27.0-wmf.10/includes/api/ApiQueryRecentChanges.php: https://gerrit.wikimedia.org/r/264231 (duration: 00m 42s) |
[production] |
| 02:29 |
<mwdeploy@tin> |
mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 14m 00s) |
[production] |
| 02:23 |
<YuviPanda> |
pull annualreport git repo on bromine for Krenair |
[production] |
| 02:20 |
<madhuvishy> |
Add rest of analytics team as wikimetrics project admins |
[wikimetrics] |
| 01:09 |
<madhuvishy> |
Found more config path issues for the scheduler. Fixed. All services are running on prod wikimetrics |
[wikimetrics] |
| 01:00 |
<krenair@tin> |
krenair@tin Synchronized php-1.27.0-wmf.10/includes/api/ApiQueryWatchlist.php: https://gerrit.wikimedia.org/r/#/c/264224/ (duration: 00m 31s) |
[production] |
| 00:53 |
<madhuvishy> |
Setup temporary proxy to metrics-prod.wmflabs.org. All good, web and queue are up, scheduler seems to be failing though |
[wikimetrics] |
| 00:49 |
<madhuvishy> |
Deploying to prod wikimetrics, which also restarts all services. So good so far |
[wikimetrics] |
| 00:47 |
<madhuvishy> |
Fixed some things in wikimetrics and wikimetrics-deploy for db creation and migration to work. Prod server initialize stage complete |
[wikimetrics] |
| 00:27 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/263905/ (duration: 00m 32s) |
[production] |
| 00:24 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings.php: touch (duration: 00m 31s) |
[production] |
| 00:22 |
<krenair@tin> |
krenair@tin Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/264091/ (duration: 00m 32s) |
[production] |
| 00:06 |
<mobrovac> |
restbase started a dump of enwiki to populate storage with mobileapps renders |
[production] |
|
2016-01-14
§
|
| 23:56 |
<mobrovac> |
restbase end deploy of dac31a8c |
[production] |
| 23:49 |
<mobrovac> |
restbase start deploy of dac31a8c |
[production] |
| 23:35 |
<madhuvishy> |
Tried setting up prod server - figured mysql-client missing on prod - added to puppet |
[wikimetrics] |
| 23:26 |
<madhuvishy> |
Add role::wikimetrics(prod role), rerun puppet |
[wikimetrics] |
| 23:25 |
<madhuvishy> |
Make new prod instance wikimetrics-01.wikimetrics, force puppet run |
[wikimetrics] |
| 23:11 |
<madhuvishy> |
ran into some path issues - fixed in source - redeploying |
[wikimetrics] |
| 22:47 |
<madhuvishy> |
Last step: fab staging restart_wikimetrics, setup proxy again - and something is wrong |
[wikimetrics] |
| 22:38 |
<madhuvishy> |
Deployment step 2: Ran fab staging deploy - good again |
[wikimetrics] |
| 22:36 |
<madhuvishy> |
Deployment step 1: Ran fab staging initialize_server - all good |
[wikimetrics] |
| 22:35 |
<madhuvishy> |
Recreated wikimetrics-staging, force puppet run |
[wikimetrics] |
| 22:33 |
<madhuvishy> |
Recreated wikimetrics-staging, force puppet run |
[wikimetrics] |
| 22:30 |
<madhuvishy> |
Taking down wikimetrics-staging.wikimetrics to replace with non self hosted instance |
[wikimetrics] |
| 22:29 |
<YuviPanda> |
wikimetrics <whatever> |
[analytics] |