|
2016-03-10
§
|
| 12:39 |
<mforns> |
restarted eventlogging to fix consumption of server side events |
[analytics] |
| 12:22 |
<hashar> |
Nodeppol Image ci-jessie-wikimedia-1457612269 in wmflabs-eqiad is ready |
[releng] |
| 12:18 |
<hashar> |
Nodepool: rebuilding image to get mathoid/graphoid packages included (hopefully) T119693 T128280 |
[releng] |
| 12:01 |
<_joe_> |
restarted hhvm on mw1122, stuck at startup |
[production] |
| 11:07 |
<mafk> |
Merged https://gerrit.wikimedia.org/r/276206 and https://gerrit.wikimedia.org/r/276433 |
[tools.stewardbots] |
| 10:39 |
<mobrovac> |
restbase deploy end of 26bd4aa28 |
[production] |
| 10:31 |
<jynus> |
setting up the rest of the cross-datacenter master-master connections pending in wmf databases |
[production] |
| 10:09 |
<godog> |
decommissioning restbase1002.eqiad.wmnet : T125842 |
[production] |
| 09:50 |
<volans@tin> |
Synchronized wmf-config/db-codfw.php: Rebalance external storage servers in codfw T127330 (duration: 00m 34s) |
[production] |
| 09:50 |
<mobrovac> |
restbase deploy start of 26bd4aa28 on restbase1001 |
[production] |
| 08:55 |
<ema> |
apache2 and hhvm restarted on mw1107, mw1122 and mw1119 |
[production] |
| 03:08 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Mar 10 03:08:54 UTC 2016 (duration 9m 38s) |
[production] |
| 02:59 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.16) (duration: 12m 33s) |
[production] |
| 02:33 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.15) (duration: 14m 08s) |
[production] |
| 00:45 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.15/extensions/ZeroBanner: SWAT (duration: 00m 28s) |
[production] |
| 00:44 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.15/extensions/MobileFrontend: SWAT (duration: 00m 32s) |
[production] |
| 00:39 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.16/extensions/ZeroBanner: SWAT (duration: 00m 28s) |
[production] |
| 00:39 |
<catrope@tin> |
Synchronized php-1.27.0-wmf.16/extensions/MobileFrontend: SWAT (duration: 00m 36s) |
[production] |
| 00:30 |
<ori> |
Upgrade of HHVM package to 3.12.1+dfsg-1 complete on all eqiad hosts save terbium |
[production] |
| 00:23 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Set NS_PROJECT on azwiktionary (duration: 00m 29s) |
[production] |
| 00:13 |
<catrope@tin> |
Synchronized wmf-config/CommonSettings.php: Remove svnadmins group (duration: 00m 41s) |
[production] |
|
2016-03-09
§
|
| 23:54 |
<bblack> |
ending caches codfw->direct testing |
[production] |
| 23:15 |
<bblack> |
restarted puppetmaster on palladium |
[production] |
| 23:03 |
<bblack> |
caches: starting test codfw->direct (codfw caches -> eqiad apps) |
[production] |
| 22:42 |
<bearND> |
mobileapps deployed 26d4031 |
[production] |
| 22:36 |
<bearND> |
starting mobileapps deploy, second try (without dead link removal patch) |
[production] |
| 21:54 |
<bearND> |
mobileapps deployed 95a2d76 |
[production] |
| 21:35 |
<Luke081515> |
deleted rcm-4 (was unused) |
[rcm] |
| 21:28 |
<ori> |
Depooled mw1107 |
[production] |
| 21:26 |
<bearND> |
starting mobileapps deploy |
[production] |
| 21:22 |
<ori> |
restarted HHVM on mw1107; lock-up |
[production] |
| 21:06 |
<twentyafterfour@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.16 |
[production] |
| 20:57 |
<ori> |
Upgrade HHVM on CODFW app servers to 3.12.1+dfsg-1 |
[production] |
| 20:49 |
<ottomata> |
restarted oozie server |
[analytics] |
| 20:19 |
<urandom> |
decommissioning restbase1001.eqiad.wmnet : T125842 |
[production] |
| 20:11 |
<krinkle@tin> |
Synchronized wmf-config/db-eqiad.php: Clean up - Ibb4bb0b32f5 (duration: 00m 29s) |
[production] |
| 20:11 |
<krinkle@tin> |
Synchronized wmf-config/db-codfw.php: Clean up - Ibb4bb0b32f5 (duration: 00m 35s) |
[production] |
| 19:30 |
<nuria> |
restarted eventlogging after puppet fix: https://gerrit.wikimedia.org/r/#/c/276244/ |
[analytics] |
| 19:21 |
<nuria> |
removing IP encryption from eventlogging. Service restart. |
[analytics] |
| 19:20 |
<nuria> |
deploying new capsule version to eventlogging |
[analytics] |
| 19:01 |
<volans@tin> |
Synchronized wmf-config/db-codfw.php: Change codfw external storage topology T127330 (duration: 00m 27s) |
[production] |
| 18:52 |
<anomie@tin> |
Synchronized php-1.27.0-wmf.16/includes/session/SessionManager.php: Add backtrace to log for MW_NO_SESSION warning mode [[gerrit:276232]] (duration: 00m 50s) |
[production] |
| 18:44 |
<volans> |
Changing topology of local codfw masters for es2 and es3 before merging https://gerrit.wikimedia.org/r/#/c/276229/1 T127330 |
[production] |
| 17:56 |
<bd808> |
Cleaned up git clone state in deployment-tin.deployment-prep:/srv/mediawiki-staging/php-master and queued beta-code-update-eqiad to try again (T129371) |
[releng] |
| 17:53 |
<csteipp> |
deployed patch for T110143 to wmf16 |
[production] |
| 17:48 |
<bd808> |
Git clone at deployment-tin.deployment-prep:/srv/mediawiki-staging/php-master in completely horrible state. Investigating |
[releng] |
| 17:40 |
<csteipp> |
deployed patch for T122056 |
[production] |
| 17:22 |
<bd808> |
Fixed https://integration.wikimedia.org/ci/job/beta-mediawiki-config-update-eqiad/4452/ |
[releng] |
| 17:19 |
<bd808> |
Manually cleaning up broken rebase in deployment-tin.deployment-prep:/srv/mediawiki-staging |
[releng] |
| 16:27 |
<bd808> |
Removed cherry-pick of https://gerrit.wikimedia.org/r/#/c/274696 ; manually cleaned up systemd unit and restarted logstash on deployment-logstash2 |
[releng] |