|
2016-04-07
§
|
| 11:57 |
<godog> |
bump raid rebuild to 20MB/s on restbase2003 |
[production] |
| 10:03 |
<_joe_> |
restarted hhvm on mw1114 (one of the child processes was defunct) and on mw1173 (deadlock in HPHP::Treadmill::getAgeOldestRequest) |
[production] |
| 09:35 |
<gehel> |
deploying HHVM HTTP pool sizing on all MW nodes (T131839 / https://gerrit.wikimedia.org/r/#/c/281881), not used yet, no impact expected |
[production] |
| 09:28 |
<elukey> |
de-pooling/re-pooling aqs100[23].eqiad.wmnet for nodejs upgrade |
[production] |
| 09:19 |
<elukey> |
re-added aqs1001.eqiad.wmnet to LVS pool via confctl |
[production] |
| 08:59 |
<elukey> |
removed aqs1001.eqiad.wmnet from LVS pool via confd for nodejs upgrade |
[production] |
| 08:22 |
<godog> |
stop cassandra bootstrap of restbase2004-b, not enough disk |
[production] |
| 08:17 |
<akosiaris> |
renamed duplicate cxserver-deploy and cxserver repos in phabricator to citoid-deploy and citoid respectively |
[production] |
| 07:04 |
<volans@tin> |
Synchronized wmf-config/db-codfw.php: Depool crashed db2047, needs to be reimaged T132011 (duration: 00m 38s) |
[production] |
| 03:08 |
<bd808> |
backfilled missing SAL entries from 2016-04-04T17:30Z to 2016-04-05T20:36Z to https://tools.wmflabs.org/sal/production |
[production] |
| 03:08 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Apr 7 03:08:28 UTC 2016 (duration 9m 52s) |
[production] |
| 02:58 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.20) (duration: 10m 00s) |
[production] |
| 02:38 |
<mwdeploy@tin> |
sync-l10n completed (1.27.0-wmf.19) (duration: 17m 51s) |
[production] |
| 02:09 |
<chasemp> |
reboot db2047 (it's totally stuck w/ BUG: soft lockup - CPU#26 stuck for 22s! [migration/26:267] across) |
[production] |
|
2016-04-06
§
|
| 23:44 |
<mattflaschen@tin> |
Synchronized php-1.27.0-wmf.19/extensions/Flow/includes/Data/Listener/NotificationListener.php: Fix new topic notifications (duration: 00m 29s) |
[production] |
| 23:40 |
<mattflaschen@tin> |
Synchronized php-1.27.0-wmf.20/extensions/Flow/includes/Data/Listener/NotificationListener.php: Fix new topic notifications (duration: 00m 37s) |
[production] |
| 23:11 |
<mattflaschen@tin> |
Synchronized docroot/wikipedia.org/apple-app-site-association: Support handoff and credential sharing with the iOS app (duration: 00m 34s) |
[production] |
| 22:49 |
<bd808> |
Testing stashbot response in #wikimedia-fundraising |
[production] |
| 20:17 |
<subbu> |
finished deploying parsoid sha 5f6c0c60 |
[production] |
| 20:10 |
<subbu> |
synced code; restarted parsoid on wtp1001 as a canary |
[production] |
| 20:06 |
<subbu> |
starting parsoid deploy |
[production] |
| 19:50 |
<dduvall@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: Promote labswiki to 1.27.0-wmf.20 following temporary rollback and fix |
[production] |
| 19:49 |
<dduvall@tin> |
Synchronized php-1.27.0-wmf.20/extensions/SemanticMediaWiki/includes/SMW_ParserExtensions.php: Replace usage of Title::newFromRedirect() (duration: 00m 38s) |
[production] |
| 19:33 |
<dduvall@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: (no message) |
[production] |
| 19:24 |
<akosiaris> |
pool maps-test200{1,2,3} for kartotherian.svc.codfw.wmnet |
[production] |
| 19:22 |
<dduvall@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.20 |
[production] |
| 19:22 |
<akosiaris> |
bounce hhvm on mw1135, mw1145 |
[production] |
| 19:16 |
<akosiaris> |
restart tilerator, tileratorui on maps-test200{1,2,3} |
[production] |
| 19:15 |
<akosiaris> |
restart kartotherian on maps-test200{1,2,3} |
[production] |
| 18:58 |
<YuviPanda> |
reboot notebook1001 |
[production] |
| 18:50 |
<akosiaris> |
disable salt-minion on maps-test200{1,2,3} for maps services deployment. nodejs upgrade is in place |
[production] |
| 17:00 |
<thcipriani@tin> |
Finished scap: SWAT: Add user_wpzero AbuseFilter variable [[gerrit:281867]] (duration: 27m 39s) |
[production] |
| 16:32 |
<thcipriani@tin> |
Started scap: SWAT: Add user_wpzero AbuseFilter variable [[gerrit:281867]] |
[production] |
| 16:06 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Remove Language Overlay experiment [[gerrit:277837]] (duration: 00m 26s) |
[production] |
| 15:54 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable Translate extension on uawikimedia [[gerrit:281403]] (duration: 00m 28s) |
[production] |
| 15:33 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: REVERT Enable Translate extension on uawikimedia [[gerrit:281403]] (duration: 00m 25s) |
[production] |
| 15:31 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable Translate extension on uawikimedia [[gerrit:281403]] (duration: 00m 27s) |
[production] |
| 15:23 |
<urandom> |
Restoring default stream throughput on restbase200{3,4-a}.codfw.wmnet |
[production] |
| 15:20 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: REVERT Bump CirrusSearchRequestSet rev to 121456865906 PART II [[gerrit:280448]] (duration: 00m 31s) |
[production] |
| 15:17 |
<thcipriani@tin> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Bump CirrusSearchRequestSet rev to 121456865906 PART II [[gerrit:280448]] (duration: 00m 30s) |
[production] |
| 15:17 |
<thcipriani@tin> |
Synchronized wmf-config/event-schemas: SWAT: Bump CirrusSearchRequestSet rev to 121456865906 PART I [[gerrit:280448]] (duration: 00m 27s) |
[production] |
| 15:15 |
<ema> |
Upgrading cp* to jessie 8.4 point release and linux 4.4 (T131746, T131928). Not rebooting yet. |
[production] |
| 15:07 |
<thcipriani@tin> |
Synchronized docroot/mediawiki/xml: SWAT: Add Flow dumps schema [[gerrit:281640]] (duration: 00m 28s) |
[production] |
| 15:03 |
<hashar> |
rebased php-1.27.0-wmf.19/MobileFrontend and php-1.27.0-wmf.20/MobileFrontend (single commit related to CI) |
[production] |
| 14:34 |
<filippo@tin> |
Synchronized wmf-config/ProductionServices.php: move mediawiki traffic back to restbase eqiad (duration: 00m 34s) |
[production] |
| 14:26 |
<elukey> |
hhvm restarted on mw1187 |
[production] |
| 14:15 |
<bblack> |
rebooting baham (ns1) for 4.4 kernel + package updates |
[production] |
| 14:05 |
<godog> |
move restbase/citoid/cxserver varnish traffic back to eqiad |
[production] |
| 13:45 |
<ema> |
Upgrading cp1052 to jessie 8.4 point release and linux 4.4 (T131746, T131928) |
[production] |
| 08:28 |
<_joe_> |
disabling puppet on the mw servers to test hhvm changes |
[production] |