|
2016-01-20
§
|
| 07:56 |
<_joe_> |
powercycling mw1162, unable to login from console, memory exhaustion |
[production] |
| 07:24 |
<ebernhardson@tin> |
ebernhardson@tin Synchronized php-1.27.0-wmf.10/extensions/CirrusSearch/includes/DataSender.php: stop checking for frozen indices while codfw elasticsearch recovers (duration: 01m 42s) |
[production] |
| 06:24 |
<ebernhardson> |
codfw elasticsearch cluster stopped responding during load test, idling test to see if it recovers |
[production] |
| 03:44 |
<l10nupdate@tin> |
l10nupdate@tin ResourceLoader cache refresh completed at Wed Jan 20 03:44:48 UTC 2016 (duration 7m 29s) |
[production] |
| 03:37 |
<mwdeploy@tin> |
mwdeploy@tin sync-l10n completed (1.27.0-wmf.11) (duration: 16m 21s) |
[production] |
| 03:02 |
<mwdeploy@tin> |
mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 10m 06s) |
[production] |
| 02:35 |
<mwdeploy@tin> |
mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 11m 20s) |
[production] |
| 01:27 |
<aaron@tin> |
aaron@tin Synchronized wmf-config: Configure $wgCdnReboundPurgeDelay (duration: 00m 32s) |
[production] |
| 01:01 |
<mobrovac> |
restbase deploy end of d621b76 |
[production] |
| 00:57 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264917/ (duration: 00m 32s) |
[production] |
| 00:56 |
<legoktm> |
delete from localuser where lu_name ="Αντώνης Μανιός" and lu_wiki ="mediawikiwiki" limit 1 on centralauth db for T119736 |
[production] |
| 00:53 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264920/ (duration: 00m 33s) |
[production] |
| 00:49 |
<krenair@tin> |
krenair@tin Synchronized php-1.27.0-wmf.10/extensions/MobileFrontend/includes/api/ApiMobileView.php: https://gerrit.wikimedia.org/r/#/c/264973/ (duration: 00m 32s) |
[production] |
| 00:49 |
<mobrovac> |
restbase deploy start of d621b76 |
[production] |
| 00:38 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/264961/ (duration: 00m 31s) |
[production] |
| 00:37 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264961/ (duration: 00m 33s) |
[production] |
| 00:22 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264260/ (duration: 00m 32s) |
[production] |
| 00:21 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/#/c/264260/ (duration: 00m 32s) |
[production] |
| 00:17 |
<krenair@tin> |
krenair@tin Synchronized php-1.27.0-wmf.10/extensions/CirrusSearch: https://gerrit.wikimedia.org/r/#/c/265146/ (duration: 00m 33s) |
[production] |
| 00:10 |
<krenair@tin> |
krenair@tin Synchronized php-1.27.0-wmf.10/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: https://gerrit.wikimedia.org/r/#/c/264989/ (duration: 00m 32s) |
[production] |
|
2016-01-19
§
|
| 23:33 |
<aaron@tin> |
aaron@tin Synchronized wmf-config/CommonSettings.php: Bump $wgJobBackoffThrottling to lower the htmlcacheupdate backlog (duration: 00m 32s) |
[production] |
| 23:22 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/265145 (duration: 02m 24s) |
[production] |
| 23:18 |
<dduvall@tin> |
dduvall@tin rebuilt wikiversions.php and synchronized wikiversions files: group0 to 1.27.0-wmf.11 |
[production] |
| 23:13 |
<dduvall@tin> |
dduvall@tin Finished scap: testwiki to php-1.27.0-wmf.11 and rebuild l10n cache (duration: 72m 03s) |
[production] |
| 22:01 |
<dduvall@tin> |
dduvall@tin Started scap: testwiki to php-1.27.0-wmf.11 and rebuild l10n cache |
[production] |
| 21:35 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/265135 (duration: 00m 32s) |
[production] |
| 21:33 |
<krenair@tin> |
krenair@tin Synchronized dblists/nonglobal.dblist: https://gerrit.wikimedia.org/r/265135 (duration: 03m 21s) |
[production] |
| 21:33 |
<ema> |
Finished migrating mobile traffic to text cluster in codfw (Mexico + green US states on this map https://phabricator.wikimedia.org/T114659) |
[production] |
| 21:14 |
<dduvall@tin> |
dduvall@tin scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="testwiki" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.qyk48j8kem" ' returned non-zero exit status 1 (duration: 16m 11s) |
[production] |
| 20:59 |
<Krenair> |
sync-common on labtestweb2001 |
[production] |
| 20:58 |
<dduvall@tin> |
dduvall@tin Started scap: testwiki to php-1.27.0-wmf.11 and rebuild l10n cache |
[production] |
| 20:48 |
<mutante> |
tin: deleted unused things from /srv/deployment (T120157) |
[production] |
| 20:46 |
<catrope@tin> |
catrope@tin Synchronized wmf-config/InitialiseSettings.php: Disable global AbuseFilters on non-global wikis (duration: 02m 04s) |
[production] |
| 20:25 |
<dduvall@tin> |
dduvall@tin scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="labtestwiki" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.jRNpeW67FO" ' returned non-zero exit status 1 (duration: 01m 31s) |
[production] |
| 20:23 |
<dduvall@tin> |
dduvall@tin Started scap: testwiki to php-1.27.0-wmf.11 and rebuild l10n cache |
[production] |
| 20:13 |
<mutante> |
ruthenium: disable puppet, copy data over to osmium (screen) |
[production] |
| 20:12 |
<mutante> |
ruthenium: service mysql stop |
[production] |
| 19:15 |
<catrope@tin> |
catrope@tin Synchronized wmf-config/CommonSettings.php: EventBus plumbing (duration: 00m 30s) |
[production] |
| 19:14 |
<catrope@tin> |
catrope@tin Synchronized wmf-config/InitialiseSettings.php: Disable Flow on wikitech; add EventBus plumbing (duration: 00m 31s) |
[production] |
| 19:13 |
<catrope@tin> |
catrope@tin Synchronized wmf-config/extension-list: Add EventBus (duration: 00m 31s) |
[production] |
| 19:00 |
<marxarelli> |
starting branch cut for 1.27.0-wmf.11 |
[production] |
| 18:42 |
<ema> |
Starting migration of mobile traffic to text cluster https://phabricator.wikimedia.org/T109286 |
[production] |
| 17:54 |
<krenair@tin> |
krenair@tin Synchronized php-1.27.0-wmf.10/extensions/UploadWizard/UploadWizard.config.php: https://gerrit.wikimedia.org/r/#/c/264969/ (duration: 00m 31s) |
[production] |
| 16:51 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/264964/ (duration: 00m 31s) |
[production] |
| 16:47 |
<krenair@tin> |
krenair@tin Synchronized php-1.27.0-wmf.10/extensions/Graph/modules/graph-loader.js: https://gerrit.wikimedia.org/r/#/c/264715/ (duration: 00m 31s) |
[production] |
| 16:45 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264469/ (duration: 00m 31s) |
[production] |
| 16:41 |
<krenair@tin> |
krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/264437/ (duration: 00m 32s) |
[production] |
| 14:58 |
<cmjohnson1> |
reseating asw-c-eqiad uplink module (xe-1/1/0 and xe-1/1/2) |
[production] |
| 14:29 |
<jynus> |
reimporting some fawiki tables from production into labsdb hosts |
[production] |
| 13:52 |
<godog> |
powercycle ms-be1001 |
[production] |