|
2016-09-14
§
|
| 17:19 |
<volans> |
reimage mw2198 as it failed before |
[production] |
| 16:32 |
<jynus> |
stopping mysql and shutting down db1082 |
[production] |
| 16:05 |
<elukey> |
restarting cassandra on aqs100[23] T130861 |
[production] |
| 16:01 |
<jynus> |
starting mysql on db1082 |
[production] |
| 15:56 |
<elukey> |
restarting cassandra on aqs1001 T130861 |
[production] |
| 15:54 |
<akosiaris> |
updated cr1-eqiad,cr2-eqiad puppet rules |
[production] |
| 15:50 |
<hoo> |
Ran T132839-Workarounds.sh from my home in terbium (see T132839) |
[production] |
| 15:49 |
<urandom> |
T130861: Performing rolling Cassandra restart, restbase staging |
[production] |
| 15:45 |
<urandom> |
T130861: Restarting Cassandra, xenon.eqiad.wmnet |
[production] |
| 15:41 |
<urandom> |
T130861: Forcing puppet run in restbase staging |
[production] |
| 14:38 |
<mobrovac> |
change-prop deploying ddc091e |
[production] |
| 14:30 |
<gehel> |
increasing delayed allocation to 10m on elasticsearch codfw to speed up cluster restart - T145404 |
[production] |
| 14:26 |
<gehel> |
upgrading elasticsearch codfw to elasticsearch 2.3.5 - T145404 |
[production] |
| 13:33 |
<gehel> |
upgrading logstash to elasticsearch 2.3.5 - T145404 |
[production] |
| 13:20 |
<marostegui> |
renaming tables in s3 codfw - T132837 |
[production] |
| 13:11 |
<hashar@tin> |
Synchronized portals: Bumping portals to master T128546 (duration: 00m 47s) |
[production] |
| 13:10 |
<hashar@tin> |
Synchronized portals/prod/wikipedia.org/assets: Bumping portals to master T128546 (duration: 00m 48s) |
[production] |
| 13:07 |
<akosiaris> |
stop ircecho (icinga-wm) temporarily on neon |
[production] |
| 12:12 |
<akosiaris> |
stop ircecho (icinga-wm) temporarily on neon |
[production] |
| 12:05 |
<_joe_> |
restarting apache on puppetmaster1001 |
[production] |
| 10:31 |
<akosiaris> |
stopped temporarily ircecho (icinga-wm) on neon |
[production] |
| 10:25 |
<ema> |
varnish-be restarted on cp4005 |
[production] |
| 10:03 |
<marostegui> |
alter localuser table in db2054 - T141951 |
[production] |
| 09:47 |
<marostegui> |
Renaming tables before dropping them T54924 |
[production] |
| 08:43 |
<marostegui> |
alter localuser table in db2047 - T141951 |
[production] |
| 07:35 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.19/includes/MediaWiki.php: Use cpPosTime cookie for same-domain redirects on DB change - https://gerrit.wikimedia.org/r/#/c/310494/ (duration: 00m 45s) |
[production] |
| 07:33 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.19/includes/db/ChronologyProtector.php: Use cpPosTime cookie for same-domain redirects on DB change - https://gerrit.wikimedia.org/r/#/c/310494/ (duration: 00m 47s) |
[production] |
| 07:32 |
<legoktm@tin> |
Synchronized php-1.28.0-wmf.19/includes/db/loadbalancer/LBFactory.php: Use cpPosTime cookie for same-domain redirects on DB change - https://gerrit.wikimedia.org/r/#/c/310494/ (duration: 00m 46s) |
[production] |
| 03:22 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Wed Sep 14 03:22:13 UTC 2016 (duration 7m 6s) |
[production] |
| 03:15 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.19) (duration: 18m 08s) |
[production] |
| 02:46 |
<mutante> |
restarted grrrrit-wm |
[production] |
| 02:44 |
<mutante> |
gerrit back to normal |
[production] |
| 02:42 |
<mutante> |
gerrit restarting to apply config changes 256663 and 308885 |
[production] |
| 02:40 |
<mwdeploy@tin> |
scap sync-l10n completed (1.28.0-wmf.18) (duration: 17m 56s) |
[production] |
| 01:56 |
<aaron@tin> |
Synchronized wmf-config: Set $wgAPIMaxLagThreshold => 3 and "max lag" => 6 (duration: 00m 51s) |
[production] |
|
2016-09-13
§
|
| 23:32 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.18/resources/lib/moment/locale: T145382 (duration: 00m 47s) |
[production] |
| 23:19 |
<catrope@tin> |
Synchronized php-1.28.0-wmf.19/resources/lib/moment/locale: T145382 (duration: 00m 49s) |
[production] |
| 22:44 |
<awight> |
update paymentswiki config to da01ae9045af7b8b5f3d34a5f81419fde3bde1d5 |
[production] |
| 22:13 |
<demon@tin> |
Synchronized php-1.28.0-wmf.19/includes/MediaWiki.php: Avoid stupid warnings on url parsing (duration: 00m 47s) |
[production] |
| 21:45 |
<demon@tin> |
Synchronized php-1.28.0-wmf.19/includes/DefaultSettings.php: for lego <3 (duration: 00m 47s) |
[production] |
| 20:46 |
<demon@tin> |
Synchronized php-1.28.0-wmf.19/extensions/Echo: For Roan <3 (duration: 00m 54s) |
[production] |
| 20:23 |
<Krenair> |
restarted designate-api on labtestservices2001, now designate in labtest is working again |
[production] |
| 20:21 |
<Krenair> |
restart rabbitmq-server on labtestcontrol2001 |
[production] |
| 19:01 |
<demon@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: group0 to wmf.19 |
[production] |
| 18:35 |
<gehel> |
starting data import on wdqs200? |
[production] |
| 18:24 |
<demon@tin> |
Finished scap: testwiki to wmf.19 + l10n bootstrap (try 2) (duration: 47m 21s) |
[production] |
| 18:08 |
<gehel> |
moving to scap deployed configuration for wdqs - T144380 |
[production] |
| 17:58 |
<mutante> |
wtp2019 - powercycled, back up without the error, services started |
[production] |
| 17:55 |
<mutante> |
wtp2019 Uncorrectable Memory Error |
[production] |
| 17:55 |
<mutante> |
wtp2019 - down sinc a couple days. console says "Alert! System fatal error during previous boot" |
[production] |