|
2015-11-10
§
|
| 15:32 |
<godog> |
upgrade rsync on ms-be1* per T93587 |
[production] |
| 14:02 |
<godog> |
swift codfw-prod: ms-be2017 / ms-be2019 / ms-be2021 weight 3000 |
[production] |
| 10:00 |
<jynus> |
performing mysql maintenance of db1035 (depooled from production). Replication delay and reboots are expected. |
[production] |
| 09:46 |
<godog> |
swift codfw-prod: ms-be2017 / ms-be2019 / ms-be2021 weight 2000 |
[production] |
| 09:39 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1035 for maintenance (duration: 00m 36s) |
[production] |
| 03:12 |
<ottomata> |
restarted statsv on hafnium just in case it stopped after kafka1013 broker crash and restart (ping ori :) ) |
[production] |
| 03:10 |
<ottomata> |
kafka preferred-replica-election to bring kafka1013 back as leader for its partitions |
[production] |
| 03:00 |
<mutante> |
started kafka on kafka1013 |
[production] |
| 02:55 |
<mutante> |
tried to start kafka on kafka1018 |
[production] |
| 02:31 |
<mutante> |
switched people.wm.org to new backend. please use "rutherfordium.eqiad.wmnet" instead of terbium now. all files have been rsynced already |
[production] |
| 02:22 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.5/cache/l10n: l10nupdate for 1.27.0-wmf.5 (duration: 06m 49s) |
[production] |
| 00:13 |
<hoo> |
Attached Richardelainechambers@commonswiki to the global account of the same name |
[production] |
| 00:06 |
<bd808@tin> |
Synchronized wmf-config/logging.php: Disable microsecond timestamps on all loggers (bf61bfc) (duration: 00m 34s) |
[production] |
|
2015-11-09
§
|
| 23:55 |
<ebernhardson@tin> |
Synchronized wmf-config/InitialiseSettings.php: Disable cirrus writes to labsearch, the machine cant take the load and some jobs are timing out (duration: 00m 34s) |
[production] |
| 23:13 |
<ebernhardson@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable CirrusSearch writes to labsearch for all but enwiki and dewiki (duration: 00m 35s) |
[production] |
| 22:49 |
<ottomata> |
rebooting analytics1032: https://phabricator.wikimedia.org/T118175 |
[production] |
| 21:52 |
<bearND> |
MobileApps deployed sha1 6c63984 |
[production] |
| 19:40 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1060 after performance patch (duration: 00m 35s) |
[production] |
| 18:42 |
<jynus> |
restarting mysql in db1060 to test new performance configuration |
[production] |
| 17:55 |
<bblack> |
upgrading pybal (1.10 -> 1.12) on lvs200[456].codfw.wmnet |
[production] |
| 16:50 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/250454/ (duration: 00m 34s) |
[production] |
| 16:44 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.5/extensions/VisualEditor/modules/ve-mw/init: https://gerrit.wikimedia.org/r/#/c/251972/ (duration: 00m 34s) |
[production] |
| 16:40 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/251674/ (duration: 00m 35s) |
[production] |
| 16:35 |
<krenair@tin> |
Synchronized docroot/wikipedia.org/apple-app-site-association: https://gerrit.wikimedia.org/r/#/c/250897/ (duration: 00m 34s) |
[production] |
| 16:30 |
<matt_flaschen> |
Ran mwscript extensions/Flow/maintenance/FlowUpdateBetaFeaturePreference.php --wiki=wikidatawiki |
[production] |
| 16:24 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/251677/ (duration: 00m 35s) |
[production] |
| 16:14 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.5/extensions/Flow/modules/mw.flow.Initializer.js: https://gerrit.wikimedia.org/r/#/c/251560/ (duration: 00m 44s) |
[production] |
| 16:10 |
<_joe_> |
removing old builds from copper |
[production] |
| 15:45 |
<godog> |
reimage graphite1002 |
[production] |
| 15:06 |
<ottomata> |
running kafka preferred-replica-election |
[production] |
| 15:02 |
<ottomata> |
starting kafka broker on kafka1014 |
[production] |
| 14:56 |
<ottomata> |
stopping kafka broker on kafka1014 |
[production] |
| 14:48 |
<godog> |
swift codfw-prod: ms-be2017 / ms-be2019 / ms-be2021 weight 1000 |
[production] |
| 14:40 |
<ottomata> |
restarting eventlogging to see if it is ok after enabling firewall rules on kafka1014 |
[production] |
| 14:36 |
<_joe_> |
reducing /tmp size on copper by shrinking the logical volume |
[production] |
| 13:16 |
<godog> |
nodetool decomission restbase2001 |
[production] |
| 13:13 |
<mobrovac> |
restbase finished deploying ae2a44f |
[production] |
| 12:59 |
<mobrovac> |
restbase start deploying ae2a44f |
[production] |
| 12:56 |
<hashar> |
restarting Jenkins to refresh the cli-shutdown.groovy script -- https://gerrit.wikimedia.org/r/251935 (https://phabricator.wikimedia.org/T118064) |
[production] |
| 12:10 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depooling db1060 again (duration: 01m 13s) |
[production] |
| 11:16 |
<hashar> |
Jenkins back and apparently happy |
[production] |
| 11:09 |
<hashar> |
Upgrading Jenkins from LTS 1.609.3 to LTS 1.625.1 https://phabricator.wikimedia.org/T118157 |
[production] |
| 10:53 |
<dcausse> |
resuming writes to elasticsearch indices in eqiad (test) |
[production] |
| 10:02 |
<_joe_> |
repooled mw1061 |
[production] |
| 09:29 |
<godog> |
swift codfw-prod: ms-be2016 / ms-be2018 / ms-be2020 weight 3000 |
[production] |
| 09:19 |
<dcausse> |
err (1007 no 1008): restarting elastic on elastic1007.eqiad.wmnet (test) |
[production] |
| 09:18 |
<dcausse> |
restarting elastic on elastic1008.eqiad.wmnet (test) |
[production] |
| 09:10 |
<dcausse> |
freezing elasticsearch indices in eqiad (test) |
[production] |
| 02:21 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.5/cache/l10n: l10nupdate for 1.27.0-wmf.5 (duration: 06m 49s) |
[production] |