|
2015-11-05
§
|
| 17:44 |
<jynus> |
Deploying schema change on officewiki - flow (s3) |
[production] |
| 17:11 |
<godog> |
nodetool decommission on praseodymium |
[production] |
| 14:12 |
<jynus> |
reinstalling db2056.codfw.wmnet |
[production] |
| 13:57 |
<jynus> |
shuting down mysql and cloning db2042 -> db2062, db2038 -> db2059, db2039 -> db2060, db2040 -> db2061 |
[production] |
| 13:43 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2051; Depool db2042, 38, 39, 40 for cloning (duration: 00m 18s) |
[production] |
| 12:33 |
<_joe_> |
manually disabling mod_cgi on terbium |
[production] |
| 09:56 |
<godog> |
run removenode on cerium.eqiad.wmnet -- decomission was missed before reimaging |
[production] |
| 09:33 |
<jynus> |
stopping mysql and cloning db2034 -> db2062, db2035 -> db2063, db2051 -> db2058 |
[production] |
| 09:25 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2048, db2049, db2050,db2055, db2057, db2063. Depool db2034, db2035, db2051 (duration: 00m 17s) |
[production] |
| 08:05 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Nov 5 08:05:13 UTC 2015 (duration 5m 12s) |
[production] |
| 03:21 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.5) at 2015-11-05 03:21:45+00:00 |
[production] |
| 03:15 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.5/cache/l10n: l10nupdate for 1.27.0-wmf.5 (duration: 10m 17s) |
[production] |
| 02:47 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.4) at 2015-11-05 02:47:20+00:00 |
[production] |
| 02:40 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.4/cache/l10n: l10nupdate for 1.27.0-wmf.4 (duration: 10m 31s) |
[production] |
| 02:17 |
<ori@tin> |
Synchronized wmf-config/CommonSettings.php: I3c397e892e: Only set $wgDisableOutputCompression to 'true' on the scalers (duration: 00m 18s) |
[production] |
| 02:04 |
<Krenair> |
Someone has left tin:/srv/mediawiki-staging/php-1.27.0-wmf.5 in a mess, see `git log origin/wmf/1.27.0-wmf.5..HEAD --oneline`. Note there is one commit waiting to be merged on tin (https://gerrit.wikimedia.org/r/#/c/251168/) that hasn't been yet because of this. |
[production] |
|
2015-11-04
§
|
| 23:56 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.5/maintenance/findOrphanedFiles.php: ced9a62893c (duration: 00m 17s) |
[production] |
| 23:09 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.5/maintenance/findOrphanedFiles.php: 33d378bee6c4ac (duration: 00m 17s) |
[production] |
| 22:33 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.5/maintenance/findOrphanedFiles.php: fb43e95858b9 (duration: 00m 18s) |
[production] |
| 21:17 |
<subbu> |
finished deploying parsoid sha 04893a18 |
[production] |
| 21:05 |
<subbu> |
synced code + restarted parsoid on wtp1002 as a canary |
[production] |
| 21:01 |
<subbu> |
starting parsoid deploy |
[production] |
| 20:13 |
<twentyafterfour@tin> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.5 |
[production] |
| 18:37 |
<mutante> |
correction. ^ that was mw1159 |
[production] |
| 18:35 |
<mutante> |
mw1158: power cycled - crashed, no console output |
[production] |
| 18:23 |
<bd808@tin> |
Synchronized php-1.27.0-wmf.5/includes/debug/logger/MonologSpi.php: MonologSpi: add support for customizing Monolog\Logger instances (59eed42) (duration: 00m 17s) |
[production] |
| 18:23 |
<mutante> |
ms-be2016-2020 - accepted salt keys, hafnium.wm.org - deleted salt key - still unaccepted: capella, haedus, rhodium |
[production] |
| 17:57 |
<mutante> |
ms-be2020 - added to puppet |
[production] |
| 17:54 |
<mutante> |
ms-be2021 - fixing puppet certs after reinstall |
[production] |
| 17:51 |
<papaul> |
ms-be2021 installation complete |
[production] |
| 17:45 |
<papaul> |
reinstalling ms-be2021 |
[production] |
| 17:12 |
<chasemp> |
rebooting cp4007.mgmt.ulsfo.wmnet as it's uresponsive |
[production] |
| 17:11 |
<bblack> |
cp4007 depooled in pybal + confctl |
[production] |
| 17:10 |
<papaul> |
ms-be2020 installation complete |
[production] |
| 16:30 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.5/extensions/VisualEditor: https://gerrit.wikimedia.org/r/#/c/250971/ (duration: 00m 18s) |
[production] |
| 16:29 |
<bd808> |
restarted elasticsearch on logstash1004; it fell out of the cluster during ferm firewall application |
[production] |
| 16:28 |
<jynus> |
db2056 clone failed due to wrong partitioning, needs reinstall. Cloning db2049 -> db2063 instead |
[production] |
| 16:01 |
<godog> |
reimage cerium for multi-instance |
[production] |
| 15:22 |
<_joe_> |
moving email batch, flagged reviews, update article count off of terbium |
[production] |
| 15:20 |
<jynus> |
syncing sqldata on db2049 -> db2056, db2050 -> db2057, eta 2-4 hours |
[production] |
| 15:15 |
<_joe_> |
moved translationnotification, updatetranslationstats, pagetriage from terbium to mw1152 |
[production] |
| 14:55 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2049 ans db2050 for maintenance (duration: 00m 17s) |
[production] |
| 14:30 |
<jynus> |
cloning sqldata from db2048 to db2055 |
[production] |
| 14:04 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Depool db2048 for maintenance (duration: 00m 18s) |
[production] |
| 13:03 |
<bblack> |
upgrading pybal (1.10 -> 1.12) on lvs4004.ulsfo.wmnet (inactive low-traffic LVS @ ulsfo) |
[production] |
| 11:59 |
<_joe_> |
moved refreshlink jobs off terbium |
[production] |
| 11:32 |
<_joe_> |
migrating wikidata jobs off of terbium |
[production] |
| 11:07 |
<jynus> |
testing new mariadb-server version on db2056.codfw.wmnet |
[production] |
| 09:25 |
<godog> |
bootstrap cassandra xenon-b.eqiad.wmnet |
[production] |
| 08:47 |
<jynus> |
reduced durability on dbstore1002 and db1047 to improve performance |
[production] |