|
2015-11-05
§
|
| 23:34 |
<bd808> |
Decreased replica count of logstash-2015.10.13 and logstash-2015.10.14 to free disk space on cluster |
[production] |
| 23:30 |
<bd808> |
Logging volume into ELK cluster down dramatically; investigating |
[production] |
| 23:16 |
<ori@tin> |
Synchronized wmf-config/logging.php: Ieb8c602a: monolog: Ensure that context data added by WebProcessor is utf-8 safe (duration: 00m 36s) |
[production] |
| 22:35 |
<ori@tin> |
Synchronized php-1.27.0-wmf.5/extensions/WikimediaEvents: Ic99ac31f740956: Log backend response time on edit requests (duration: 00m 35s) |
[production] |
| 21:54 |
<twentyafterfour@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: sync 1.27.0-wmf.5 to group2 |
[production] |
| 20:17 |
<demon@tin> |
Finished scap: no changes, testing permissions on mira co-master (duration: 07m 29s) |
[production] |
| 20:10 |
<demon@tin> |
Started scap: no changes, testing permissions on mira co-master |
[production] |
| 20:07 |
<_joe_> |
chown mwdeploy:wikidev recursively on mira for /srv/mediawiki-staging |
[production] |
| 19:37 |
<twentyafterfour@tin> |
Finished scap: Sync everything just to be sure (duration: 22m 01s) |
[production] |
| 19:15 |
<twentyafterfour@tin> |
Started scap: Sync everything just to be sure |
[production] |
| 18:58 |
<demon@tin> |
Synchronized README: no-op, testing new scap code (duration: 00m 19s) |
[production] |
| 18:52 |
<ostriches> |
scap: deploying master@b44c268 |
[production] |
| 18:05 |
<jynus> |
schema change on x1 - flowdb |
[production] |
| 17:46 |
<ostriches> |
17:45:13 Synchronized wmf-config/: Remove +superprotect, I579c11a2 (duration: 00m 18s) |
[production] |
| 17:44 |
<jynus> |
Deploying schema change on officewiki - flow (s3) |
[production] |
| 17:11 |
<godog> |
nodetool decommission on praseodymium |
[production] |
| 14:12 |
<jynus> |
reinstalling db2056.codfw.wmnet |
[production] |
| 13:57 |
<jynus> |
shuting down mysql and cloning db2042 -> db2062, db2038 -> db2059, db2039 -> db2060, db2040 -> db2061 |
[production] |
| 13:43 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2051; Depool db2042, 38, 39, 40 for cloning (duration: 00m 18s) |
[production] |
| 12:33 |
<_joe_> |
manually disabling mod_cgi on terbium |
[production] |
| 09:56 |
<godog> |
run removenode on cerium.eqiad.wmnet -- decomission was missed before reimaging |
[production] |
| 09:33 |
<jynus> |
stopping mysql and cloning db2034 -> db2062, db2035 -> db2063, db2051 -> db2058 |
[production] |
| 09:25 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Pool db2048, db2049, db2050,db2055, db2057, db2063. Depool db2034, db2035, db2051 (duration: 00m 17s) |
[production] |
| 08:05 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Nov 5 08:05:13 UTC 2015 (duration 5m 12s) |
[production] |
| 03:21 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.5) at 2015-11-05 03:21:45+00:00 |
[production] |
| 03:15 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.5/cache/l10n: l10nupdate for 1.27.0-wmf.5 (duration: 10m 17s) |
[production] |
| 02:47 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.27.0-wmf.4) at 2015-11-05 02:47:20+00:00 |
[production] |
| 02:40 |
<l10nupdate@tin> |
Synchronized php-1.27.0-wmf.4/cache/l10n: l10nupdate for 1.27.0-wmf.4 (duration: 10m 31s) |
[production] |
| 02:17 |
<ori@tin> |
Synchronized wmf-config/CommonSettings.php: I3c397e892e: Only set $wgDisableOutputCompression to 'true' on the scalers (duration: 00m 18s) |
[production] |
| 02:04 |
<Krenair> |
Someone has left tin:/srv/mediawiki-staging/php-1.27.0-wmf.5 in a mess, see `git log origin/wmf/1.27.0-wmf.5..HEAD --oneline`. Note there is one commit waiting to be merged on tin (https://gerrit.wikimedia.org/r/#/c/251168/) that hasn't been yet because of this. |
[production] |
|
2015-11-04
§
|
| 23:56 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.5/maintenance/findOrphanedFiles.php: ced9a62893c (duration: 00m 17s) |
[production] |
| 23:09 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.5/maintenance/findOrphanedFiles.php: 33d378bee6c4ac (duration: 00m 17s) |
[production] |
| 22:33 |
<aaron@tin> |
Synchronized php-1.27.0-wmf.5/maintenance/findOrphanedFiles.php: fb43e95858b9 (duration: 00m 18s) |
[production] |
| 21:17 |
<subbu> |
finished deploying parsoid sha 04893a18 |
[production] |
| 21:05 |
<subbu> |
synced code + restarted parsoid on wtp1002 as a canary |
[production] |
| 21:01 |
<subbu> |
starting parsoid deploy |
[production] |
| 20:13 |
<twentyafterfour@tin> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.5 |
[production] |
| 18:37 |
<mutante> |
correction. ^ that was mw1159 |
[production] |
| 18:35 |
<mutante> |
mw1158: power cycled - crashed, no console output |
[production] |
| 18:23 |
<bd808@tin> |
Synchronized php-1.27.0-wmf.5/includes/debug/logger/MonologSpi.php: MonologSpi: add support for customizing Monolog\Logger instances (59eed42) (duration: 00m 17s) |
[production] |
| 18:23 |
<mutante> |
ms-be2016-2020 - accepted salt keys, hafnium.wm.org - deleted salt key - still unaccepted: capella, haedus, rhodium |
[production] |
| 17:57 |
<mutante> |
ms-be2020 - added to puppet |
[production] |
| 17:54 |
<mutante> |
ms-be2021 - fixing puppet certs after reinstall |
[production] |
| 17:51 |
<papaul> |
ms-be2021 installation complete |
[production] |
| 17:45 |
<papaul> |
reinstalling ms-be2021 |
[production] |
| 17:12 |
<chasemp> |
rebooting cp4007.mgmt.ulsfo.wmnet as it's uresponsive |
[production] |
| 17:11 |
<bblack> |
cp4007 depooled in pybal + confctl |
[production] |
| 17:10 |
<papaul> |
ms-be2020 installation complete |
[production] |
| 16:30 |
<krenair@tin> |
Synchronized php-1.27.0-wmf.5/extensions/VisualEditor: https://gerrit.wikimedia.org/r/#/c/250971/ (duration: 00m 18s) |
[production] |
| 16:29 |
<bd808> |
restarted elasticsearch on logstash1004; it fell out of the cluster during ferm firewall application |
[production] |