|
2017-11-07
ยง
|
| 19:44 |
<volans> |
restarted apache2 on rhodium (puppet master failing) |
[production] |
| 19:31 |
<herron> |
restart apache2 service on rhodium |
[production] |
| 19:12 |
<demon@tin> |
Finished scap: wmf.7 bootstrap (duration: 48m 15s) |
[production] |
| 19:03 |
<awight> |
begin stress test on ores* |
[production] |
| 18:58 |
<awight@tin> |
Finished deploy [ores/deploy@29905e5]: test deployment to repair ores1009 (non-production) (duration: 00m 31s) |
[production] |
| 18:57 |
<awight@tin> |
Started deploy [ores/deploy@29905e5]: test deployment to repair ores1009 (non-production) |
[production] |
| 18:56 |
<awight@tin> |
Finished deploy [ores/deploy@29905e5]: test deployment to repair ores1008 (non-production) (duration: 00m 32s) |
[production] |
| 18:56 |
<awight@tin> |
Started deploy [ores/deploy@29905e5]: test deployment to repair ores1008 (non-production) |
[production] |
| 18:50 |
<awight@tin> |
Finished deploy [ores/deploy@29905e5]: test deployment to repair ores1002 (non-production) (duration: 00m 33s) |
[production] |
| 18:49 |
<awight@tin> |
Started deploy [ores/deploy@29905e5]: test deployment to repair ores1002 (non-production) |
[production] |
| 18:39 |
<volans> |
slowly running puppet on failed hosts with cumin (concurrency=5) |
[production] |
| 18:38 |
<awight@tin> |
Finished deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) (duration: 00m 20s) |
[production] |
| 18:38 |
<awight@tin> |
Started deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) |
[production] |
| 18:36 |
<_joe_> |
restarting apache2 on rhodium |
[production] |
| 18:24 |
<demon@tin> |
Started scap: wmf.7 bootstrap |
[production] |
| 18:16 |
<awight@tin> |
Finished deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) (duration: 01m 04s) |
[production] |
| 18:15 |
<awight@tin> |
Started deploy [ores/deploy@29905e5]: test deployment to ores* (non-production) |
[production] |
| 18:06 |
<herron> |
restarting puppetdb service on nitrogen |
[production] |
| 17:56 |
<urandom> |
Decommissioning restbase2001-a.codfw.wmnet (T179422) |
[production] |
| 17:55 |
<elukey> |
stop ircecho on einstenium (puppet shower from nitrogen) |
[production] |
| 17:40 |
<ema> |
stop cache_text/upload rolling reboots, resuming tomorrow |
[production] |
| 17:38 |
<urandom> |
Restart Cassandra, restbase1010-{a,b,c}.eqiad.wmnet (T178177) |
[production] |
| 17:33 |
<urandom> |
Clearing Cassandra snapshots (T179422) |
[production] |
| 17:32 |
<moritzm> |
rolling reboot of scb in codfw to pick up new kernel (and openssl updates) |
[production] |
| 17:02 |
<urandom> |
Restarting Cassandra, restbase2001-{a,b,c} to apply OpenJDK upgrade |
[production] |
| 16:47 |
<demon@tin> |
Synchronized multiversion/submodules.json: no op (duration: 00m 47s) |
[production] |
| 16:12 |
<ema> |
start cache_text/upload rolling reboots: upgrading kernel to 4.9.51, libssl to 1.0.2m and 1.1.0g |
[production] |
| 15:46 |
<gehel> |
rolling restart of cassandra maps-test for logging change |
[production] |
| 15:43 |
<godog> |
roll-restart thumbor in eqiad for kernel upgrade |
[production] |
| 15:37 |
<urandom> |
T179420: recreating wiktionary definition schemas |
[production] |
| 15:29 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@eab2948]: revert definition switch, wrong schema - T179420 (duration: 06m 46s) |
[production] |
| 15:22 |
<mobrovac@tin> |
Started deploy [restbase/deploy@eab2948]: revert definition switch, wrong schema - T179420 |
[production] |
| 15:12 |
<_joe_> |
added a runner for htmlCacheUpdate on cewiki too |
[production] |
| 15:10 |
<mobrovac@tin> |
Finished deploy [restbase/deploy@c5dd1e2]: Switch wiktionary definitions to use the next-gen storage - T179420 (duration: 07m 52s) |
[production] |
| 15:02 |
<mobrovac@tin> |
Started deploy [restbase/deploy@c5dd1e2]: Switch wiktionary definitions to use the next-gen storage - T179420 |
[production] |
| 14:52 |
<godog> |
roll-restart thumbor for kernel upgrades |
[production] |
| 14:38 |
<mobrovac> |
restbase creating wiktionary definition schemas for T179420 |
[production] |
| 14:36 |
<godog> |
reboot wezen for kernel upgrade |
[production] |
| 14:33 |
<godog> |
reboot lithium for kernel upgrade |
[production] |
| 14:26 |
<elukey> |
rolling restart of kafka on kafka-jumbo* for jvm security updates |
[production] |
| 14:20 |
<moritzm> |
rebooting mw canaries to 4.9.51 kernel (also picking up openssl/openssl1.1 updates) |
[production] |
| 14:19 |
<moritzm> |
rearming keyholder on naos |
[production] |
| 14:14 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable ShortUrl on pa.wiki - T178919 (duration: 00m 46s) |
[production] |
| 14:11 |
<hashar> |
mwscript extensions/WikimediaMaintenance/createExtensionTables.php --wiki=pawiki ShortUrl - T178919 |
[production] |
| 14:08 |
<moritzm> |
rebooting tureis for kernel update |
[production] |
| 14:07 |
<hashar@tin> |
Synchronized wmf-config/InitialiseSettings.php: add _ in appendix talk namespace at mywiktionary - T179907 (duration: 00m 45s) |
[production] |
| 14:02 |
<moritzm> |
actually holding mw canary reboots until SWAT is over |
[production] |
| 14:01 |
<moritzm> |
rebooting mw canaries to 4.9.51 kernel (also picking up openssl/openssl1.1 updates) |
[production] |
| 13:50 |
<herron> |
rebooted fermium (lists) for kernel update |
[production] |
| 13:46 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Add comment about db1105 current status (duration: 00m 47s) |
[production] |