|
2016-02-18
ยง
|
| 18:04 |
<mobrovac> |
restbase deploy end of a42976cc82 |
[production] |
| 18:03 |
<twentyafterfour> |
applied a hotfix from https://secure.phabricator.com/D15306 on iridium to test a fix for https://phabricator.wikimedia.org/T127290 |
[production] |
| 18:00 |
<godog> |
reenable puppet on restbase1008 |
[production] |
| 17:49 |
<mobrovac> |
restbase deploy start of a42976cc82 |
[production] |
| 17:47 |
<elukey> |
manual failover of hadoop master node (analytics1001) to secondary (analytics1002) for maintenance (plus service restarts) |
[analytics] |
| 17:47 |
<elukey> |
manual failover of hadoop master node (analytics1001) to secondary (analytics1002) for maintenance (plus service restarts) |
[production] |
| 17:41 |
<urandom> |
upgrading Cassandra to 2.1.13 on cerium.eqiad.wmnet (restbase staging) T126629 |
[production] |
| 17:36 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/271555 |
[releng] |
| 17:29 |
<jzerebecki> |
restarted to get the bot back |
[tools.meetbot] |
| 17:28 |
<mobrovac> |
restbase deploying a42976cc82 to restbase1002 |
[production] |
| 17:27 |
<urandom> |
Cassandra on xenon.eqiad.wmnet killed by kernel after Cassandra package upgrade (coincidence?): [1482254.046078] Out of memory: Kill process 21854 (java) score 595 or sacrifice child : T126629 |
[production] |
| 17:26 |
<urandom> |
Cassandra on xenon.eqiad.wmnet killed by kernel after Cassandra package upgrade (coincidence): [1482254.046078] Out of memory: Kill process 21854 (java) score 595 or sacrifice child |
[production] |
| 17:21 |
<urandom> |
upgrading Cassandra to 2.1.13 on xenon.eqiad.wmnet (restbase staging) T126629 |
[production] |
| 17:20 |
<elukey> |
disabled puppet on analytics1027 to avoid any Camus job to run |
[production] |
| 17:19 |
<elukey> |
disabled puppet and camus on analytics1027 |
[analytics] |
| 17:18 |
<bd808> |
Force killed stuck job |
[tools.meetbot] |
| 17:12 |
<bd808> |
Added JanZerebecki as co-maintainer |
[tools.meetbot] |
| 17:04 |
<dcausse> |
updating completion suggester indices in eqiad |
[production] |
| 16:54 |
<elukey> |
restarting hadoop services on analytics105* nodes for security updates |
[analytics] |
| 16:54 |
<elukey> |
restarting hadoop services on analytics105* nodes for security updates |
[production] |
| 16:49 |
<gehel> |
removing cirrus maintenance crons from mw1152 (T127322) |
[production] |
| 16:29 |
<ottomata> |
restarting eventlogging with pykafka 2.2.0 |
[analytics] |
| 16:01 |
<hashar> |
deleting instance integration-slave-precise-1003 think we have enough precise slaves |
[releng] |
| 15:52 |
<dcausse> |
creating adywiki indices in codfw |
[production] |
| 15:44 |
<elukey> |
restarting hadoop services on analytics104* nodes for security updates |
[analytics] |
| 15:44 |
<elukey> |
restarting hadoop services on analytics104* nodes for security updates |
[production] |
| 15:37 |
<elukey> |
restarting hadoop services on analytics102* nodes for security updates |
[analytics] |
| 15:37 |
<elukey> |
restarting hadoop services on analytics102* nodes for security update |
[production] |
| 15:33 |
<moritzm> |
restarting apache on silver/wikitech |
[production] |
| 15:11 |
<elukey> |
restarting hadoop services on analytics103* hosts for security upgrades |
[analytics] |
| 15:10 |
<elukey> |
restarting hadoop services on analytics103* hosts for security upgrades |
[production] |
| 14:06 |
<bblack> |
restarting apache on gallium (integration) |
[production] |
| 13:25 |
<gehel> |
upgrading elasticsearch to 1.7.5 on cirrus-browser-bot |
[search] |
| 13:24 |
<gehel> |
upgrading elasticsearch to 1.7.5 on cirrus-browser-bot |
[deployment-prep] |
| 13:13 |
<mark> |
decreased raid md2 sync_speed_max to 6000 on restbase1008 |
[production] |
| 13:03 |
<elukey> |
executed kafka preferred-replica-election |
[analytics] |
| 12:55 |
<elukey> |
rebooted kafka1022.eqiad.wmnet |
[analytics] |
| 12:55 |
<elukey> |
rebooted kafka1022.eqiad.wmnet for kernel upgrade |
[production] |
| 12:51 |
<godog> |
decrease raid min_speed to 8000 on restbase1008 |
[production] |
| 12:50 |
<hoo@tin> |
Synchronized wmf-config/Wikibase.php: Bump $wgCacheEpoch for Wikidata (duration: 01m 54s) |
[production] |
| 12:46 |
<elukey> |
launched preferred-replica-election |
[analytics] |
| 12:41 |
<elukey> |
kafka1020 rebooted |
[analytics] |
| 12:41 |
<elukey> |
rebooted kafka1020 for kernel upgrade. |
[production] |
| 12:40 |
<godog> |
decrease raid min_speed to 10000 on restbase1008 |
[production] |
| 12:24 |
<godog> |
increase stripe_cache_size to 32470 on restbase1008 |
[production] |
| 12:21 |
<godog> |
expand raid0 on restbase1008 to sdd and sde |
[production] |
| 12:13 |
<elukey> |
launched Kafka preferred-replica-election |
[analytics] |
| 11:36 |
<paravoid> |
upgrading mr1-ulsfo to its pre-recovery version and rebooting (T127295) |
[production] |
| 11:34 |
<hashar> |
Hard restarting Jenkins T127294 |
[production] |
| 11:32 |
<jynus> |
logical import of db1021 starting for data consistency check and defragmenting purposes |
[production] |