3001-3050 of 10000 results (66ms)
2019-02-25 §
08:52 <marostegui> Deploy schema change on s2 on codfw master - lag will happen on s2 codfw - T187295 [production]
08:49 <_joe_> generating mcrouter certificate for mw2151 T192457 [production]
07:09 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1104 after MySQL upgrade (duration: 00m 45s) [production]
06:28 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1104 in API after MySQL upgrade (duration: 00m 45s) [production]
06:13 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1104 after MySQL upgrade (duration: 00m 45s) [production]
06:02 <marostegui> Stop MySQL on db1104 for mysql upgrade [production]
06:02 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1104 for MySQL upgrade (duration: 00m 50s) [production]
2019-02-24 §
21:49 <eileen> civicrm revision changed from 1b5d974569 to d1fc603677, config revision is 00f9c08766 [production]
18:20 <elukey> clean up 2017/2018 log files in /var/log/jmxtrans on kafka1013-22 - root partitions filling up [production]
18:15 <elukey> clean up 2017/2018 log files in /var/log/jmxtrans - root partition almost filled up [production]
10:22 <elukey> force remount of /mnt/hdfs on an-coord1001 (fuse-hdfs stuck) [production]
2019-02-22 §
18:02 <gehel> rolling upgrade on elasticsearch / cirrus / eqiad completed - T215931 [production]
18:00 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
18:00 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
17:33 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.rolling-upgrade (exit_code=0) [production]
17:33 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-upgrade [production]
17:33 <gehel@cumin2001> END (ERROR) - Cookbook sre.elasticsearch.rolling-upgrade (exit_code=97) [production]
17:33 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
17:33 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
17:14 <bblack> cp5007: repooling into service - T216716 [production]
17:13 <bblack> cp5006: repooling into service - T216717 [production]
17:06 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
17:06 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
16:29 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
16:29 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
15:33 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
15:32 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
15:15 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
15:15 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
14:23 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
14:22 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
14:03 <moritzm> removed labvirt1008 from debmonitor (T216661) [production]
14:02 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
14:02 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
13:54 <akosiaris> reboot helium for kernel/microcode updates [production]
13:25 <moritzm> installing wireshark security updates [production]
13:19 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
13:17 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
13:09 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
13:09 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
13:01 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
13:00 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
12:56 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-upgrade [production]
12:48 <gehel@cumin2001> END (ERROR) - Cookbook sre.elasticsearch.rolling-upgrade (exit_code=97) [production]
12:43 <moritzm> rebooting auth1002 for kernel update [production]
12:17 <moritzm> rebooting tungsten to pick up updated microcode to address SSBD/L1TF [production]
12:13 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
12:12 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
12:12 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-upgrade [production]
11:54 <moritzm> various reboots of servers with Westmere-EP CPUs to pick up updated microcode to address SSBD/L1TF [production]