651-700 of 10000 results (55ms)
2019-02-25 §
12:27 <gilles@deploy1001> Finished deploy [3d2png/deploy@ca39432]: (no justification provided) (duration: 00m 05s) [production]
12:27 <gilles@deploy1001> Started deploy [3d2png/deploy@ca39432]: (no justification provided) [production]
12:22 <gilles@deploy1001> Finished deploy [3d2png/deploy@ca39432]: (no justification provided) (duration: 01m 15s) [production]
12:21 <gilles@deploy1001> Started deploy [3d2png/deploy@ca39432]: (no justification provided) [production]
11:49 <moritzm> rolling out intel-microcode 3.20180807a.2 on all jessie/stretch servers, tests on a number of previously unsupported servers with Westmere CPU were successful and I've verified that all other microcode files are identical compared to the current 3.20180807a.1 microcode [production]
11:19 <jijiki> Reimageing thumbor1001 - T214597 [production]
10:40 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:492636| Bumping portals to master (T128546, T202497)]] (duration: 00m 46s) [production]
10:39 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:492636| Bumping portals to master (T128546, T202497)]] (duration: 00m 46s) [production]
10:31 <gtirloni> labstore1004 restarted nfsd and killed stuck rpc.mountd.real processed (T216988) [production]
10:16 <jijiki> Depooling thumbor1001 to reimage - T214597 [production]
09:54 <marostegui> Deploy schema change on db1074, this will generate lag on labsdb:s2 - T187295 [production]
09:07 <marostegui@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Increase ParserCache TTL from 24 days to 30 - T210992 (duration: 00m 46s) [production]
08:52 <marostegui> Deploy schema change on s2 on codfw master - lag will happen on s2 codfw - T187295 [production]
08:49 <_joe_> generating mcrouter certificate for mw2151 T192457 [production]
07:09 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1104 after MySQL upgrade (duration: 00m 45s) [production]
06:28 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1104 in API after MySQL upgrade (duration: 00m 45s) [production]
06:13 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1104 after MySQL upgrade (duration: 00m 45s) [production]
06:02 <marostegui> Stop MySQL on db1104 for mysql upgrade [production]
06:02 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1104 for MySQL upgrade (duration: 00m 50s) [production]
2019-02-24 §
21:49 <eileen> civicrm revision changed from 1b5d974569 to d1fc603677, config revision is 00f9c08766 [production]
18:20 <elukey> clean up 2017/2018 log files in /var/log/jmxtrans on kafka1013-22 - root partitions filling up [production]
18:15 <elukey> clean up 2017/2018 log files in /var/log/jmxtrans - root partition almost filled up [production]
10:22 <elukey> force remount of /mnt/hdfs on an-coord1001 (fuse-hdfs stuck) [production]
2019-02-22 §
18:02 <gehel> rolling upgrade on elasticsearch / cirrus / eqiad completed - T215931 [production]
18:00 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
18:00 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
17:33 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.rolling-upgrade (exit_code=0) [production]
17:33 <gehel@cumin2001> START - Cookbook sre.elasticsearch.rolling-upgrade [production]
17:33 <gehel@cumin2001> END (ERROR) - Cookbook sre.elasticsearch.rolling-upgrade (exit_code=97) [production]
17:33 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
17:33 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
17:14 <bblack> cp5007: repooling into service - T216716 [production]
17:13 <bblack> cp5006: repooling into service - T216717 [production]
17:06 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
17:06 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
16:29 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
16:29 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
15:33 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
15:32 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
15:15 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
15:15 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
14:23 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
14:22 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
14:03 <moritzm> removed labvirt1008 from debmonitor (T216661) [production]
14:02 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
14:02 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
13:54 <akosiaris> reboot helium for kernel/microcode updates [production]
13:25 <moritzm> installing wireshark security updates [production]
13:19 <gehel@cumin2001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
13:17 <gehel@cumin2001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]