3451-3500 of 10000 results (92ms)
2019-05-21 ยง
17:18 <reedy@deploy1001> Synchronized php-1.34.0-wmf.6/extensions/Collection/includes/CollectionHooks.php: Fix paths (duration: 00m 56s) [production]
17:17 <bblack> eqiad LVS: high-traffic2 (upload): puppeting lvs1005, basically no-op [production]
17:15 <bblack> eqiad LVS: high-traffic2 (upload): puppeting lvs1002, bringing back pybal in backup role, no traffic shift [production]
17:13 <bblack> eqiad LVS: high-traffic2 (upload): puppeting lvs1014, bringing back pybal in primary role, shifting traffic to lvs1014 [production]
17:11 <bblack> eqiad LVS: high-traffic2 (upload): disable pybal on lvs1014 + lvs1002, shifting traffic to lvs1005 [production]
17:09 <bblack> eqiad LVS: high-traffic1 (text): puppeting lvs1004, basically no-op [production]
17:07 <bblack> eqiad LVS: high-traffic1 (text): puppeting lvs1001, bringing back pybal in backup role, no traffic shift [production]
17:06 <bblack> eqiad LVS: high-traffic1 (text): puppeting lvs1013, bringing back pybal in primary role, shifting traffic to lvs1013 [production]
17:04 <bblack> eqiad LVS: high-traffic1 (text): disable pybal on lvs1013 + lvs1001, shifting traffic to lvs1004 [production]
16:55 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:55 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:55 <jbond42> rebooting wtp1046-1048 [production]
16:55 <bblack> starting Eqiad LVS re-arrangement shortly - T184293 - https://gerrit.wikimedia.org/r/c/operations/puppet/+/511717 (eqiad front edge is still depooled from public traffic) [production]
16:50 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:50 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:50 <jbond42> rebooting wtp1043-1045 [production]
16:46 <mutante> rebooting phab1003 (non-prod) [production]
16:44 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:44 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:44 <jbond42> rebooting wtp1040-1042 [production]
16:39 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:39 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:39 <jbond42> rebooting wtp1037-1039 [production]
16:26 <mobrovac> truncate "others_T_parsoid".data [production]
16:25 <mobrovac> restbase truncate "commons_T_parsoid".data [production]
16:24 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:24 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:24 <jbond42> rebooting wtp1033-1034 [production]
16:18 <mobrovac> restbase truncate "enwiki_T_parsoid".data [production]
16:16 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:16 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:16 <jbond42> rebooting wtp1031-1032 [production]
16:10 <mobrovac> restbase truncate "wikipedia_T_parsoid".data [production]
16:09 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:09 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:09 <jbond42> rebooting wtp1029-2030 [production]
16:01 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:01 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
16:01 <jbond42> rebooting wtp1027-2028 [production]
15:56 <urandom> decommissioning restbase1007-a -- T208087 [production]
15:54 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:54 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:54 <jbond42> rebooting wtp1025-2026 [production]
15:45 <mobrovac@deploy1001> Finished deploy [restbase/deploy@cf00120]: Revert Temporarily copy from old tables to new ones if the data is not found, rb1007 (duration: 02m 43s) [production]
15:42 <mobrovac@deploy1001> Started deploy [restbase/deploy@cf00120]: Revert Temporarily copy from old tables to new ones if the data is not found, rb1007 [production]
15:42 <mobrovac@deploy1001> Finished deploy [restbase/deploy@cf00120]: Revert Temporarily copy from old tables to new ones if the data is not found (duration: 02m 40s) [production]
15:40 <jbond@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:40 <jbond@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:40 <jbond42> rebooting wtp2019-2020 [production]
15:39 <mobrovac@deploy1001> Started deploy [restbase/deploy@cf00120]: Revert Temporarily copy from old tables to new ones if the data is not found [production]