801-850 of 10000 results (59ms)
2019-11-28 ยง
13:32 <phamhi@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:31 <marostegui> Remove ar_comment triggers from db1124:3311 for enwiki.archive - T234704 [production]
13:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1106 for schema change, temporarily pool db1080 as vslow,dump', diff saved to https://phabricator.wikimedia.org/P9778 and previous config saved to /var/cache/conftool/dbconfig/20191128-133013-marostegui.json [production]
13:28 <volans> cleanup root's crontab entries on netmon hosts from netbox/postres stuff - T238919 [production]
13:26 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1118 after schema change', diff saved to https://phabricator.wikimedia.org/P9777 and previous config saved to /var/cache/conftool/dbconfig/20191128-132647-marostegui.json [production]
13:21 <volans> cumin 'netmon*' 'rm -v /var/spool/cron/crontabs/postgres' T238919 [production]
13:18 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:15 <effie> enable puppet on thumbor* [production]
13:15 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:11 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:08 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:51 <effie> disable puppet on thumbor* [production]
12:29 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:27 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:23 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:21 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:02 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:00 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:59 <effie> reimage mw1267.eqiad.wmnet mw1277.eqiad.wmnet [production]
11:52 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:50 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:36 <effie> reimage mw1344.eqiad.wmnet mw1334.eqiad.wmnet mw1324.eqiad.wmnet [production]
11:15 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:13 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:55 <effie> reimage mw2279 mw2278 mw2277 mw2276 mw2275 [production]
10:53 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:51 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:45 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
10:43 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:39 <marostegui> Compress labsdb1009 [production]
09:56 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:54 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:51 <godog> swift eqiad-prod: more weight to ms-be105[7-9] - T237438 [production]
09:42 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:40 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:39 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:37 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:19 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:17 <effie> reimage mw1266, mw1276 [production]
09:17 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:56 <marostegui> Compress labsdb1011 [production]
08:42 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:40 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:24 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:22 <jiji@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:19 <marostegui> Remove m4 from tendril and zarcillo - T159170 [production]
08:15 <effie> reimage mw2280, mw2281, mw2282 [production]
08:06 <marostegui> Compress labsdb1012 [production]
07:56 <effie> reimage mw1345, mw1335, mw1325 [production]
06:56 <elukey> remove log files on an-tool1007 to free root partition space [production]