|
2019-11-28
ยง
|
| 14:20 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:20 |
<marostegui> |
Deploy schema change on s3 codfw on the master, lag will appear on s3 codfw (T234066) |
[production] |
| 13:57 |
<Amir1> |
start of mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --batch-size 5 (T237984) |
[production] |
| 13:57 |
<marostegui> |
Deploy schema change on s4 codfw master with replication - T234066 |
[production] |
| 13:52 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:50 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:37 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:37 |
<marostegui> |
Deploy schema change on db1106 with replication (lag will appear on s1 on labs) - T234066 T233135 |
[production] |
| 13:37 |
<marostegui> |
Recreate views for enwiki_p.protected_titles for all labsdb hosts - T233135 |
[production] |
| 13:35 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:33 |
<phamhi@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:32 |
<phamhi@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:31 |
<marostegui> |
Remove ar_comment triggers from db1124:3311 for enwiki.archive - T234704 |
[production] |
| 13:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1106 for schema change, temporarily pool db1080 as vslow,dump', diff saved to https://phabricator.wikimedia.org/P9778 and previous config saved to /var/cache/conftool/dbconfig/20191128-133013-marostegui.json |
[production] |
| 13:28 |
<volans> |
cleanup root's crontab entries on netmon hosts from netbox/postres stuff - T238919 |
[production] |
| 13:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1118 after schema change', diff saved to https://phabricator.wikimedia.org/P9777 and previous config saved to /var/cache/conftool/dbconfig/20191128-132647-marostegui.json |
[production] |
| 13:21 |
<volans> |
cumin 'netmon*' 'rm -v /var/spool/cron/crontabs/postgres' T238919 |
[production] |
| 13:18 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:15 |
<effie> |
enable puppet on thumbor* |
[production] |
| 13:15 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:11 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:08 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:51 |
<effie> |
disable puppet on thumbor* |
[production] |
| 12:29 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:27 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:23 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:21 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:02 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:00 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:59 |
<effie> |
reimage mw1267.eqiad.wmnet mw1277.eqiad.wmnet |
[production] |
| 11:52 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:50 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:36 |
<effie> |
reimage mw1344.eqiad.wmnet mw1334.eqiad.wmnet mw1324.eqiad.wmnet |
[production] |
| 11:15 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:13 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 10:55 |
<effie> |
reimage mw2279 mw2278 mw2277 mw2276 mw2275 |
[production] |
| 10:53 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 10:51 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 10:45 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 10:43 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 10:39 |
<marostegui> |
Compress labsdb1009 |
[production] |
| 09:56 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:54 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:51 |
<godog> |
swift eqiad-prod: more weight to ms-be105[7-9] - T237438 |
[production] |
| 09:42 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:40 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:39 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:37 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:19 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:17 |
<effie> |
reimage mw1266, mw1276 |
[production] |