|
2019-11-28
ยง
|
| 18:57 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:54 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:52 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:50 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:48 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:41 |
<marostegui> |
Deploy schema change on db1134 |
[production] |
| 18:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1134 for schema change', diff saved to https://phabricator.wikimedia.org/P9780 and previous config saved to /var/cache/conftool/dbconfig/20191128-183918-marostegui.json |
[production] |
| 18:29 |
<effie> |
reimage w1319.eqiad.wmnet mw1318.eqiad.wmnet |
[production] |
| 18:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1106 after schema change', diff saved to https://phabricator.wikimedia.org/P9779 and previous config saved to /var/cache/conftool/dbconfig/20191128-180517-marostegui.json |
[production] |
| 17:54 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:52 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:44 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:42 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:19 |
<effie> |
reimage mw1340.eqiad.wmnet mw1339.eqiad.wmnet |
[production] |
| 17:15 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:12 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:34 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:32 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:23 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:21 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:18 |
<phamhi@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:15 |
<phamhi@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:07 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:05 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:58 |
<effie> |
reimage mw1311.eqiad.wmnet |
[production] |
| 15:30 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:28 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:04 |
<effie> |
reimage mw1333.eqiad.wmnet mw1332.eqiad.wmnet mw1331.eqiad.wmnet |
[production] |
| 14:53 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:51 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:29 |
<effie> |
reimage mw1343.eqiad.wmnet mw1342.eqiad.wmnet mw1341.eqiad.wmnet |
[production] |
| 14:22 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:20 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 14:20 |
<marostegui> |
Deploy schema change on s3 codfw on the master, lag will appear on s3 codfw (T234066) |
[production] |
| 13:57 |
<Amir1> |
start of mwscript extensions/Wikibase/repo/maintenance/rebuildPropertyTerms.php --wiki=wikidatawiki --batch-size 5 (T237984) |
[production] |
| 13:57 |
<marostegui> |
Deploy schema change on s4 codfw master with replication - T234066 |
[production] |
| 13:52 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:50 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:37 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:37 |
<marostegui> |
Deploy schema change on db1106 with replication (lag will appear on s1 on labs) - T234066 T233135 |
[production] |
| 13:37 |
<marostegui> |
Recreate views for enwiki_p.protected_titles for all labsdb hosts - T233135 |
[production] |
| 13:35 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:33 |
<phamhi@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:32 |
<phamhi@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:31 |
<marostegui> |
Remove ar_comment triggers from db1124:3311 for enwiki.archive - T234704 |
[production] |
| 13:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1106 for schema change, temporarily pool db1080 as vslow,dump', diff saved to https://phabricator.wikimedia.org/P9778 and previous config saved to /var/cache/conftool/dbconfig/20191128-133013-marostegui.json |
[production] |
| 13:28 |
<volans> |
cleanup root's crontab entries on netmon hosts from netbox/postres stuff - T238919 |
[production] |
| 13:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1118 after schema change', diff saved to https://phabricator.wikimedia.org/P9777 and previous config saved to /var/cache/conftool/dbconfig/20191128-132647-marostegui.json |
[production] |
| 13:21 |
<volans> |
cumin 'netmon*' 'rm -v /var/spool/cron/crontabs/postgres' T238919 |
[production] |
| 13:18 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |