|
2020-02-27
§
|
| 11:47 |
<vgutierrez@cumin2001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 11:45 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1084 at 10% T245621', diff saved to https://phabricator.wikimedia.org/P10538 and previous config saved to /var/cache/conftool/dbconfig/20200227-114542-jynus.json |
[production] |
| 11:35 |
<addshore> |
pause item migration script at Q50 million T219123 |
[production] |
| 11:02 |
<vgutierrez> |
start pybal on lvs2003 - T196560 T245984 |
[production] |
| 10:58 |
<vgutierrez> |
stop pybal on lvs2003 to let lvs2010 take the traffic for a little bit - T196560 T245984 |
[production] |
| 10:54 |
<vgutierrez> |
replacing lvs2006 with lvs2010 - T196560 T245984 |
[production] |
| 09:35 |
<jynus> |
upgrade and restart db1084 T246323 |
[production] |
| 09:03 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Depool db1098 (s6 & s7)', diff saved to https://phabricator.wikimedia.org/P10536 and previous config saved to /var/cache/conftool/dbconfig/20200227-090344-jynus.json |
[production] |
| 08:26 |
<jynus> |
killed SpecialFewestRevisions::reallyDoQuery long running query on db1101:s8, causing lag |
[production] |
| 08:14 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Depool db1098 at 50%', diff saved to https://phabricator.wikimedia.org/P10535 and previous config saved to /var/cache/conftool/dbconfig/20200227-081449-jynus.json |
[production] |
| 03:52 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 03:50 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 03:31 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 03:28 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 03:27 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 03:26 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 02:53 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 02:50 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 02:49 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 02:47 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 02:27 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 02:24 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 02:23 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 02:22 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 01:45 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 01:43 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 01:37 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 01:34 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 01:27 |
<XioNoX> |
re-enable BGP to telia in esams |
[production] |
| 01:13 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 01:10 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 00:56 |
<cdanis> |
repool esams 🙌 😎 |
[production] |
| 00:52 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 00:49 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 00:42 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 00:39 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 00:21 |
<jforrester@deploy1001> |
Synchronized w/extract2.php: T239975: Use Article::getPage()->getTouched(), not Article::getTouched (duration: 01m 04s) |
[production] |
| 00:17 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Bonus sync for cache clearance (duration: 01m 04s) |
[production] |
| 00:15 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T232140: Merge definition of wgLogos and wgLogo (duration: 01m 04s) |
[production] |
| 00:13 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T232140: Stop setting wgLogoHD from wgLogos (duration: 01m 05s) |
[production] |
| 00:02 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Bonus sync for cache clearance (duration: 01m 03s) |
[production] |
| 00:01 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T246212 Stop setting wgULSLanguageDetection in IS, set in CS (duration: 01m 05s) |
[production] |
|
2020-02-26
§
|
| 23:59 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T246212 Set wgULSLanguageDetection false in CS (duration: 01m 04s) |
[production] |
| 23:55 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Bonus sync for cache clearance (duration: 01m 04s) |
[production] |
| 23:54 |
<James_F> |
jforrester@deploy1001 Synchronized wmf-config/InitialiseSettings.php: T246193 Stop setting wgAllowTitlesInSVG, never read (and this was default anyway) (duration: 01m 05s) |
[production] |
| 23:19 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 23:16 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 23:16 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 23:15 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:58 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |