|
2021-11-23
§
|
| 01:05 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:51 |
<urbanecm@deploy1002> |
Started scap: 69aa4a7: 7c0e074: Revert "Create redirect Special Pages for delete and protect action" (T295611; T296203; 4/4) |
[production] |
| 00:50 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.9/autoload.php: 7c0e074: Revert "Create redirect Special Pages for delete and protect action" (T295611; T296203; 3/4) (duration: 00m 55s) |
[production] |
| 00:49 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.9/includes/specials/: 7c0e074: Revert "Create redirect Special Pages for delete and protect action" (T295611; T296203; 2/4) (duration: 00m 55s) |
[production] |
| 00:48 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.9/includes/specialpage/SpecialPageFactory.php: 7c0e074: Revert "Create redirect Special Pages for delete and protect action" (T295611; T296203; 1/4) (duration: 00m 56s) |
[production] |
| 00:41 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: b9209433dfc8b1f81a165ec75867337800db24b1: Enable reading depth instrumentation at low sampling rate (T294777) (duration: 00m 56s) |
[production] |
| 00:39 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:35 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:30 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.9/extensions/WikimediaEvents: 3f860c7: fa9fbf1: WikimediaEvents bbackports (2/2; T294777) (duration: 00m 55s) |
[production] |
| 00:28 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.9/extensions/WikimediaEvents/extension.json: 3f860c72bca817c40486b90f0d8e0ffca72b2690: Restore ReadingDepth instrument (1/2) (duration: 00m 56s) |
[production] |
| 00:25 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:21 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:20 |
<jeena> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/739908 |
[production] |
|
2021-11-22
§
|
| 23:55 |
<mutante> |
acmechief1001, acmechief-test1001: sudo systemctl restart reload-acme-chief-backend.timer |
[production] |
| 23:54 |
<mutante> |
acmechief1001, acmechief-test1001: sudo systemctl start reload-acme-chief-backend.timer |
[production] |
| 23:48 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ms-fe2011.codfw.wmnet with OS stretch |
[production] |
| 23:22 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ms-fe2010.codfw.wmnet with OS stretch |
[production] |
| 23:20 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host ms-fe2011.codfw.wmnet with OS stretch |
[production] |
| 23:18 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-fe2009.codfw.wmnet with OS stretch |
[production] |
| 22:53 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host ms-fe2010.codfw.wmnet with OS stretch |
[production] |
| 22:44 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host ms-fe2009.codfw.wmnet with OS stretch |
[production] |
| 22:20 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti2028.codfw.wmnet with OS buster |
[production] |
| 21:51 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host ganeti2028.codfw.wmnet with OS buster |
[production] |
| 21:42 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti2027.codfw.wmnet with OS buster |
[production] |
| 21:07 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host ganeti2027.codfw.wmnet with OS buster |
[production] |
| 20:43 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:39 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:28 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:24 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:23 |
<legoktm@deploy1002> |
Synchronized wmf-config/PoolCounterSettings.php: Lower CirrusSearch maxqueues to be closer to number of workers (duration: 00m 56s) |
[production] |
| 20:01 |
<ryankemper@cumin1001> |
END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) restart with plugin upgrade (3 nodes at a time) for ElasticSearch cluster search_codfw: codfw plugin upgrade + restart - ryankemper@cumin1001 - T295705 |
[production] |
| 19:49 |
<ryankemper@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-operation restart with plugin upgrade (3 nodes at a time) for ElasticSearch cluster search_codfw: codfw plugin upgrade + restart - ryankemper@cumin1001 - T295705 |
[production] |
| 19:49 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:46 |
<urbanecm> |
Evening B&C window completed |
[production] |
| 19:45 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:44 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.9/extensions/ProofreadPage/: 10b8440069ac71434274462c545c6b2b2c9182d9: Use the WikiEditor ready hook instead of using() the lib (T296033) (duration: 00m 56s) |
[production] |
| 19:34 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:30 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:24 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: b6b05e30b3c9b4007fd31ab0698507d7a48d1caf: kswiki: set wgTranslateNumerals to false (T296055) (duration: 00m 55s) |
[production] |
| 19:20 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:18 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 4aa8d5bf465bfc3fee2ec547718af0c779f88ef4: Enable SandboxLink on lawiki (T296073) (duration: 00m 56s) |
[production] |
| 19:16 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 1c082bec4c74c156b26af4349488835902c5bacd: Enable mapframe on the Indonesian Wikipedia (T295571) (duration: 00m 56s) |
[production] |
| 19:15 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:11 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:05 |
<pt1979@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:01 |
<vgutierrez> |
pool cp4032 (text) using HAProxy as TLS terminator - T290005 |
[production] |
| 18:20 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 18:14 |
<pt1979@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:04 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) restart with plugin upgrade (3 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic plugin upgrade + restart - ryankemper@cumin1001 |
[production] |
| 17:50 |
<ryankemper@cumin1001> |
END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) restart with plugin upgrade (3 nodes at a time) for ElasticSearch cluster search_codfw: codfw plugin upgrade + restart - ryankemper@cumin1001 - T295705 |
[production] |