|
2021-10-27
§
|
| 09:25 |
<godog> |
another run of backfill on graphite1004 - T294355 |
[production] |
| 09:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove watchlist replicas from s6 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P17615 and previous config saved to /var/cache/conftool/dbconfig/20211027-092043-marostegui.json |
[production] |
| 09:09 |
<oblivian@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 09:04 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 08:50 |
<topranks> |
Enabling Telxius circuit from cr1-eqiad to asw1-b12-drmrs with homer. |
[production] |
| 07:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Contributions replicas from s6.codfw T263127', diff saved to https://phabricator.wikimedia.org/P17614 and previous config saved to /var/cache/conftool/dbconfig/20211027-074935-marostegui.json |
[production] |
| 07:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove watchlist replicas from s6.codfw T263127', diff saved to https://phabricator.wikimedia.org/P17613 and previous config saved to /var/cache/conftool/dbconfig/20211027-072546-marostegui.json |
[production] |
| 06:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove recentchanges and recentchangeslinked replicas from s6.codfw T263127', diff saved to https://phabricator.wikimedia.org/P17612 and previous config saved to /var/cache/conftool/dbconfig/20211027-060634-marostegui.json |
[production] |
| 05:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove logpager replicas from s6.codfw T263127', diff saved to https://phabricator.wikimedia.org/P17611 and previous config saved to /var/cache/conftool/dbconfig/20211027-053104-marostegui.json |
[production] |
|
2021-10-26
§
|
| 22:59 |
<legoktm> |
uploaded python-logstash to buster-wikimedia for T294393 |
[production] |
| 21:29 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd1021.eqiad.wmnet with OS bullseye |
[production] |
| 21:15 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:13 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudcephosd1021.eqiad.wmnet with OS bullseye |
[production] |
| 21:12 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:04 |
<reedy@deploy1002> |
Synchronized php-1.38.0-wmf.5/tests/phpunit/includes/api/query/ApiQueryImageInfoTest.php: T293783 (duration: 01m 02s) |
[production] |
| 21:03 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:03 |
<reedy@deploy1002> |
Synchronized php-1.38.0-wmf.6/tests/phpunit/includes/api/query/ApiQueryImageInfoTest.php: T293783 (duration: 01m 02s) |
[production] |
| 21:01 |
<reedy@deploy1002> |
Synchronized php-1.38.0-wmf.6/includes/api/ApiQueryImageInfo.php: T293783 (duration: 01m 03s) |
[production] |
| 21:00 |
<reedy@deploy1002> |
Synchronized php-1.38.0-wmf.5/includes/api/ApiQueryImageInfo.php: T293783 (duration: 01m 03s) |
[production] |
| 21:00 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:23 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:15 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 20:00 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:57 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:51 |
<twentyafterfour@deploy1002> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.38.0-wmf.6 refs T293947 |
[production] |
| 19:48 |
<eileen> |
civicrm revision changed from 733a8fceda to dba74c443b, config revision is eed79486d5 |
[production] |
| 19:45 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:40 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:38 |
<twentyafterfour@deploy1002> |
Finished scap: testwikis wikis to 1.38.0-wmf.6 refs T293947 (duration: 25m 28s) |
[production] |
| 19:20 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:17 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:16 |
<cmjohnson@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd1021.eqiad.wmnet with OS bullseye |
[production] |
| 19:12 |
<twentyafterfour@deploy1002> |
Started scap: testwikis wikis to 1.38.0-wmf.6 refs T293947 |
[production] |
| 18:47 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.reimage for host cloudcephosd1021.eqiad.wmnet with OS bullseye |
[production] |
| 17:52 |
<ryankemper@deploy1002> |
Finished deploy [wdqs/wdqs@e908052] (wcqs): Deploy 0.3.90 to WCQS (duration: 01m 34s) |
[production] |
| 17:50 |
<ryankemper@deploy1002> |
Started deploy [wdqs/wdqs@e908052] (wcqs): Deploy 0.3.90 to WCQS |
[production] |
| 17:09 |
<ryankemper@deploy1002> |
Finished deploy [wdqs/wdqs@e908052] (wcqs): Deploy 0.3.90 to WCQS (duration: 02m 37s) |
[production] |
| 17:06 |
<ryankemper@deploy1002> |
Started deploy [wdqs/wdqs@e908052] (wcqs): Deploy 0.3.90 to WCQS |
[production] |
| 17:05 |
<ryankemper@deploy1002> |
Finished deploy [wdqs/wdqs@e908052] (wcqs): Deploy 0.3.90 to WCQS (duration: 1100m 51s) |
[production] |
| 16:25 |
<cdanis@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 16:25 |
<cdanis@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 16:24 |
<mutante> |
[mwmaint1002:~] $ sudo systemctl start mediawiki_job_wikidata_resubmit_changes_for_dispatch |
[production] |
| 16:23 |
<mutante> |
mwmaint1002 - running puppet, created new mw periodic job from gerrit:732972 (T294031) |
[production] |
| 16:07 |
<cmooney@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 16:04 |
<cmooney@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 15:45 |
<lucaswerkmeister-wmde@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 15:41 |
<lucaswerkmeister-wmde@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 15:38 |
<lucaswerkmeister-wmde@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 15:27 |
<cdanis@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 15:27 |
<cdanis@cumin1001> |
START - Cookbook sre.network.cf |
[production] |