|
2020-06-29
ยง
|
| 15:43 |
<akosiaris@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:37 |
<akosiaris@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:37 |
<akosiaris@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1105:3312', diff saved to https://phabricator.wikimedia.org/P11696 and previous config saved to /var/cache/conftool/dbconfig/20200629-153140-marostegui.json |
[production] |
| 15:20 |
<gehel> |
repool wdqs1004 - catched up on lag |
[production] |
| 14:50 |
<hnowlan@deploy1001> |
Finished deploy [restbase/deploy@900bcf6]: Redeploy to fix transient error in gom wiktionary deploy (duration: 00m 06s) |
[production] |
| 14:50 |
<hnowlan@deploy1001> |
Started deploy [restbase/deploy@900bcf6]: Redeploy to fix transient error in gom wiktionary deploy |
[production] |
| 14:48 |
<hnowlan@deploy1001> |
Finished deploy [restbase/deploy@900bcf6]: Enable gom wiktionary (duration: 13m 40s) |
[production] |
| 14:34 |
<hnowlan@deploy1001> |
Started deploy [restbase/deploy@900bcf6]: Enable gom wiktionary |
[production] |
| 14:33 |
<hnowlan@deploy1001> |
Finished deploy [restbase/deploy@900bcf6]: Enable gom wiktionary (duration: 17m 49s) |
[production] |
| 14:28 |
<ema> |
A:cp rolling purged upgrade to 0.16 T256479 |
[production] |
| 14:24 |
<arturo> |
starting rabbitmq-server in all 3 cloudcontrol servers |
[admin] |
| 14:23 |
<arturo> |
stopping rabbitmq-server in all 3 cloudcontrol servers |
[admin] |
| 14:22 |
<lucaswerkmeister-wmde@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:608309|Add "E" as an alias of EntitySchema namespace on wikidata (T245529)]] (duration: 00m 57s) |
[production] |
| 14:20 |
<ema> |
upload purged 0.16 to apt.wm.org T256479 |
[production] |
| 14:16 |
<hnowlan@deploy1001> |
Started deploy [restbase/deploy@900bcf6]: Enable gom wiktionary |
[production] |
| 14:14 |
<hnowlan@deploy1001> |
Finished deploy [restbase/deploy@ce5177e]: Enable gom wiktionary (duration: 20m 44s) |
[production] |
| 14:02 |
<jforrester@deploy1001> |
Synchronized multiversion/MWConfigCacheGenerator.php: Fix 'closed-labs' reading as 'closed' for static config (duration: 00m 56s) |
[production] |
| 13:54 |
<jforrester@deploy1001> |
Synchronized dblists/: Drop nonbetafeatures dblist, unused (duration: 00m 57s) |
[production] |
| 13:54 |
<hnowlan@deploy1001> |
Started deploy [restbase/deploy@ce5177e]: Enable gom wiktionary |
[production] |
| 13:50 |
<jforrester@deploy1001> |
Synchronized multiversion/MWConfigCacheGenerator.php: Drop 'nonbetafeatures' dblist from production reads (duration: 00m 56s) |
[production] |
| 13:49 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Switch uses from nonbetafeatures to lockeddown (duration: 00m 57s) |
[production] |
| 13:47 |
<jforrester@deploy1001> |
Synchronized multiversion/MWConfigCacheGenerator.php: Add 'lockeddown' dblist to production reads (duration: 00m 57s) |
[production] |
| 13:43 |
<jforrester@deploy1001> |
Synchronized dblists/lockeddown.dblist: Add lockddown dblist (unused as yet) (duration: 00m 59s) |
[production] |
| 13:35 |
<vgutierrez> |
depool cp3053 due to nvme hardware issues |
[production] |
| 13:09 |
<elukey> |
archiva.wikimedia.org migrated to archiva1002 |
[analytics] |
| 13:02 |
<XioNoX> |
test pfw3-codfw uplinks failover |
[production] |
| 13:00 |
<elukey> |
move archiva.wikimedia.org to archiva1002 (new buster vm); create archiva-old.wikimedia.org to archiva1001 |
[production] |
| 12:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1105:3312', diff saved to https://phabricator.wikimedia.org/P11693 and previous config saved to /var/cache/conftool/dbconfig/20200629-125824-marostegui.json |
[production] |
| 12:56 |
<dsaez> |
test |
[wmf-research-tools] |
| 12:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1085', diff saved to https://phabricator.wikimedia.org/P11692 and previous config saved to /var/cache/conftool/dbconfig/20200629-125630-marostegui.json |
[production] |
| 12:48 |
<hashar> |
Updating Jenkins jobs for releng/ci-src-setup-simple:0.3.0 # https://gerrit.wikimedia.org/r/c/integration/config/+/608069 | T256462 |
[releng] |
| 12:41 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:38 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:32 |
<jayme> |
deleted all tags for docker-registry.wikimedia.org/envoy-tls-local-proxy from docker registry - T253396 |
[production] |
| 12:20 |
<marostegui> |
Stop MySQL on db2096 (codfw x1 master) for reimage T254871 |
[production] |
| 12:03 |
<cdanis> |
re-pool eqiad T256512 |
[production] |
| 12:01 |
<arturo> |
introduced spam filter in the mail server (T120210) |
[tools] |
| 11:59 |
<cdanis> |
deployed I132075ee on cr1-eqiad T256512 |
[production] |
| 11:58 |
<cdanis> |
deployed I132075ee on cr2-eqiad T256512 |
[production] |
| 11:58 |
<cdanis> |
deployed I132075ee on cr2-eqiad |
[production] |
| 11:45 |
<wm-bot> |
<lucaswerkmeister-wmde> deployed 281c5aaa6d (remove unneeded HTTPS redirect) |
[tools.wdvd] |
| 11:42 |
<wm-bot> |
<lucaswerkmeister-wmde> deployed 539d8b3c2a (php7.3) |
[tools.wdvd] |
| 11:41 |
<cdanis> |
depool eqiad T256512 |
[production] |
| 11:39 |
<wm-bot> |
<lucaswerkmeister-wmde> deployed b500edb33e (toolforge.org) |
[tools.wdvd] |
| 11:38 |
<wm-bot> |
<lucaswerkmeister-wmde> deployed 5dca2c92f7 (service.template file) |
[tools.wdvd] |
| 11:35 |
<wm-bot> |
<lucaswerkmeister-wmde> deployed cb4b443274 (remove debug code) |
[tools.wdvd] |
| 11:15 |
<awight> |
EU BACON cooked |
[production] |
| 11:08 |
<marostegui> |
Deploy schema change on db1095:3312 (lag will show up) |
[production] |
| 10:41 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:608284| Bumping portals to master (608284)]] (duration: 00m 57s) |
[production] |