|
2020-06-10
§
|
| 09:35 |
<marostegui> |
Stop mysql on db1127 to clone db1103 |
[production] |
| 09:34 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1137 for cloning db1103 - T253217', diff saved to https://phabricator.wikimedia.org/P11443 and previous config saved to /var/cache/conftool/dbconfig/20200610-093440-marostegui.json |
[production] |
| 09:31 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 09:31 |
<godog> |
configure thanos-be1* HDDs as raid0 - T252186 |
[production] |
| 09:26 |
<marostegui@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 09:26 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Add db1103 to dbctl, depooled T253217', diff saved to https://phabricator.wikimedia.org/P11442 and previous config saved to /var/cache/conftool/dbconfig/20200610-092603-marostegui.json |
[production] |
| 09:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove db1103:3312 and db1103:3314', diff saved to https://phabricator.wikimedia.org/P11441 and previous config saved to /var/cache/conftool/dbconfig/20200610-092406-marostegui.json |
[production] |
| 09:14 |
<jayme> |
T254581 disabling puppet on all mw, api and jobrunner servers to move termbox envoy config to TLS |
[production] |
| 09:08 |
<kormat@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 09:08 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 08:50 |
<XioNoX> |
make asw1-eqsin interfaces Homer like - T250429 |
[production] |
| 08:45 |
<jayme@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 08:45 |
<jayme@deploy1001> |
helmfile [EQIAD] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 08:45 |
<elukey@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 08:17 |
<jayme@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 08:14 |
<kormat@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 08:14 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 08:13 |
<jayme@deploy1001> |
helmfile [CODFW] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 07:52 |
<kormat> |
reimaging db1077 T252027 |
[production] |
| 07:36 |
<jayme@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'eventstreams' for release 'production' . |
[production] |
| 07:36 |
<XioNoX> |
make asw2-ulsfo interfaces Homer like - T250429 |
[production] |
| 07:33 |
<jayme@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'eventstreams' for release 'canary' . |
[production] |
| 07:31 |
<moritzm> |
upgrade mw1298-mw1309 (job runners) to PHP 7.2.31 |
[production] |
| 07:26 |
<XioNoX> |
trunk public vlan to esams ganeti hosts - T254157 |
[production] |
| 07:16 |
<XioNoX> |
trunk public vlan to eqsin ganeti hosts - T254157 |
[production] |
| 07:15 |
<moritzm> |
upgrade remaining API servers in eqiad to PHP 7.2.31 |
[production] |
| 07:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1103 for reimage - T253217', diff saved to https://phabricator.wikimedia.org/P11439 and previous config saved to /var/cache/conftool/dbconfig/20200610-070822-marostegui.json |
[production] |
| 07:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db2113 after on-site maintenance T251570', diff saved to https://phabricator.wikimedia.org/P11438 and previous config saved to /var/cache/conftool/dbconfig/20200610-070508-marostegui.json |
[production] |
| 06:53 |
<XioNoX> |
trunk public vlan to ulsfo ganeti hosts - T254157 |
[production] |
| 06:23 |
<James_F> |
Docker: Publishing mediawiki-phan:0.6.0 |
[releng] |
| 06:08 |
<James_F> |
Zuul: Drop extension-seccheck template, unused |
[releng] |
| 06:06 |
<James_F> |
Zuul: [mediawiki/extensions/Petition] Replace seccheck with phan |
[releng] |
| 06:05 |
<James_F> |
Zuul: [mediawiki/extensions/TEI] Replace seccheck with phan |
[releng] |
| 06:01 |
<James_F> |
Zuul: [mediawiki/extensions/Sudo] Replace seccheck with phan |
[releng] |
| 05:59 |
<James_F> |
Zuul: [mediawiki/extensions/ChessBrowser] Replace seccheck with phan |
[releng] |
| 05:10 |
<marostegui> |
Deploy schema change on s3 master with 2 minutes sleep between wikis - T206103 |
[production] |
| 00:22 |
<wm-bot> |
<samwilson> Updated to latest master. T254632. |
[tools.ia-upload] |
|
2020-06-09
§
|
| 23:18 |
<Reedy> |
run namespaceDupes.php --fix for hiwikibooks T254012 |
[production] |
| 23:10 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T254706 T254012 T241893 (duration: 01m 06s) |
[production] |
| 23:03 |
<Reedy> |
created wikilove_log on slwiki T254706 |
[production] |
| 22:55 |
<bstorm_> |
Reset the passwords for T254931 |
[clouddb-services] |
| 21:44 |
<RhinosF1> |
keeping logging_channel off with debug logs and hoping the bot timesout to get some decent non log spammed logs to try and fix bug 2/3 in T254348 |
[tools.zppixbot-test] |
| 21:24 |
<bd808> |
Credentials missing from es7 cluster for tool access. Fallout from T254491 |
[tools.bash] |
| 21:17 |
<RhinosF1> |
- messing stuff up to test T254348 |
[tools.zppixbot-test] |
| 20:00 |
<jhuneidi@deploy1001> |
Pruned MediaWiki: 1.35.0-wmf.32 (duration: 05m 11s) |
[production] |
| 19:51 |
<jhuneidi@deploy1001> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.36 |
[production] |
| 19:42 |
<jhuneidi@deploy1001> |
Finished scap: testwikis wikis to 1.35.0-wmf.36 (duration: 57m 47s) |
[production] |
| 19:29 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 19:26 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |