|
2021-03-11
§
|
| 22:11 |
<marxarelli> |
reverted https://gerrit.wikimedia.org/r/670963 and re-running failed job(T277236) |
[releng] |
| 22:10 |
<marxarelli> |
running `tox -e jenkins-jobs -- --conf jenkins_jobs.ini update ./jjb/ trigger-research-mwaddlink-pipeline-test research-mwaddlink-pipeline-test` to revert https://gerrit.wikimedia.org/r/c/integration/config/+/668199 for select jobs (T277236) |
[releng] |
| 22:03 |
<marxarelli> |
running `tox -e jenkins-jobs -- --conf jenkins_jobs.ini update ./jjb/ trigger-research-mwaddlink-pipeline-test research-mwaddlink-pipeline-test` to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/668199 for select jobs (T277236) |
[releng] |
| 16:57 |
<Majavah> |
copy a tarball of deployment-fluorine02 /home to deployment-mwlog01 root home dir, delete deployment-fluorine02 T276419 |
[releng] |
| 16:49 |
<Majavah> |
delete deployment-etcd-01 T276462 |
[releng] |
| 13:51 |
<Majavah> |
shut down deployment-db06, now unused T277070 |
[releng] |
| 13:48 |
<Majavah> |
set deployment-db07 as r/w T277070 |
[releng] |
| 13:48 |
<Majavah> |
stop mariadb to ensure reads have stopped on deployment-db06 T277070 |
[releng] |
| 13:41 |
<Majavah> |
stop slave on deployment-db06 T276968 |
[releng] |
| 13:37 |
<Majavah> |
make deployment-db06 and deployment-db08 be replicas of deployment-db07 T277070 |
[releng] |
| 13:34 |
<Majavah> |
stop and reset slave on deployment-db07 T277070 |
[releng] |
| 13:32 |
<Majavah> |
set deployment-db06 as read only T277070 |
[releng] |
| 09:12 |
<hashar> |
Updated plugins on https://releases-jenkins.wikimedia.org/ |
[releng] |
|
2021-03-10
§
|
| 19:36 |
<Majavah> |
shutdown deployment-ircd T277081 |
[releng] |
| 18:46 |
<Majavah> |
switch floating ip 185.15.56.34 to deployment-ircd02 T277081 |
[releng] |
| 18:05 |
<Majavah> |
create deployment-ircd02 for T277081 |
[releng] |
| 17:26 |
<marxarelli> |
`rm -rf /srv/dump` on deployment-db06 and reenabling puppet |
[releng] |
| 17:25 |
<marxarelli> |
`rm -rf /srv/restore` on deployment-db08 and reenabling puppet |
[releng] |
| 17:24 |
<marxarelli> |
`rm -rf /srv/backup /srv/restore` on deployment-db07 and reenabling puppet |
[releng] |
| 17:09 |
<Majavah> |
set beta cluster mediawiki as read write on mw config (T276968) |
[releng] |
| 17:03 |
<Majavah> |
make deployment-db06 read-write T276968 |
[releng] |
| 16:50 |
<Majavah> |
`reset slave;` on new master deployment-db06 T276968 |
[releng] |
| 16:49 |
<Majavah> |
add deployment-db07 as a replica of db06 for T276968 |
[releng] |
| 16:45 |
<Urbanecm> |
root@deployment-db07:/opt/wmf-mariadb104/bin# ./mysql_upgrade -h 127.0.0.1 # T276968 |
[releng] |
| 16:12 |
<Majavah> |
deployment-db08 CHANGE MASTER to MASTER_USER='repl', MASTER_PASSWORD='redacted', MASTER_PORT=3306, MASTER_HOST='deployment-db06.deployment-prep.eqiad1.wikimedia.cloud', MASTER_LOG_FILE='deployment-db06-bin.000059', MASTER_LOG_POS=522469730; (T276968) |
[releng] |
| 16:06 |
<Urbanecm> |
start root@deployment-db07:/srv/sqldata.db06# rsync --progress -r deployment-db06:/srv/sqldata/ . (T276968) |
[releng] |
| 15:57 |
<Majavah> |
set deployment-db06 as readonly from mysql side T276968 |
[releng] |
| 15:54 |
<Urbanecm> |
Start `root@deployment-db08:/opt/wmf-mariadb104/bin# ./mysql_upgrade -h 127.0.0.1` (T276968) |
[releng] |
| 15:54 |
<Urbanecm> |
Start mariadb on db08 (T276968) |
[releng] |
| 15:22 |
<Urbanecm> |
rsync deployment-db06:/srv/sqldata to deployment-db08:/srv/sqldata in a tmux session on deploymdeployment-db08 (T276968) |
[releng] |
| 14:52 |
<Majavah> |
delete deployment-db08 /srv/sqldata to attempt procedure in https://phabricator.wikimedia.org/T276968#6900199 |
[releng] |
| 10:16 |
<arturo> |
briefly stopping deployment-puppetdb03 to disable VMX CPU flag |
[releng] |
| 00:28 |
<marxarelli> |
mariadb successfully started on db07 following transfer/extraction using mariabackup and following mysql_upgrade (T276968) |
[releng] |
| 00:10 |
<marxarelli> |
restore of db06 failed yet again. trying mariabackup db06 -> db07 instead of mysqldump (after fixing docs/usage of the former) (T276968) |
[releng] |
|
2021-03-09
§
|
| 21:54 |
<marxarelli> |
restoring from db06 dump on db07 and db08 following `DROP VIEW IF EXISTS user` workaround (T276968) |
[releng] |
| 20:53 |
<marxarelli> |
restore on db07 failed. appears to be a bug w/ mariadb/mysqldump 10.4 compat https://jira.mariadb.org/browse/MDEV-22127 (T276968) |
[releng] |
| 20:53 |
<marxarelli> |
restore on db07 failed. appears to be a bug w/ mariadb/mysqldump 10.4 compat https://jira.mariadb.org/browse/MDEV-22127 |
[releng] |
| 20:39 |
<marxarelli> |
doing `--skip-grant-tables` on deployment-db08 and creating a new root@127.0.0.1 user (T276968) |
[releng] |
| 20:33 |
<Majavah> |
install mariadb on deployment-db08 T276968 |
[releng] |
| 19:59 |
<marxarelli> |
creating new instance deployment-db08 to use as new beta replica db (T276968) |
[releng] |
| 19:56 |
<marxarelli> |
deleting deployment-db05 to free up quota for new replica (T276968) |
[releng] |
| 19:50 |
<marxarelli> |
restoring database dump on deployment-db07 (T276968) |
[releng] |
| 18:49 |
<marxarelli> |
restarting db dump on db06 `mysqldump -h 127.0.0.1 --events --routines --triggers --all-databases -f --single-transaction` (T276968) |
[releng] |
| 18:38 |
<Majavah> |
installing mariadb 10.4 via role::mariadb::beta to db07 T276968 |
[releng] |
| 18:25 |
<marxarelli> |
"View 'labswiki.tag_summary' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them" when using LOCK TABLES" during mysqldump on db06 (T276968) |
[releng] |
| 18:21 |
<Majavah> |
create deployment-db07 as g2.cores8.ram16.disk160 Buster T276968 |
[releng] |
| 18:20 |
<marxarelli> |
disabled puppet on deployment-db06 and started mysqldump (T276968) |
[releng] |
| 18:09 |
<Majavah> |
set deployment-db05 to read-only to avoid issues with T276968 |
[releng] |
| 18:04 |
<marxarelli> |
deleting shut down memc* deployment-prep instances to free up quota for replacement db instances (T276968) |
[releng] |