|
2021-04-19
ยง
|
| 14:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 30%: Slowly pool db1182 for the first time in s2 T275633', diff saved to https://phabricator.wikimedia.org/P15466 and previous config saved to /var/cache/conftool/dbconfig/20210419-142608-root.json |
[production] |
| 14:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 20%: Slowly pool db1182 for the first time in s2 T275633', diff saved to https://phabricator.wikimedia.org/P15465 and previous config saved to /var/cache/conftool/dbconfig/20210419-141105-root.json |
[production] |
| 13:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 15%: Slowly pool db1182 for the first time in s2 T275633', diff saved to https://phabricator.wikimedia.org/P15464 and previous config saved to /var/cache/conftool/dbconfig/20210419-135601-root.json |
[production] |
| 13:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 10%: Slowly pool db1182 for the first time in s2 T275633', diff saved to https://phabricator.wikimedia.org/P15463 and previous config saved to /var/cache/conftool/dbconfig/20210419-134057-root.json |
[production] |
| 13:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1182 (re)pooling @ 5%: Slowly pool db1182 for the first time in s2 T275633', diff saved to https://phabricator.wikimedia.org/P15462 and previous config saved to /var/cache/conftool/dbconfig/20210419-132554-root.json |
[production] |
| 13:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Pool db1182 in s2 for the first time with minimal weight T275633', diff saved to https://phabricator.wikimedia.org/P15461 and previous config saved to /var/cache/conftool/dbconfig/20210419-131936-marostegui.json |
[production] |
| 13:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Pool db1182 in s2 for the first time with minimal weight T275633', diff saved to https://phabricator.wikimedia.org/P15460 and previous config saved to /var/cache/conftool/dbconfig/20210419-131501-marostegui.json |
[production] |
| 12:58 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: bd076306c0ae0428ff13743f499b2a02d42b6eab: wgGEMentorshipMigrationStage: Set to WRITE_BOTH/READ_OLD everywhere (T279853) (duration: 00m 57s) |
[production] |
| 12:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1182', diff saved to https://phabricator.wikimedia.org/P15459 and previous config saved to /var/cache/conftool/dbconfig/20210419-125600-marostegui.json |
[production] |
| 12:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Pool db1182 in s2 for the first time with minimal weight T275633', diff saved to https://phabricator.wikimedia.org/P15458 and previous config saved to /var/cache/conftool/dbconfig/20210419-125407-marostegui.json |
[production] |
| 12:53 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Add db1182 to dbctl T275633', diff saved to https://phabricator.wikimedia.org/P15457 and previous config saved to /var/cache/conftool/dbconfig/20210419-125301-marostegui.json |
[production] |
| 12:51 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: ef0f68e2a9c1c638911bb06c47ba6e8ef88ee393: testwiki: wgGEMentorshipMigrationStage: Set to WRITE_BOTH/READ_NEW (T279853) (duration: 00m 57s) |
[production] |
| 12:38 |
<Urbanecm> |
mwscript extensions/GrowthExperiments/maintenance/migrateMentorMenteeRelationship.php --wiki=cswiki # T279853 |
[production] |
| 12:34 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2126.codfw.wmnet with reason: REIMAGE |
[production] |
| 12:34 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 3e3cce192f1e99cbcae739f234271411d10974ac: cswiki: wgGEMentorshipMigrationStage: Set to WRITE_BOTH/READ_OLD (T279853) (duration: 00m 58s) |
[production] |
| 12:32 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db2126.codfw.wmnet with reason: REIMAGE |
[production] |
| 12:26 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2072.codfw.wmnet with reason: REIMAGE |
[production] |
| 12:24 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db2072.codfw.wmnet with reason: REIMAGE |
[production] |
| 11:39 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudgw1002.eqiad.wmnet |
[production] |
| 11:37 |
<aborrero@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudgw1001.eqiad.wmnet |
[production] |
| 11:33 |
<moritzm> |
imported debdeploy 0.0.99.13-1+deb11u1 to bullseye-wikimedia T275873 |
[production] |
| 11:27 |
<Urbanecm> |
[urbanecm@mwmaint1002 ~]$ mwscript extensions/GrowthExperiments/maintenance/migrateMentorMenteeRelationship.php --wiki=testwiki --force # T279853 |
[production] |
| 11:11 |
<Urbanecm> |
[urbanecm@mwmaint1002 ~]$ mwscript extensions/GrowthExperiments/maintenance/migrateMentorMenteeRelationship.php --wiki=testwiki # T279853 |
[production] |
| 11:10 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 03f8ed819091624f5ae4a8d7ed3631dc322fabcd: testwiki: wgGEMentorshipMigrationStage: Set to WRITE_BOTH/READ_OLD (T279853) (duration: 00m 57s) |
[production] |
| 11:05 |
<ladsgroup@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:680871|Disable legacy javascript variable for the rest of wikis (T72470)]] (duration: 00m 57s) |
[production] |
| 11:02 |
<moritzm> |
import promethus-rsyslog-exporter for bullseye-wikimedia/main |
[production] |
| 11:01 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host cloudgw1002.eqiad.wmnet |
[production] |
| 11:01 |
<aborrero@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host cloudgw1001.eqiad.wmnet |
[production] |
| 10:46 |
<jdrewniak@deploy1002> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:681008| Bumping portals to master (T128546)]] (duration: 00m 57s) |
[production] |
| 10:45 |
<jdrewniak@deploy1002> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:681008| Bumping portals to master (T128546)]] (duration: 00m 58s) |
[production] |
| 10:34 |
<jmm@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on theemin.codfw.wmnet with reason: REIMAGE |
[production] |
| 10:32 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on theemin.codfw.wmnet with reason: REIMAGE |
[production] |
| 10:24 |
<hnowlan> |
imported 1.16.3 into envoy-future |
[production] |
| 10:22 |
<moritzm> |
reimaging theemin to bullseye |
[production] |
| 10:15 |
<dcausse> |
depooling wdqs1005 |
[production] |
| 10:10 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1156.eqiad.wmnet with reason: REIMAGE |
[production] |
| 10:08 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1156.eqiad.wmnet with reason: REIMAGE |
[production] |
| 10:05 |
<arturo> |
aborrero@apt1001:~ $ sudo -i reprepro --component thirdparty/kubeadm-k8s-1-18 update buster-wikimedia |
[production] |
| 10:04 |
<arturo> |
aborrero@apt1001:~ $ sudo -i reprepro --delete clearvanished (remove old buster-wikimedia|thirdparty/kubeadm-k8s-1-15,16 repos and packages) |
[production] |
| 09:56 |
<ema> |
cp3051: varnish-frontend-restart to apply exp policy settings changes starting from empty cache T275809 |
[production] |
| 09:53 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on thanos-fe2002.codfw.wmnet with reason: REIMAGE |
[production] |
| 09:51 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on thanos-fe2002.codfw.wmnet with reason: REIMAGE |
[production] |
| 09:42 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1161.eqiad.wmnet with reason: REIMAGE |
[production] |
| 09:39 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1161.eqiad.wmnet with reason: REIMAGE |
[production] |
| 09:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1179 (re)pooling @ 100%: Slowly pool db1179 for the first time in s3 T275633', diff saved to https://phabricator.wikimedia.org/P15454 and previous config saved to /var/cache/conftool/dbconfig/20210419-092251-root.json |
[production] |
| 09:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1161 T280492', diff saved to https://phabricator.wikimedia.org/P15453 and previous config saved to /var/cache/conftool/dbconfig/20210419-092234-marostegui.json |
[production] |
| 09:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1079 (re)pooling @ 100%: Repool db1079', diff saved to https://phabricator.wikimedia.org/P15452 and previous config saved to /var/cache/conftool/dbconfig/20210419-091535-root.json |
[production] |
| 09:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1179 (re)pooling @ 90%: Slowly pool db1179 for the first time in s3 T275633', diff saved to https://phabricator.wikimedia.org/P15451 and previous config saved to /var/cache/conftool/dbconfig/20210419-090747-root.json |
[production] |
| 09:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1079 (re)pooling @ 75%: Repool db1079', diff saved to https://phabricator.wikimedia.org/P15450 and previous config saved to /var/cache/conftool/dbconfig/20210419-090031-root.json |
[production] |
| 08:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1179 (re)pooling @ 80%: Slowly pool db1179 for the first time in s3 T275633', diff saved to https://phabricator.wikimedia.org/P15449 and previous config saved to /var/cache/conftool/dbconfig/20210419-085243-root.json |
[production] |