|
2021-10-21
ยง
|
| 14:26 |
<otto@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . |
[production] |
| 14:19 |
<otto@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . |
[production] |
| 14:19 |
<otto@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . |
[production] |
| 13:56 |
<otto@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . |
[production] |
| 13:55 |
<otto@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . |
[production] |
| 13:49 |
<otto@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . |
[production] |
| 13:49 |
<otto@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . |
[production] |
| 13:34 |
<volans> |
uploaded spicerack_1.0.6 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia |
[production] |
| 13:08 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 13:05 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 13:04 |
<hashar@deploy1002> |
rebuilt and synchronized wikiversions files: all wikis to 1.38.0-wmf.5 refs T281169 |
[production] |
| 12:56 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 7 hosts with reason: Schema change s3 T278619 |
[production] |
| 12:56 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on 7 hosts with reason: Schema change s3 T278619 |
[production] |
| 12:52 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 14 hosts with reason: Schema change s1 T278619 |
[production] |
| 12:52 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on 14 hosts with reason: Schema change s1 T278619 |
[production] |
| 12:48 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 13 hosts with reason: Schema change s4 T278619 |
[production] |
| 12:48 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on 13 hosts with reason: Schema change s4 T278619 |
[production] |
| 12:43 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 9 hosts with reason: Schema change s2 T278619 |
[production] |
| 12:43 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime for 1:00:00 on 9 hosts with reason: Schema change s2 T278619 |
[production] |
| 12:34 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 11 hosts with reason: Schema change s7 T278619 |
[production] |
| 12:34 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime for 1:00:00 on 11 hosts with reason: Schema change s7 T278619 |
[production] |
| 11:55 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 9 hosts with reason: Schema change s5 T278619 |
[production] |
| 11:54 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime for 1:00:00 on 9 hosts with reason: Schema change s5 T278619 |
[production] |
| 11:47 |
<kormat@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 9 hosts with reason: Schema change s6 T278619 |
[production] |
| 11:47 |
<kormat@cumin1001> |
START - Cookbook sre.hosts.downtime for 1:00:00 on 9 hosts with reason: Schema change s6 T278619 |
[production] |
| 11:13 |
<Lucas_WMDE> |
UTC morning backport+config window done |
[production] |
| 11:10 |
<Lucas_WMDE> |
lucaswerkmeister-wmde@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/ResubmitChanges.php wikidatawiki --minimum-age $((60*60*12)) # T294008 |
[production] |
| 11:10 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 11:07 |
<jgiannelos@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:730848|Configure event stream for map tiles state change (T289771)]] (duration: 01m 04s) |
[production] |
| 11:07 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 10:48 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 10:48 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 10:48 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 10:47 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 10:14 |
<jbond> |
mergeing refactor of P:base Gerrit:714975 |
[production] |
| 09:54 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 09:49 |
<ayounsi@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 08:56 |
<urbanecm@deploy1002> |
Synchronized private/PrivateSettings.php: Update T250887 mitigations (duration: 01m 03s) |
[production] |
| 08:33 |
<ema@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp3062.esams.wmnet,service=(varnish-fe|ats-tls) |
[production] |
| 08:26 |
<ema@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=cp3062.esams.wmnet,service=(varnish-fe|ats-tls) |
[production] |
| 08:25 |
<ema> |
cp3062: revert vsl_space experiment T293879 |
[production] |
| 08:24 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host graphite1004.eqiad.wmnet with OS bullseye |
[production] |
| 08:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1118 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17563 and previous config saved to /var/cache/conftool/dbconfig/20211021-080330-root.json |
[production] |
| 07:56 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reimage for host graphite1004.eqiad.wmnet with OS bullseye |
[production] |
| 07:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1118 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17562 and previous config saved to /var/cache/conftool/dbconfig/20211021-074826-root.json |
[production] |
| 07:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1118 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17561 and previous config saved to /var/cache/conftool/dbconfig/20211021-073323-root.json |
[production] |
| 07:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1118 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17560 and previous config saved to /var/cache/conftool/dbconfig/20211021-071819-root.json |
[production] |
| 07:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1118 (re)pooling @ 10%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17559 and previous config saved to /var/cache/conftool/dbconfig/20211021-070315-root.json |
[production] |
| 06:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1118 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17558 and previous config saved to /var/cache/conftool/dbconfig/20211021-064812-root.json |
[production] |
| 06:35 |
<elukey> |
`systemctl reload nginx` on cloudelastic100[5,6] to pick up the new TLS certificate and clear alerts - T293826 |
[production] |