|
2021-03-26
§
|
| 09:27 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1010.eqiad.wmnet |
[production] |
| 09:20 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ganeti1010.eqiad.wmnet |
[production] |
| 08:38 |
<moritzm> |
drain ganeti1010 |
[production] |
| 08:38 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1009.eqiad.wmnet |
[production] |
| 08:30 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single for host ganeti1009.eqiad.wmnet |
[production] |
| 06:11 |
<ryankemper> |
[WDQS Deploy] Deploy complete. Successful test query placed on query.wikidata.org, there's no relevant criticals in Icinga, and Grafana looks good |
[production] |
| 06:09 |
<ryankemper> |
[WDQS Deploy] Restarted `wdqs-updater` across all hosts, 4 hosts at a time: `sudo -E cumin -b 4 'A:wdqs-all' 'systemctl restart wdqs-updater'` |
[production] |
| 06:09 |
<ryankemper> |
[WDQS Deploy] Restarted `wdqs-categories` across all test hosts simultaneously: `sudo -E cumin 'A:wdqs-test' 'systemctl restart wdqs-categories'` |
[production] |
| 06:09 |
<ryankemper> |
[WDQS Deploy] Restarted `wdqs-categories` across lvs-managed hosts, one node at a time: `sudo -E cumin -b 1 'A:wdqs-all and not A:wdqs-test' 'depool && sleep 45 && systemctl restart wdqs-categories && sleep 45 && pool'` |
[production] |
| 05:06 |
<ryankemper@deploy1002> |
Finished deploy [wdqs/wdqs@bb5a072]: 0.3.68 (duration: 07m 31s) |
[production] |
| 05:00 |
<ryankemper> |
[WDQS Deploy] Tests passing following deploy of `0.3.68` on canary `wdqs1003`; proceeding to rest of fleet |
[production] |
| 04:58 |
<ryankemper@deploy1002> |
Started deploy [wdqs/wdqs@bb5a072]: 0.3.68 |
[production] |
| 04:58 |
<ryankemper> |
[WDQS Deploy] Gearing up for deploy of wdqs `0.3.68`. Pre-deploy tests passing on canary `wdqs1003` |
[production] |
|
2021-03-25
§
|
| 23:47 |
<thcipriani@deploy1002> |
Synchronized php-1.36.0-wmf.36/extensions/3D/package.json: No-op demo sync (duration: 01m 07s) |
[production] |
| 23:37 |
<stran@deploy1002> |
Synchronized README: (no justification provided) (duration: 01m 06s) |
[production] |
| 23:20 |
<jhuneidi@deploy1002> |
Synchronized README: [[gerrit:674984|DEMO: README]] (duration: 01m 07s) |
[production] |
| 22:59 |
<brennen> |
no patches for upcoming deploy window, but we'll be conducting a deployment training using DEMO patches to READMEs. |
[production] |
| 22:16 |
<Urbanecm> |
[urbanecm@mwmaint1002 ~]$ mwscript deleteEqualMessages.php --wiki=hrwiki --delete |
[production] |
| 21:35 |
<kharlan@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 21:35 |
<kharlan@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 21:31 |
<kharlan@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'production' . |
[production] |
| 21:31 |
<kharlan@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'linkrecommendation' for release 'external' . |
[production] |
| 21:27 |
<kharlan@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'linkrecommendation' for release 'staging' . |
[production] |
| 19:48 |
<hashar@deploy1002> |
rebuilt and synchronized wikiversions files: Revert group 1 and 2 wikis to 1.36.0-wmf.35 - T274940 |
[production] |
| 19:37 |
<hashar@deploy1002> |
rebuilt and synchronized wikiversions files: Revert group2 wikis to 1.36.0-wmf.35 - T274940 |
[production] |
| 19:36 |
<hashar@deploy1002> |
sync-wikiversions aborted: (no justification provided) (duration: 00m 03s) |
[production] |
| 19:11 |
<hashar@deploy1002> |
rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.36 |
[production] |
| 19:04 |
<urbanecm@deploy1002> |
Synchronized wmf-config/flaggedrevs.php: ce7d2d7a51bd2e3717b4de7b2f7e8ae427c221ad: ruwiki: flaggedrevs: Delete autoeditor group (T275337) (duration: 01m 08s) |
[production] |
| 19:01 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: ce7d2d7a51bd2e3717b4de7b2f7e8ae427c221ad: ruwiki: flaggedrevs: Delete autoeditor group (T275337) (duration: 01m 06s) |
[production] |
| 18:59 |
<Urbanecm> |
`mwscript migrateUserGroup.php --wiki=ruwiki 'autoeditor' 'autoreview' ` finished (T275337) |
[production] |
| 18:53 |
<Urbanecm> |
[urbanecm@mwmaint1002 ~/uploads]$ mwscript importImages.php --wiki=commonswiki --comment-ext=txt --user=Sturm . # T278391 |
[production] |
| 18:50 |
<Urbanecm> |
mwscript migrateUserGroup.php --wiki=ruwiki 'autoeditor' 'autoreview' # T275337 |
[production] |
| 18:49 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 39cd4f15a3900783ac0e9a213004a28f18298a23: ruwiki: flaggedrevs: Do not allow sysops to modify users in autoeditor group (T275337) (duration: 01m 09s) |
[production] |
| 18:45 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: dcfb7feaace1f397169e5e1bab7efd4e5f605a0f: ruwiki: flaggedrevs: Do not remove autoreview group (T275337) (duration: 01m 14s) |
[production] |
| 18:39 |
<urbanecm@deploy1002> |
Synchronized wmf-config/flaggedrevs.php: 3fb664682bea3c4d1448b0937f938e810268bac3: ruwiki: flaggedrevs: Revoke review from sysop group (T275811) (duration: 01m 06s) |
[production] |
| 18:29 |
<urbanecm@deploy1002> |
Synchronized logos/config.yaml: 29660f9ae8468aac1578b2905606ba9dd41d095f: Update altwiki logo (3/3; T275819) (duration: 01m 06s) |
[production] |
| 18:28 |
<urbanecm@deploy1002> |
Synchronized wmf-config/logos.php: 29660f9ae8468aac1578b2905606ba9dd41d095f: Update altwiki logo (2/3; T275819) (duration: 01m 06s) |
[production] |
| 18:26 |
<urbanecm@deploy1002> |
Synchronized static/images/project-logos/: 29660f9ae8468aac1578b2905606ba9dd41d095f: Update altwiki logo (1/3; T275819) (duration: 01m 10s) |
[production] |
| 18:21 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 62be4e738a4fd45256027bb09b010ab152f19850: Disable magic links on enwiki (T275951) (duration: 01m 20s) |
[production] |
| 18:14 |
<mutante> |
alert1001 - sudo systemctl restart tcpircbot-logmsgbot |
[production] |
| 18:09 |
<marxarelli> |
scap sync-file .pipeline Config: [[gerrit:674132|Include patches in restricted image (T271274)]] |
[production] |
| 18:06 |
<hnowlan> |
draining and restarting aqs1004-b cassandra |
[production] |
| 17:45 |
<hnowlan> |
draining and restarting aqs1004-a cassandra |
[production] |
| 17:16 |
<mbsantos@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 17:14 |
<mbsantos@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 17:08 |
<mbsantos@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'proton' for release 'production' . |
[production] |
| 16:39 |
<hashar> |
Restarted Apache 2 on contint2001 / contint1001 |
[production] |
| 16:35 |
<ryankemper@cumin1001> |
START - Cookbook sre.elasticsearch.rolling-upgrade |
[production] |
| 16:32 |
<moritzm> |
restarting apache on an-tool1007/turnilo |
[production] |
| 16:27 |
<moritzm> |
restarting dnsdist/rdns-recursor on malmok |
[production] |