|
2021-06-23
§
|
| 07:46 |
<kartik@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 07:26 |
<legoktm> |
uploaded mailman3_3.3.3-1~bpo10+6_amd64.changes on apt1001 |
[production] |
| 07:08 |
<legoktm> |
updating mailman packages on lists1001 and restarting (T285120, T280889) |
[production] |
| 06:56 |
<ryankemper> |
[WDQS] `ryankemper@wdqs1006:~$ sudo pool` |
[production] |
| 06:37 |
<ryankemper> |
[WDQS] `ryankemper@wdqs2001:~$ sudo pool` |
[production] |
| 06:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 100%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16703 and previous config saved to /var/cache/conftool/dbconfig/20210623-062819-root.json |
[production] |
| 06:13 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 75%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16702 and previous config saved to /var/cache/conftool/dbconfig/20210623-061316-root.json |
[production] |
| 05:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 50%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16701 and previous config saved to /var/cache/conftool/dbconfig/20210623-055812-root.json |
[production] |
| 05:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Start repooling db1100', diff saved to https://phabricator.wikimedia.org/P16700 and previous config saved to /var/cache/conftool/dbconfig/20210623-054252-marostegui.json |
[production] |
| 04:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 25%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16699 and previous config saved to /var/cache/conftool/dbconfig/20210623-045217-root.json |
[production] |
| 01:04 |
<eileen> |
process-control config revision is 6a88618c3e |
[production] |
| 00:50 |
<eileen> |
civicrm revision changed from c745d4f075 to 03bead707d, config revision is 4ab72c1033 |
[production] |
| 00:40 |
<legoktm> |
uploaded new versions of flufl.bounce_4.0-1_amd64.changes hyperkitty_1.3.4-2~bpo10+4_amd64.changes mailman3_3.3.3-1~bpo10+5_amd64.changes mailman-hyperkitty_1.1.0-10~bpo10+1_amd64.changes to apt1001 |
[production] |
| 00:02 |
<Trey314159> |
reindexing Portuguese wikis on elastic@eqiad, elastic@codfw, and cloudelastic complete (T284185) |
[production] |
|
2021-06-22
§
|
| 23:23 |
<ebernhardson@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Enable canary events for search event streams (duration: 01m 05s) |
[production] |
| 23:13 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 7865f27430f8eea2975d7154f6009a9206fc75d6: Add unwatchedpages to rollbacker on frwiki (T285334) (duration: 01m 06s) |
[production] |
| 23:08 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 9a594f0ce249e2b4752ea2b8d7c4258bf14ad86a: Enable Growth features in dark mode at nlwiki (T285254; 3/3) (duration: 01m 07s) |
[production] |
| 23:05 |
<urbanecm@deploy1002> |
Synchronized wmf-config/config/nlwiki.yaml: 9a594f0ce249e2b4752ea2b8d7c4258bf14ad86a: Enable Growth features in dark mode at nlwiki (T285254; 2/3) (duration: 01m 05s) |
[production] |
| 23:04 |
<urbanecm@deploy1002> |
Synchronized dblists/growthexperiments.dblist: 9a594f0ce249e2b4752ea2b8d7c4258bf14ad86a: Enable Growth features in dark mode at nlwiki (T285254; 1/3) (duration: 01m 37s) |
[production] |
| 22:42 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ mwscript recountCategories.php --wiki=zhwiki --mode=subcats # T170737 |
[production] |
| 22:41 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ mwscript recountCategories.php --wiki=zhwiki --mode=pages # T170737 |
[production] |
| 22:38 |
<urbanecm> |
mwscript recountCategories.php --wiki=eowiktionary --mode={pages,subcats,files} (T170737) |
[production] |
| 21:05 |
<eileen> |
civicrm revision changed from 629bd3b7b7 to c745d4f075, config revision is 4ab72c1033 |
[production] |
| 21:05 |
<ejegg> |
updated payments-wiki from 7be0534b91 to 42cfbe832d |
[production] |
| 20:46 |
<brennen> |
gitlab1001: running ansible to deploy [[gerrit:699819|CAS: stop marking users as external]] (T274461) |
[production] |
| 20:23 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-web1001.eqiad.wmnet with reason: REIMAGE |
[production] |
| 20:19 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on an-web1001.eqiad.wmnet with reason: REIMAGE |
[production] |
| 20:12 |
<Trey314159> |
reindexing Portuguese wikis on elastic@eqiad, elastic@codfw, and cloudelastic (T284185) |
[production] |
| 20:12 |
<Trey314159> |
reindexing Dutch wikis on elastic@eqiad, elastic@codfw, and cloudelastic complete (T284185) |
[production] |
| 19:58 |
<brennen> |
gitlab1001: run ansible to deploy https://gerrit.wikimedia.org/r/c/operations/gitlab-ansible/+/699812 (T264231) |
[production] |
| 19:26 |
<legoktm> |
set mediawiki-l message acceptance to discard non-member posts instead of reject |
[production] |
| 19:09 |
<dduvall@deploy1002> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.37.0-wmf.11 |
[production] |
| 19:06 |
<dduvall> |
preparing to promote wmf.11 group0 (T281152) cc'ing risking patch contacts Amir1, Krinkle, DannyS712 |
[production] |
| 19:01 |
<dduvall@deploy1002> |
Pruned MediaWiki: 1.37.0-wmf.6 (duration: 03m 35s) |
[production] |
| 18:46 |
<ebernhardson@deploy1002> |
Finished deploy [wikimedia/discovery/analytics@75d35b4]: revert expect eventgate canary events in all dcs (duration: 04m 23s) |
[production] |
| 18:42 |
<ebernhardson@deploy1002> |
Started deploy [wikimedia/discovery/analytics@75d35b4]: revert expect eventgate canary events in all dcs |
[production] |
| 18:31 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on thumbor1006.eqiad.wmnet with reason: REIMAGE |
[production] |
| 18:30 |
<awight@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/VisualEditor: Backport: [[gerrit:700734|Revert "Fall back from explicit parameter order to TemplateData sort" ()]] (duration: 01m 09s) |
[production] |
| 18:28 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on thumbor1006.eqiad.wmnet with reason: REIMAGE |
[production] |
| 18:28 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on thumbor1005.eqiad.wmnet with reason: REIMAGE |
[production] |
| 18:27 |
<awight@deploy1002> |
sync-file aborted: Backport: [[gerrit:700734|Revert "Fall back from explicit parameter order to TemplateData sort" ()]] (duration: 00m 40s) |
[production] |
| 18:26 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on thumbor1005.eqiad.wmnet with reason: REIMAGE |
[production] |
| 18:19 |
<legoktm> |
pulled in updates for thirdparty/kubeadm-k8s-1-18 buster-wikimedia on apt1001 |
[production] |
| 17:47 |
<brennen> |
gitlab1001: run ansible to deploy https://gerrit.wikimedia.org/r/700851 (T274463) |
[production] |
| 17:42 |
<dduvall> |
testwikis to 1.37.0-wmf.11 (cc open blockers T285125 T285118 T271011) |
[production] |
| 17:41 |
<dduvall@deploy1002> |
Finished scap: testwikis wikis to 1.37.0-wmf.11 (duration: 30m 59s) |
[production] |
| 17:21 |
<moritzm> |
installing isc-dhcp security updates |
[production] |
| 17:18 |
<mbsantos@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 17:16 |
<mbsantos@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 17:14 |
<mbsantos@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |