|
2021-06-23
ยง
|
| 18:54 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:54 |
<urbanecm@deploy1002> |
Scap failed!: 6/9 canaries failed their endpoint checks(https://en.wikipedia.org) |
[production] |
| 18:53 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/GrowthExperiments/: 76e5fc91083736d14049a05ed227cdea015c113e: Add custom signup flow for donors (T284740; T284800; T285281) (duration: 01m 07s) |
[production] |
| 18:39 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.11/extensions/WikimediaEvents/extension.json: 01f034b466ff7bdd274e18c9ad7cefe88245548d: Finalize WMDEBanner* schema migration to Event Platform (T282562) (duration: 01m 05s) |
[production] |
| 18:35 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.9/extensions/GrowthExperiments/includes/Specials/SpecialEditGrowthConfig.php: 17efbafc300c6745928415d7a10f2dad8f406de4: EditGrowthConfig: Suggested edit "Learn more" link should support interwiki (T279886; T285385) (duration: 01m 06s) |
[production] |
| 18:33 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 3a2fc6e3687817db5c774e7e527a10dd9e974138: Enable $wgSecurePollSingleTransferableVoteEnabled on beta sites (duration: 01m 05s) |
[production] |
| 18:31 |
<ebernhardson@deploy1002> |
Finished deploy [wikimedia/discovery/analytics@0535b94]: expect eventgate events for all datacenters, second try (duration: 09m 11s) |
[production] |
| 18:22 |
<ebernhardson@deploy1002> |
Started deploy [wikimedia/discovery/analytics@0535b94]: expect eventgate events for all datacenters, second try |
[production] |
| 18:09 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: b4a786799a249d2012b9c47553a0b64fdce1bac0: Make Growth features available to newcomers at lvwiki and skwiki (T278191; T284149) (duration: 01m 06s) |
[production] |
| 17:58 |
<herron> |
beginning rolling reboots of kafka-main100[1-5] for updates |
[production] |
| 17:57 |
<otto@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Enable canary events for NavigationTiming ext streams - T271208, T266798 (duration: 01m 29s) |
[production] |
| 17:07 |
<herron> |
beginning rolling reboots of kafka-main200[1-5] for updates |
[production] |
| 16:42 |
<XioNoX> |
re-start sending traffic on the codfw-eqsin Telia transport link |
[production] |
| 15:17 |
<topranks> |
Removing peering to AS64050 / "BGP Consultancy Pte Ltd" at AMS-IX (cr2-esams). Peer has left IX. |
[production] |
| 14:54 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist $SHARD recountCategories.php --mode=pages && foreachwikiindblist $SHARD recountCategories.php --mode=subcats && foreachwikiindblist $SHARD recountCategories.php --mode=files # T170737, SHARD=s1 |
[production] |
| 14:53 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist $SHARD recountCategories.php --mode=pages && foreachwikiindblist $SHARD recountCategories.php --mode=subcats && foreachwikiindblist $SHARD recountCategories.php --mode=files # T170737, SHARD=s8 |
[production] |
| 13:54 |
<effie> |
rolling restart thanos-fe* to pick up new tegola-vector-tiles account - T283049 |
[production] |
| 13:45 |
<volans> |
uploaded cumin_4.1.1 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia |
[production] |
| 13:27 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist $SHARD recountCategories.php --mode=pages && foreachwikiindblist $SHARD recountCategories.php --mode=subcats && foreachwikiindblist $SHARD recountCategories.php --mode=files # T170737, SHARD=s4 |
[production] |
| 12:59 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist $SHARD recountCategories.php --mode=pages && foreachwikiindblist $SHARD recountCategories.php --mode=subcats && foreachwikiindblist $SHARD recountCategories.php --mode=files # T170737, SHARD=s3 |
[production] |
| 12:46 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist $SHARD recountCategories.php --mode=pages && foreachwikiindblist $SHARD recountCategories.php --mode=subcats && foreachwikiindblist $SHARD recountCategories.php --mode=files # T170737, SHARD=s7 |
[production] |
| 12:35 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist $SHARD recountCategories.php --mode=pages && foreachwikiindblist $SHARD recountCategories.php --mode=subcats && foreachwikiindblist $SHARD recountCategories.php --mode=files # T170737, SHARD=s6 |
[production] |
| 12:26 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist $SHARD recountCategories.php --mode=pages && foreachwikiindblist $SHARD recountCategories.php --mode=subcats && foreachwikiindblist $SHARD recountCategories.php --mode=files # T170737, SHARD=s5 |
[production] |
| 12:15 |
<urbanecm> |
[urbanecm@mwmaint1002 ~]$ foreachwikiindblist s2 recountCategories.php --mode=pages && foreachwikiindblist s2 recountCategories.php --mode=subcats && foreachwikiindblist s2 recountCategories.php --mode=files # T170737 |
[production] |
| 11:46 |
<XioNoX> |
Simplify labs-in4/6 firewall filters - CR700939 |
[production] |
| 11:10 |
<topranks> |
Removing peering to AS39651 / "Com Hem AB" at AMS-IX (cr2-esams). Peer has left IX. |
[production] |
| 10:44 |
<oblivian@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 09:35 |
<mbsantos@deploy1002> |
Finished deploy [kartotherian/deploy@9f16a6b]: (no justification provided) (duration: 00m 20s) |
[production] |
| 09:35 |
<mbsantos@deploy1002> |
Started deploy [kartotherian/deploy@9f16a6b]: (no justification provided) |
[production] |
| 09:22 |
<oblivian@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 08:48 |
<volans> |
sudo systemctl start ferm.service on thanos-fe2002 (DNS query timeout) |
[production] |
| 08:34 |
<mbsantos@deploy1002> |
Finished deploy [kartotherian/deploy@9f16a6b]: (no justification provided) (duration: 00m 14s) |
[production] |
| 08:34 |
<mbsantos@deploy1002> |
Started deploy [kartotherian/deploy@9f16a6b]: (no justification provided) |
[production] |
| 07:57 |
<kart_> |
cxserver: Removed Matxin MT support and added more language support to Elia MT (T285199, T284900) |
[production] |
| 07:54 |
<kartik@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 07:49 |
<kartik@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 07:46 |
<kartik@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 07:26 |
<legoktm> |
uploaded mailman3_3.3.3-1~bpo10+6_amd64.changes on apt1001 |
[production] |
| 07:08 |
<legoktm> |
updating mailman packages on lists1001 and restarting (T285120, T280889) |
[production] |
| 06:56 |
<ryankemper> |
[WDQS] `ryankemper@wdqs1006:~$ sudo pool` |
[production] |
| 06:37 |
<ryankemper> |
[WDQS] `ryankemper@wdqs2001:~$ sudo pool` |
[production] |
| 06:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 100%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16703 and previous config saved to /var/cache/conftool/dbconfig/20210623-062819-root.json |
[production] |
| 06:13 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 75%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16702 and previous config saved to /var/cache/conftool/dbconfig/20210623-061316-root.json |
[production] |
| 05:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 50%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16701 and previous config saved to /var/cache/conftool/dbconfig/20210623-055812-root.json |
[production] |
| 05:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Start repooling db1100', diff saved to https://phabricator.wikimedia.org/P16700 and previous config saved to /var/cache/conftool/dbconfig/20210623-054252-marostegui.json |
[production] |
| 04:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1100 (re)pooling @ 25%: Repool db1100 after upgrade', diff saved to https://phabricator.wikimedia.org/P16699 and previous config saved to /var/cache/conftool/dbconfig/20210623-045217-root.json |
[production] |
| 01:04 |
<eileen> |
process-control config revision is 6a88618c3e |
[production] |
| 00:50 |
<eileen> |
civicrm revision changed from c745d4f075 to 03bead707d, config revision is 4ab72c1033 |
[production] |
| 00:40 |
<legoktm> |
uploaded new versions of flufl.bounce_4.0-1_amd64.changes hyperkitty_1.3.4-2~bpo10+4_amd64.changes mailman3_3.3.3-1~bpo10+5_amd64.changes mailman-hyperkitty_1.1.0-10~bpo10+1_amd64.changes to apt1001 |
[production] |
| 00:02 |
<Trey314159> |
reindexing Portuguese wikis on elastic@eqiad, elastic@codfw, and cloudelastic complete (T284185) |
[production] |