|
2021-06-22
§
|
| 14:57 |
<dcausse@deploy1002> |
Finished deploy [wdqs/wdqs@b082ccc]: wdqs 0.3.74 (duration: 13m 26s) |
[production] |
| 14:43 |
<dcausse@deploy1002> |
Started deploy [wdqs/wdqs@b082ccc]: wdqs 0.3.74 |
[production] |
| 14:37 |
<XioNoX> |
start updating analytics firewall rules to capirca generated ones on cr2-eqiad - T279429 |
[production] |
| 14:35 |
<hoo> |
Updated the Wikidata property suggester with data from the 2021-05-31 JSON dump (with pre-applied T132839 workarounds) |
[production] |
| 14:01 |
<XioNoX> |
start updating analytics firewall rules to capirca generated ones on cr1-eqiad - T279429 |
[production] |
| 13:49 |
<kormat> |
disabling puppet on A:db-all for T285079 |
[production] |
| 13:38 |
<urbanecm> |
[urbanecm@mwmaint1002 /srv/mediawiki-staging/php]$ mwscript extensions/GrowthExperiments/maintenance/initWikiConfig.php --wiki=nlwiki --phab=T285254 # T285254 |
[production] |
| 13:37 |
<urbanecm> |
[urbanecm@mwmaint1002 /srv/mediawiki-staging]$ mwscript extensions/WikimediaMaintenance/createExtensionTables.php --wiki=nlwiki growthexperiments # T285254 |
[production] |
| 13:33 |
<lucaswerkmeister-wmde@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:700905|Correctly enable Vector language switcher treatment A/B test (T269093)]] (duration: 00m 57s) |
[production] |
| 13:29 |
<urbanecm> |
urbanecm@mwmaint1002:~$ foreachwikiindblist growthexperiments extensions/WikimediaMaintenance/createExtensionTables.php growthexperiments # T266913 |
[production] |
| 13:29 |
<Trey314159> |
reindexing German wikis on elastic@eqiad, elastic@codfw, and cloudelastic complete (T284185) |
[production] |
| 12:04 |
<Lucas_WMDE> |
backport+config window done |
[production] |
| 12:03 |
<lucaswerkmeister-wmde@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:700705|Enable new Vector Languages-in-header feature & AB test for pilot wikis (T269093)]] (duration: 00m 56s) |
[production] |
| 11:58 |
<Lucas_WMDE> |
lucaswerkmeister-wmde@mwdebug1001:~$ sudo -u mwdeploy sh -c 'rm /srv/mediawiki/php-1.37.0-wmf.1/cache/l10n/l10n_cache-*.cdb && rmdir /srv/mediawiki/php-1.37.0-wmf.1/cache/l10n && rmdir /srv/mediawiki/php-1.37.0-wmf.1/cache && rmdir /srv/mediawiki/php-1.37.0-wmf.1' # per comments in T157030 and similar tasks |
[production] |
| 11:54 |
<lucaswerkmeister-wmde@deploy1002> |
Synchronized php-1.37.0-wmf.9/extensions/UniversalLanguageSelector/: Backport: [[gerrit:700730|launchULS: Add context to interface.language.change hook (T280770)]] (duration: 00m 57s) |
[production] |
| 11:35 |
<moritzm> |
installing fluidsynth security updates |
[production] |
| 11:17 |
<lucaswerkmeister-wmde@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:698041|enwiki: Remove 'collectionsaveascommunitypage' from the 'autoconfirmed' user group (T283523)]] (duration: 00m 56s) |
[production] |
| 11:06 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'db1123 (re)pooling @ 100%: reimaged to buster T283131', diff saved to https://phabricator.wikimedia.org/P16691 and previous config saved to /var/cache/conftool/dbconfig/20210622-110619-kormat.json |
[production] |
| 10:51 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'db1123 (re)pooling @ 75%: reimaged to buster T283131', diff saved to https://phabricator.wikimedia.org/P16690 and previous config saved to /var/cache/conftool/dbconfig/20210622-105115-kormat.json |
[production] |
| 10:36 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'db1123 (re)pooling @ 50%: reimaged to buster T283131', diff saved to https://phabricator.wikimedia.org/P16689 and previous config saved to /var/cache/conftool/dbconfig/20210622-103612-kormat.json |
[production] |
| 10:21 |
<kormat@cumin1001> |
dbctl commit (dc=all): 'db1123 (re)pooling @ 25%: reimaged to buster T283131', diff saved to https://phabricator.wikimedia.org/P16688 and previous config saved to /var/cache/conftool/dbconfig/20210622-102108-kormat.json |
[production] |
| 09:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 100%: Repool db1166 after upgrade', diff saved to https://phabricator.wikimedia.org/P16687 and previous config saved to /var/cache/conftool/dbconfig/20210622-094019-root.json |
[production] |
| 09:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 75%: Repool db1166 after upgrade', diff saved to https://phabricator.wikimedia.org/P16686 and previous config saved to /var/cache/conftool/dbconfig/20210622-092515-root.json |
[production] |
| 09:20 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1169 (re)pooling @ 100%: Repool db1169 after schema change', diff saved to https://phabricator.wikimedia.org/P16685 and previous config saved to /var/cache/conftool/dbconfig/20210622-092056-root.json |
[production] |
| 09:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 50%: Repool db1166 after upgrade', diff saved to https://phabricator.wikimedia.org/P16684 and previous config saved to /var/cache/conftool/dbconfig/20210622-091012-root.json |
[production] |
| 09:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1169 (re)pooling @ 75%: Repool db1169 after schema change', diff saved to https://phabricator.wikimedia.org/P16683 and previous config saved to /var/cache/conftool/dbconfig/20210622-090552-root.json |
[production] |
| 08:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 25%: Repool db1166 after upgrade', diff saved to https://phabricator.wikimedia.org/P16682 and previous config saved to /var/cache/conftool/dbconfig/20210622-085508-root.json |
[production] |
| 08:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1169 (re)pooling @ 50%: Repool db1169 after schema change', diff saved to https://phabricator.wikimedia.org/P16681 and previous config saved to /var/cache/conftool/dbconfig/20210622-085049-root.json |
[production] |
| 08:49 |
<marostegui> |
Upgrade db1166 |
[production] |
| 08:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1166', diff saved to https://phabricator.wikimedia.org/P16680 and previous config saved to /var/cache/conftool/dbconfig/20210622-084915-marostegui.json |
[production] |
| 08:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1169 (re)pooling @ 25%: Repool db1169 after schema change', diff saved to https://phabricator.wikimedia.org/P16679 and previous config saved to /var/cache/conftool/dbconfig/20210622-083545-root.json |
[production] |
| 07:53 |
<joe> |
uploaded wmf-certificates package to buster-wikimedia/main, T284417 |
[production] |
| 07:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1169 T283499', diff saved to https://phabricator.wikimedia.org/P16678 and previous config saved to /var/cache/conftool/dbconfig/20210622-072828-marostegui.json |
[production] |
| 06:43 |
<dcausse> |
repool wdqs1005 |
[production] |
| 06:16 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1100.eqiad.wmnet with reason: REIMAGE |
[production] |
| 06:14 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db1100.eqiad.wmnet with reason: REIMAGE |
[production] |
| 05:06 |
<marostegui> |
Stop replication on old s5 master ( db1100) - T284529 |
[production] |
| 05:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool old master running 10.1 T284529', diff saved to https://phabricator.wikimedia.org/P16677 and previous config saved to /var/cache/conftool/dbconfig/20210622-050602-marostegui.json |
[production] |
| 05:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Promote db1130 to s5 master and set section read-write T284529', diff saved to https://phabricator.wikimedia.org/P16676 and previous config saved to /var/cache/conftool/dbconfig/20210622-050123-root.json |
[production] |
| 05:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Set s5 eqiad as read-only for maintenance - T284529', diff saved to https://phabricator.wikimedia.org/P16675 and previous config saved to /var/cache/conftool/dbconfig/20210622-050036-root.json |
[production] |
| 05:00 |
<marostegui> |
Starting s5 eqiad failover from db1100 to db1130 - T284529 |
[production] |
| 04:20 |
<marostegui> |
Start topology changes for s5 switchover T284529 |
[production] |
| 04:19 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 24 hosts with reason: Master switchover s5 T284529 |
[production] |
| 04:19 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 1:00:00 on 24 hosts with reason: Master switchover s5 T284529 |
[production] |
| 04:11 |
<eileen> |
process-control config revision is 4ab72c1033 |
[production] |
| 01:02 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on ganeti2026.codfw.wmnet with reason: REIMAGE |
[production] |
| 01:00 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti2026.codfw.wmnet with reason: REIMAGE |
[production] |
| 00:35 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on ganeti2025.codfw.wmnet with reason: REIMAGE |
[production] |
| 00:33 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti2025.codfw.wmnet with reason: REIMAGE |
[production] |