|
2023-03-08
ยง
|
| 23:49 |
<zabe@deploy2002> |
Finished scap: T308932 (duration: 07m 15s) |
[production] |
| 23:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2166 (T329260)', diff saved to https://phabricator.wikimedia.org/P45593 and previous config saved to /var/cache/conftool/dbconfig/20230308-234534-marostegui.json |
[production] |
| 23:45 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2166.codfw.wmnet with reason: Maintenance |
[production] |
| 23:45 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2166.codfw.wmnet with reason: Maintenance |
[production] |
| 23:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2164 (T329260)', diff saved to https://phabricator.wikimedia.org/P45592 and previous config saved to /var/cache/conftool/dbconfig/20230308-234502-marostegui.json |
[production] |
| 23:42 |
<zabe@deploy2002> |
Started scap: T308932 |
[production] |
| 23:42 |
<ebernhardson@deploy2002> |
Finished deploy [airflow-dags/search@29f73a4]: update virtualenv entry_points to use relative paths (duration: 00m 14s) |
[production] |
| 23:42 |
<ebernhardson@deploy2002> |
Started deploy [airflow-dags/search@29f73a4]: update virtualenv entry_points to use relative paths |
[production] |
| 23:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2164', diff saved to https://phabricator.wikimedia.org/P45591 and previous config saved to /var/cache/conftool/dbconfig/20230308-232956-marostegui.json |
[production] |
| 23:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2164', diff saved to https://phabricator.wikimedia.org/P45590 and previous config saved to /var/cache/conftool/dbconfig/20230308-231449-marostegui.json |
[production] |
| 23:04 |
<wm-bot> |
<bd808> Updated to a7f4c36 |
[tools.bash] |
| 22:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2164 (T329260)', diff saved to https://phabricator.wikimedia.org/P45589 and previous config saved to /var/cache/conftool/dbconfig/20230308-225943-marostegui.json |
[production] |
| 22:44 |
<hashar> |
Upgrading CI Jenkins |
[production] |
| 22:42 |
<tgr> |
UTC late deploys done |
[production] |
| 22:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2164 (T329260)', diff saved to https://phabricator.wikimedia.org/P45588 and previous config saved to /var/cache/conftool/dbconfig/20230308-224044-marostegui.json |
[production] |
| 22:40 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance |
[production] |
| 22:40 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance |
[production] |
| 22:40 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2164.codfw.wmnet with reason: Maintenance |
[production] |
| 22:40 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2164.codfw.wmnet with reason: Maintenance |
[production] |
| 22:40 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2163 (T329260)', diff saved to https://phabricator.wikimedia.org/P45587 and previous config saved to /var/cache/conftool/dbconfig/20230308-224018-marostegui.json |
[production] |
| 22:39 |
<tgr@deploy2002> |
Finished scap: Backport for [[gerrit:895776|Leveling up: check if the task type is registered before increasing its edit count (T331524)]], [[gerrit:895777|Leveling up: check if the task type is registered before increasing its edit count (T331524)]] (duration: 08m 31s) |
[production] |
| 22:32 |
<tgr@deploy2002> |
tgr: Backport for [[gerrit:895776|Leveling up: check if the task type is registered before increasing its edit count (T331524)]], [[gerrit:895777|Leveling up: check if the task type is registered before increasing its edit count (T331524)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 22:31 |
<bd808> |
Live hacked user-maintainer clusterrole to work around breakage in T331572 |
[tools] |
| 22:30 |
<tgr@deploy2002> |
Started scap: Backport for [[gerrit:895776|Leveling up: check if the task type is registered before increasing its edit count (T331524)]], [[gerrit:895777|Leveling up: check if the task type is registered before increasing its edit count (T331524)]] |
[production] |
| 22:29 |
<tgr@deploy2002> |
Finished scap: Backport for [[gerrit:895778|maintenance: Adjust query builder to account for no secondary namespaces (T321983 T331412)]], [[gerrit:895779|maintenance: Adjust query builder to account for no secondary namespaces (T321983 T331412)]] (duration: 07m 43s) |
[production] |
| 22:27 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sessionstore1001.eqiad.wmnet |
[production] |
| 22:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P45586 and previous config saved to /var/cache/conftool/dbconfig/20230308-222512-marostegui.json |
[production] |
| 22:23 |
<tgr@deploy2002> |
tgr: Backport for [[gerrit:895778|maintenance: Adjust query builder to account for no secondary namespaces (T321983 T331412)]], [[gerrit:895779|maintenance: Adjust query builder to account for no secondary namespaces (T321983 T331412)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet |
[production] |
| 22:21 |
<tgr@deploy2002> |
Started scap: Backport for [[gerrit:895778|maintenance: Adjust query builder to account for no secondary namespaces (T321983 T331412)]], [[gerrit:895779|maintenance: Adjust query builder to account for no secondary namespaces (T321983 T331412)]] |
[production] |
| 22:21 |
<eevans@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host sessionstore1001.eqiad.wmnet |
[production] |
| 22:20 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sessionstore1001.eqiad.wmnet |
[production] |
| 22:19 |
<wm-bot> |
<lucaswerkmeister> deployed 8da3525baf (fix lowercase item ID in portuguese-noun-biform) |
[tools.lexeme-forms] |
| 22:16 |
<wm-bot> |
<legoktm> Updated from 1b0d08e to c77defe |
[tools.stats] |
| 22:12 |
<eevans@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host sessionstore1001.eqiad.wmnet |
[production] |
| 22:10 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2163', diff saved to https://phabricator.wikimedia.org/P45585 and previous config saved to /var/cache/conftool/dbconfig/20230308-221006-marostegui.json |
[production] |
| 22:09 |
<kindrobot> |
hand off backport window UTC late to tgr for self-service |
[production] |
| 22:08 |
<wm-bot> |
<lucaswerkmeister> deployed de17c6bdf6 (fix hindustani-verb-additive-causative-double-ur label) |
[tools.lexeme-forms] |
| 22:07 |
<kindrobot@deploy2002> |
Finished scap: Backport for [[gerrit:895833|Enable new Linter UI for namespace, tag and template for all wikis (T299612)]] (duration: 09m 36s) |
[production] |
| 22:01 |
<wm-bot> |
<lucaswerkmeister> deployed a99078e1c5 (hindustani-verb-additive-causative-double templates) |
[tools.lexeme-forms] |
| 21:59 |
<kindrobot@deploy2002> |
sbailey and kindrobot: Backport for [[gerrit:895833|Enable new Linter UI for namespace, tag and template for all wikis (T299612)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 21:57 |
<kindrobot@deploy2002> |
Started scap: Backport for [[gerrit:895833|Enable new Linter UI for namespace, tag and template for all wikis (T299612)]] |
[production] |
| 21:55 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2163 (T329260)', diff saved to https://phabricator.wikimedia.org/P45584 and previous config saved to /var/cache/conftool/dbconfig/20230308-215500-marostegui.json |
[production] |
| 21:54 |
<kindrobot@deploy2002> |
Finished scap: Backport for [[gerrit:895297|Switch order of "Add topic" and language dropdown (T267444)]], [[gerrit:895375|Release DiscussionTools on mobile on enwiki (T328942)]], [[gerrit:888804|Enable history page visual diffs everywhere except Wikipedias and Wiktionaries (T314588)]] (duration: 07m 49s) |
[production] |
| 21:48 |
<kindrobot@deploy2002> |
kemayo and kindrobot and esanders: Backport for [[gerrit:895297|Switch order of "Add topic" and language dropdown (T267444)]], [[gerrit:895375|Release DiscussionTools on mobile on enwiki (T328942)]], [[gerrit:888804|Enable history page visual diffs everywhere except Wikipedias and Wiktionaries (T314588)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.cod |
[production] |
| 21:46 |
<kindrobot@deploy2002> |
Started scap: Backport for [[gerrit:895297|Switch order of "Add topic" and language dropdown (T267444)]], [[gerrit:895375|Release DiscussionTools on mobile on enwiki (T328942)]], [[gerrit:888804|Enable history page visual diffs everywhere except Wikipedias and Wiktionaries (T314588)]] |
[production] |
| 21:45 |
<bd808> |
maintain-kubeusers container in CrashLoopBackoff, investigating |
[admin] |
| 21:37 |
<eevans@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sessionstore1001.eqiad.wmnet |
[production] |
| 21:31 |
<eevans@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host sessionstore1001.eqiad.wmnet |
[production] |
| 21:30 |
<kindrobot@deploy2002> |
kemayo and kindrobot and esanders: Backport for [[gerrit:888804|Enable history page visual diffs everywhere except Wikipedias and Wiktionaries (T314588)]], [[gerrit:895375|Release DiscussionTools on mobile on enwiki (T328942)]], [[gerrit:895297|Switch order of "Add topic" and language dropdown (T267444)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqi |
[production] |
| 21:29 |
<kindrobot@deploy2002> |
Started scap: Backport for [[gerrit:888804|Enable history page visual diffs everywhere except Wikipedias and Wiktionaries (T314588)]], [[gerrit:895375|Release DiscussionTools on mobile on enwiki (T328942)]], [[gerrit:895297|Switch order of "Add topic" and language dropdown (T267444)]] |
[production] |