2501-2550 of 10000 results (60ms)
2022-06-28 ยง
13:03 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.39.0-wmf.17/extensions/WikibaseCirrusSearch/src/Hooks.php: Backport: [[gerrit:809118|Use LanguageSelectorStatementBoost instead of its plurar form (T307869)]] (duration: 03m 35s) [production]
13:02 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
13:02 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
13:02 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
13:01 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
12:57 <btullis@cumin1001> START - Cookbook sre.hosts.reimage for host stat1010.eqiad.wmnet with OS bullseye [production]
12:57 <btullis@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host stat1010.eqiad.wmnet with OS bullseye [production]
12:55 <sukhe@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp5012.eqsin.wmnet,service=ats-tls [production]
12:55 <sukhe@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp5012.eqsin.wmnet,service=varnish-fe [production]
12:55 <sukhe@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp5012.eqsin.wmnet,service=ats-be [production]
12:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180', diff saved to https://phabricator.wikimedia.org/P30580 and previous config saved to /var/cache/conftool/dbconfig/20220628-124929-marostegui.json [production]
12:48 <milimetric@deploy1002> Finished deploy [airflow-dags/analytics@68e7c64]: Deploying and enabling datahub ingestion jobs (duration: 00m 09s) [production]
12:48 <milimetric@deploy1002> Started deploy [airflow-dags/analytics@68e7c64]: Deploying and enabling datahub ingestion jobs [production]
12:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180 (T298557)', diff saved to https://phabricator.wikimedia.org/P30579 and previous config saved to /var/cache/conftool/dbconfig/20220628-123424-marostegui.json [production]
12:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1180 (T298557)', diff saved to https://phabricator.wikimedia.org/P30578 and previous config saved to /var/cache/conftool/dbconfig/20220628-122543-marostegui.json [production]
12:25 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
12:25 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
12:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T298557)', diff saved to https://phabricator.wikimedia.org/P30577 and previous config saved to /var/cache/conftool/dbconfig/20220628-122535-marostegui.json [production]
12:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P30576 and previous config saved to /var/cache/conftool/dbconfig/20220628-121030-marostegui.json [production]
11:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P30575 and previous config saved to /var/cache/conftool/dbconfig/20220628-115525-marostegui.json [production]
11:45 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
11:45 <btullis@cumin1001> START - Cookbook sre.hosts.reimage for host stat1010.eqiad.wmnet with OS bullseye [production]
11:44 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
11:44 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
11:43 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
11:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T298557)', diff saved to https://phabricator.wikimedia.org/P30574 and previous config saved to /var/cache/conftool/dbconfig/20220628-114020-marostegui.json [production]
11:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1168 (T298557)', diff saved to https://phabricator.wikimedia.org/P30573 and previous config saved to /var/cache/conftool/dbconfig/20220628-113154-marostegui.json [production]
11:31 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
11:31 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1168.eqiad.wmnet with reason: Maintenance [production]
11:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316 (T298557)', diff saved to https://phabricator.wikimedia.org/P30572 and previous config saved to /var/cache/conftool/dbconfig/20220628-113146-marostegui.json [production]
11:19 <moritzm> installing squid security updates [production]
11:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P30571 and previous config saved to /var/cache/conftool/dbconfig/20220628-111641-marostegui.json [production]
11:08 <btullis@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host stat1010.eqiad.wmnet with OS bullseye [production]
11:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P30570 and previous config saved to /var/cache/conftool/dbconfig/20220628-110136-marostegui.json [production]
10:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141 (T309311)', diff saved to https://phabricator.wikimedia.org/P30569 and previous config saved to /var/cache/conftool/dbconfig/20220628-105551-ladsgroup.json [production]
10:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316 (T298557)', diff saved to https://phabricator.wikimedia.org/P30568 and previous config saved to /var/cache/conftool/dbconfig/20220628-104631-marostegui.json [production]
10:43 <marostegui@cumin1001> dbctl commit (dc=all): 'db1161 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30567 and previous config saved to /var/cache/conftool/dbconfig/20220628-104337-root.json [production]
10:42 <marostegui@cumin1001> dbctl commit (dc=all): 'db1134 (re)pooling @ 100%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30566 and previous config saved to /var/cache/conftool/dbconfig/20220628-104252-root.json [production]
10:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141', diff saved to https://phabricator.wikimedia.org/P30565 and previous config saved to /var/cache/conftool/dbconfig/20220628-104046-ladsgroup.json [production]
10:28 <marostegui@cumin1001> dbctl commit (dc=all): 'db1161 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30564 and previous config saved to /var/cache/conftool/dbconfig/20220628-102833-root.json [production]
10:27 <marostegui@cumin1001> dbctl commit (dc=all): 'db1134 (re)pooling @ 75%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30563 and previous config saved to /var/cache/conftool/dbconfig/20220628-102748-root.json [production]
10:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141', diff saved to https://phabricator.wikimedia.org/P30562 and previous config saved to /var/cache/conftool/dbconfig/20220628-102540-ladsgroup.json [production]
10:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1096:3316 (T298557)', diff saved to https://phabricator.wikimedia.org/P30561 and previous config saved to /var/cache/conftool/dbconfig/20220628-102331-marostegui.json [production]
10:23 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
10:23 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
10:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3316 (T298557)', diff saved to https://phabricator.wikimedia.org/P30560 and previous config saved to /var/cache/conftool/dbconfig/20220628-102322-marostegui.json [production]
10:13 <moritzm> upgrading Ganeti test cluster to 3.0.2 [production]
10:13 <marostegui@cumin1001> dbctl commit (dc=all): 'db1161 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30559 and previous config saved to /var/cache/conftool/dbconfig/20220628-101329-root.json [production]
10:12 <marostegui@cumin1001> dbctl commit (dc=all): 'db1134 (re)pooling @ 50%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30558 and previous config saved to /var/cache/conftool/dbconfig/20220628-101244-root.json [production]
10:11 <btullis@cumin1001> START - Cookbook sre.hosts.reimage for host stat1010.eqiad.wmnet with OS bullseye [production]