4801-4850 of 10000 results (41ms)
2022-01-27 ยง
15:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1135', diff saved to https://phabricator.wikimedia.org/P19491 and previous config saved to /var/cache/conftool/dbconfig/20220127-155726-marostegui.json [production]
15:57 <brennen@deploy1002> Synchronized php: group1 wikis to 1.38.0-wmf.19 refs T293960 (duration: 00m 51s) [production]
15:56 <brennen@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.38.0-wmf.19 refs T293960 [production]
15:54 <otto@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: sync on canary [production]
15:53 <otto@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: sync on production [production]
15:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P19490 and previous config saved to /var/cache/conftool/dbconfig/20220127-155244-marostegui.json [production]
15:52 <otto@deploy1002> helmfile [codfw] START helmfile.d/services/eventgate-analytics: apply on production [production]
15:52 <otto@deploy1002> helmfile [codfw] START helmfile.d/services/eventgate-analytics: apply on canary [production]
15:52 <brennen> train 1.38.0-wmf.19 (T293960): no current blockers; rolling train forward to group1 before log triage meeting [production]
15:45 <otto@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-analytics: sync on production [production]
15:45 <otto@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-analytics: sync on canary [production]
15:45 <otto@deploy1002> helmfile [staging] START helmfile.d/services/eventgate-analytics: apply on canary [production]
15:45 <otto@deploy1002> helmfile [staging] START helmfile.d/services/eventgate-analytics: apply on production [production]
15:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1135', diff saved to https://phabricator.wikimedia.org/P19489 and previous config saved to /var/cache/conftool/dbconfig/20220127-154222-marostegui.json [production]
15:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P19488 and previous config saved to /var/cache/conftool/dbconfig/20220127-153739-marostegui.json [production]
15:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1135 (T298559)', diff saved to https://phabricator.wikimedia.org/P19487 and previous config saved to /var/cache/conftool/dbconfig/20220127-152717-marostegui.json [production]
15:22 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T285149)', diff saved to https://phabricator.wikimedia.org/P19486 and previous config saved to /var/cache/conftool/dbconfig/20220127-152235-marostegui.json [production]
15:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1135 (T298559)', diff saved to https://phabricator.wikimedia.org/P19485 and previous config saved to /var/cache/conftool/dbconfig/20220127-151709-marostegui.json [production]
15:17 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1135.eqiad.wmnet with reason: Maintenance [production]
15:17 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1135.eqiad.wmnet with reason: Maintenance [production]
15:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T298559)', diff saved to https://phabricator.wikimedia.org/P19484 and previous config saved to /var/cache/conftool/dbconfig/20220127-151701-marostegui.json [production]
15:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance es1023 (T300006)', diff saved to https://phabricator.wikimedia.org/P19483 and previous config saved to /var/cache/conftool/dbconfig/20220127-151032-ladsgroup.json [production]
15:09 <dcausse@deploy1002> helmfile [eqiad] DONE helmfile.d/services/eventgate-main: sync on production [production]
15:08 <dcausse@deploy1002> helmfile [eqiad] DONE helmfile.d/services/eventgate-main: sync on canary [production]
15:07 <dcausse@deploy1002> helmfile [eqiad] START helmfile.d/services/eventgate-main: apply on production [production]
15:07 <dcausse@deploy1002> helmfile [eqiad] START helmfile.d/services/eventgate-main: apply on canary [production]
15:04 <dcausse@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventgate-main: sync on production [production]
15:04 <dcausse@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventgate-main: sync on canary [production]
15:03 <dcausse@deploy1002> helmfile [codfw] START helmfile.d/services/eventgate-main: apply on canary [production]
15:03 <dcausse@deploy1002> helmfile [codfw] START helmfile.d/services/eventgate-main: apply on production [production]
15:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P19482 and previous config saved to /var/cache/conftool/dbconfig/20220127-150156-marostegui.json [production]
14:59 <mmandere@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host doh6002.wikimedia.org [production]
14:58 <dcausse@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-main: sync on production [production]
14:57 <dcausse@deploy1002> helmfile [staging] DONE helmfile.d/services/eventgate-main: apply on canary [production]
14:57 <dcausse@deploy1002> helmfile [staging] START helmfile.d/services/eventgate-main: apply on production [production]
14:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance es1023', diff saved to https://phabricator.wikimedia.org/P19481 and previous config saved to /var/cache/conftool/dbconfig/20220127-145527-ladsgroup.json [production]
14:54 <ottomata> continuing deployments of eventgate-main and eventgate-analytics to pick up CA cert changes - T296064 (also deploying eventgate-main for a schema repo bump for search) [production]
14:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P19480 and previous config saved to /var/cache/conftool/dbconfig/20220127-144652-marostegui.json [production]
14:46 <mmandere@cumin1001> START - Cookbook sre.ganeti.makevm for new host doh6002.wikimedia.org [production]
14:40 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.addnode (exit_code=0) for new host ganeti1028.eqiad.wmnet to ganeti01.svc.eqiad.wmnet [production]
14:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance es1023', diff saved to https://phabricator.wikimedia.org/P19479 and previous config saved to /var/cache/conftool/dbconfig/20220127-144022-ladsgroup.json [production]
14:39 <moritzm> added ganeti1028 to Ganeti eqiad cluster T293909 [production]
14:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T298559)', diff saved to https://phabricator.wikimedia.org/P19478 and previous config saved to /var/cache/conftool/dbconfig/20220127-143147-marostegui.json [production]
14:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1134 (T298559)', diff saved to https://phabricator.wikimedia.org/P19477 and previous config saved to /var/cache/conftool/dbconfig/20220127-142841-marostegui.json [production]
14:28 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
14:28 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
14:28 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
14:28 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
14:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163 (T298559)', diff saved to https://phabricator.wikimedia.org/P19476 and previous config saved to /var/cache/conftool/dbconfig/20220127-142829-marostegui.json [production]
14:25 <jmm@cumin2002> START - Cookbook sre.ganeti.addnode for new host ganeti1028.eqiad.wmnet to ganeti01.svc.eqiad.wmnet [production]