|
2022-10-13
ยง
|
| 14:36 |
<sukhe@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp4045.ulsfo.wmnet,service=varnish-fe |
[production] |
| 14:36 |
<sukhe@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp4045.ulsfo.wmnet,service=ats-tls |
[production] |
| 14:36 |
<sukhe@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp4045.ulsfo.wmnet,service=ats-be |
[production] |
| 14:36 |
<sukhe@puppetmaster1001> |
conftool action : set/weight=1; selector: name=cp4045.ulsfo.wmnet,service=varnish-fe |
[production] |
| 14:36 |
<sukhe@puppetmaster1001> |
conftool action : set/weight=1; selector: name=cp4045.ulsfo.wmnet,service=ats-tls |
[production] |
| 14:36 |
<sukhe@puppetmaster1001> |
conftool action : set/weight=100; selector: name=cp4045.ulsfo.wmnet,service=ats-be |
[production] |
| 14:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2150 (T318950)', diff saved to https://phabricator.wikimedia.org/P35482 and previous config saved to /var/cache/conftool/dbconfig/20221013-142730-ladsgroup.json |
[production] |
| 14:26 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.provision for host sretest2001.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 14:12 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2150', diff saved to https://phabricator.wikimedia.org/P35481 and previous config saved to /var/cache/conftool/dbconfig/20221013-141224-ladsgroup.json |
[production] |
| 14:06 |
<sukhe> |
running puppet/utils/pcc_update_facts.py to update nodes |
[production] |
| 14:06 |
<urbanecm@deploy1002> |
backport aborted: (duration: 00m 04s) |
[production] |
| 14:04 |
<Lucas_WMDE> |
UTC afternoon backport+config window done |
[production] |
| 14:04 |
<lucaswerkmeister-wmde@deploy1002> |
Finished scap: Backport for [[gerrit:842456|Revert "GrowthExperiments: enable the Vue version of the mentee overview in all wikis"]] (duration: 05m 49s) |
[production] |
| 13:59 |
<lucaswerkmeister-wmde@deploy1002> |
lucaswerkmeister-wmde and trainbranchbot: Backport for [[gerrit:842456|Revert "GrowthExperiments: enable the Vue version of the mentee overview in all wikis"]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 13:58 |
<lucaswerkmeister-wmde@deploy1002> |
Started scap: Backport for [[gerrit:842456|Revert "GrowthExperiments: enable the Vue version of the mentee overview in all wikis"]] |
[production] |
| 13:57 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2150', diff saved to https://phabricator.wikimedia.org/P35480 and previous config saved to /var/cache/conftool/dbconfig/20221013-135718-ladsgroup.json |
[production] |
| 13:56 |
<lucaswerkmeister-wmde@deploy1002> |
Sync cancelled. |
[production] |
| 13:53 |
<gmodena@deploy1002> |
helmfile [staging] DONE helmfile.d/services/eventgate-analytics: apply |
[production] |
| 13:52 |
<gmodena@deploy1002> |
helmfile [staging] START helmfile.d/services/eventgate-analytics: apply |
[production] |
| 13:47 |
<lucaswerkmeister-wmde@deploy1002> |
lucaswerkmeister-wmde and sgimeno: Backport for [[gerrit:842356|GrowthExperiments: enable the Vue version of the mentee overview in all wikis (T300532)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 13:47 |
<lucaswerkmeister-wmde@deploy1002> |
Started scap: Backport for [[gerrit:842356|GrowthExperiments: enable the Vue version of the mentee overview in all wikis (T300532)]] |
[production] |
| 13:45 |
<mlitn@deploy1002> |
Finished scap: Backport for [[gerrit:842373|Commons files can have thumbnails too]] (duration: 04m 53s) |
[production] |
| 13:42 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2150 (T318950)', diff saved to https://phabricator.wikimedia.org/P35479 and previous config saved to /var/cache/conftool/dbconfig/20221013-134211-ladsgroup.json |
[production] |
| 13:41 |
<mlitn@deploy1002> |
mlitn and mlitn: Backport for [[gerrit:842373|Commons files can have thumbnails too]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet |
[production] |
| 13:40 |
<mlitn@deploy1002> |
Started scap: Backport for [[gerrit:842373|Commons files can have thumbnails too]] |
[production] |
| 13:39 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2150 (T318950)', diff saved to https://phabricator.wikimedia.org/P35478 and previous config saved to /var/cache/conftool/dbconfig/20221013-133953-ladsgroup.json |
[production] |
| 13:39 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance |
[production] |
| 13:39 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance |
[production] |
| 13:39 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122 (T318950)', diff saved to https://phabricator.wikimedia.org/P35477 and previous config saved to /var/cache/conftool/dbconfig/20221013-133931-ladsgroup.json |
[production] |
| 13:27 |
<mlitn@deploy1002> |
Finished scap: Backport for [[gerrit:842374|Commons files can have thumbnails too]] (duration: 05m 15s) |
[production] |
| 13:24 |
<jgiannelos@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 13:24 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P35473 and previous config saved to /var/cache/conftool/dbconfig/20221013-132425-ladsgroup.json |
[production] |
| 13:24 |
<jgiannelos@deploy1002> |
helmfile [codfw] START helmfile.d/services/mobileapps: apply |
[production] |
| 13:22 |
<mlitn@deploy1002> |
mlitn and mlitn: Backport for [[gerrit:842374|Commons files can have thumbnails too]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 13:22 |
<mlitn@deploy1002> |
Started scap: Backport for [[gerrit:842374|Commons files can have thumbnails too]] |
[production] |
| 13:16 |
<jgiannelos@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 13:16 |
<jgiannelos@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mobileapps: apply |
[production] |
| 13:15 |
<jgiannelos@deploy1002> |
helmfile [staging] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 13:15 |
<jgiannelos@deploy1002> |
helmfile [staging] START helmfile.d/services/mobileapps: apply |
[production] |
| 13:09 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P35472 and previous config saved to /var/cache/conftool/dbconfig/20221013-130918-ladsgroup.json |
[production] |
| 12:57 |
<jgiannelos@deploy1002> |
helmfile [staging] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 12:56 |
<jgiannelos@deploy1002> |
helmfile [staging] START helmfile.d/services/mobileapps: apply |
[production] |
| 12:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122 (T318950)', diff saved to https://phabricator.wikimedia.org/P35471 and previous config saved to /var/cache/conftool/dbconfig/20221013-125412-ladsgroup.json |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2122 (T318950)', diff saved to https://phabricator.wikimedia.org/P35470 and previous config saved to /var/cache/conftool/dbconfig/20221013-125154-ladsgroup.json |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2122.codfw.wmnet with reason: Maintenance |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2122.codfw.wmnet with reason: Maintenance |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121 (T318950)', diff saved to https://phabricator.wikimedia.org/P35469 and previous config saved to /var/cache/conftool/dbconfig/20221013-125133-ladsgroup.json |
[production] |
| 12:45 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "sync data - jbond@cumin1001" |
[production] |
| 12:43 |
<jbond@cumin1001> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "sync data - jbond@cumin1001" |
[production] |
| 12:37 |
<jbond@cumin1001> |
END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "sync data - jbond@cumin1001" |
[production] |