|
2022-10-13
ยง
|
| 13:39 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance |
[production] |
| 13:39 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2150.codfw.wmnet with reason: Maintenance |
[production] |
| 13:39 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122 (T318950)', diff saved to https://phabricator.wikimedia.org/P35477 and previous config saved to /var/cache/conftool/dbconfig/20221013-133931-ladsgroup.json |
[production] |
| 13:27 |
<mlitn@deploy1002> |
Finished scap: Backport for [[gerrit:842374|Commons files can have thumbnails too]] (duration: 05m 15s) |
[production] |
| 13:24 |
<jgiannelos@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 13:24 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P35473 and previous config saved to /var/cache/conftool/dbconfig/20221013-132425-ladsgroup.json |
[production] |
| 13:24 |
<jgiannelos@deploy1002> |
helmfile [codfw] START helmfile.d/services/mobileapps: apply |
[production] |
| 13:22 |
<mlitn@deploy1002> |
mlitn and mlitn: Backport for [[gerrit:842374|Commons files can have thumbnails too]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 13:22 |
<mlitn@deploy1002> |
Started scap: Backport for [[gerrit:842374|Commons files can have thumbnails too]] |
[production] |
| 13:16 |
<jgiannelos@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 13:16 |
<jgiannelos@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mobileapps: apply |
[production] |
| 13:15 |
<jgiannelos@deploy1002> |
helmfile [staging] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 13:15 |
<jgiannelos@deploy1002> |
helmfile [staging] START helmfile.d/services/mobileapps: apply |
[production] |
| 13:09 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P35472 and previous config saved to /var/cache/conftool/dbconfig/20221013-130918-ladsgroup.json |
[production] |
| 12:57 |
<jgiannelos@deploy1002> |
helmfile [staging] DONE helmfile.d/services/mobileapps: apply |
[production] |
| 12:56 |
<jgiannelos@deploy1002> |
helmfile [staging] START helmfile.d/services/mobileapps: apply |
[production] |
| 12:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2122 (T318950)', diff saved to https://phabricator.wikimedia.org/P35471 and previous config saved to /var/cache/conftool/dbconfig/20221013-125412-ladsgroup.json |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2122 (T318950)', diff saved to https://phabricator.wikimedia.org/P35470 and previous config saved to /var/cache/conftool/dbconfig/20221013-125154-ladsgroup.json |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2122.codfw.wmnet with reason: Maintenance |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2122.codfw.wmnet with reason: Maintenance |
[production] |
| 12:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121 (T318950)', diff saved to https://phabricator.wikimedia.org/P35469 and previous config saved to /var/cache/conftool/dbconfig/20221013-125133-ladsgroup.json |
[production] |
| 12:45 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "sync data - jbond@cumin1001" |
[production] |
| 12:43 |
<jbond@cumin1001> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "sync data - jbond@cumin1001" |
[production] |
| 12:37 |
<jbond@cumin1001> |
END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "sync data - jbond@cumin1001" |
[production] |
| 12:37 |
<jbond@cumin1001> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "sync data - jbond@cumin1001" |
[production] |
| 12:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121', diff saved to https://phabricator.wikimedia.org/P35468 and previous config saved to /var/cache/conftool/dbconfig/20221013-123626-ladsgroup.json |
[production] |
| 12:21 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121', diff saved to https://phabricator.wikimedia.org/P35467 and previous config saved to /var/cache/conftool/dbconfig/20221013-122120-ladsgroup.json |
[production] |
| 12:06 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2121 (T318950)', diff saved to https://phabricator.wikimedia.org/P35466 and previous config saved to /var/cache/conftool/dbconfig/20221013-120613-ladsgroup.json |
[production] |
| 12:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2121 (T318950)', diff saved to https://phabricator.wikimedia.org/P35465 and previous config saved to /var/cache/conftool/dbconfig/20221013-120356-ladsgroup.json |
[production] |
| 12:03 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance |
[production] |
| 12:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db2121.codfw.wmnet with reason: Maintenance |
[production] |
| 12:03 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2120 (T318950)', diff saved to https://phabricator.wikimedia.org/P35464 and previous config saved to /var/cache/conftool/dbconfig/20221013-120334-ladsgroup.json |
[production] |
| 12:02 |
<moritzm> |
restarting FPM/Apache on mediawiki canaries to pick up new curl |
[production] |
| 11:58 |
<moritzm> |
installing curl security updates on buster |
[production] |
| 11:48 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2120', diff saved to https://phabricator.wikimedia.org/P35462 and previous config saved to /var/cache/conftool/dbconfig/20221013-114827-ladsgroup.json |
[production] |
| 11:46 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of kubetcd1005.eqiad.wmnet to plain |
[production] |
| 11:46 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.changedisk for changing disk type of kubetcd1005.eqiad.wmnet to plain |
[production] |
| 11:40 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of kubetcd1005.eqiad.wmnet to drbd |
[production] |
| 11:37 |
<ayounsi@deploy1002> |
Finished deploy [netbox/deploy@7bbf659]: Deploy ntc-netbox-plugin-metrics-ext and updated wheels (duration: 04m 41s) |
[production] |
| 11:33 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2120', diff saved to https://phabricator.wikimedia.org/P35461 and previous config saved to /var/cache/conftool/dbconfig/20221013-113320-ladsgroup.json |
[production] |
| 11:33 |
<ayounsi@deploy1002> |
Started deploy [netbox/deploy@7bbf659]: Deploy ntc-netbox-plugin-metrics-ext and updated wheels |
[production] |
| 11:31 |
<ayounsi@deploy1002> |
Finished deploy [netbox/deploy@7bbf659]: Deploy ntc-netbox-plugin-metrics-ext and updated wheels (duration: 03m 12s) |
[production] |
| 11:30 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.changedisk for changing disk type of kubetcd1005.eqiad.wmnet to drbd |
[production] |
| 11:28 |
<ayounsi@deploy1002> |
Started deploy [netbox/deploy@7bbf659]: Deploy ntc-netbox-plugin-metrics-ext and updated wheels |
[production] |
| 11:28 |
<ayounsi@deploy1002> |
Finished deploy [netbox/deploy@7bbf659]: Deploy ntc-netbox-plugin-metrics-ext and updated wheels (duration: 03m 34s) |
[production] |
| 11:27 |
<Lucas_WMDE> |
11:18 ladsgroup@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db2120 (T318950)', diff saved to https://phabricator.wikimedia.org/P35460 and previous config saved to /var/cache/conftool/dbconfig/20221013-111814-ladsgroup.json |
[production] |
| 11:27 |
<Lucas_WMDE> |
11:15 ladsgroup@cumin1001 dbctl commit (dc=all): 'Depooling db2120 (T318950)', diff saved to https://phabricator.wikimedia.org/P35458 and previous config saved to /var/cache/conftool/dbconfig/20221013-111556-ladsgroup.json |
[production] |
| 11:27 |
<Lucas_WMDE> |
11:15 ladsgroup@cumin1001 END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2120.codfw.wmnet with reason: Maintenance |
[production] |
| 11:27 |
<Lucas_WMDE> |
11:15 ladsgroup@cumin1001 START - Cookbook sre.hosts.downtime for 6:00:00 on db2120.codfw.wmnet with reason: Maintenance |
[production] |
| 11:27 |
<Lucas_WMDE> |
11:15 ladsgroup@cumin1001 dbctl commit (dc=all): 'Repooling after maintenance db2108 (T318950)', diff saved to https://phabricator.wikimedia.org/P35457 and previous config saved to /var/cache/conftool/dbconfig/20221013-111534-ladsgroup.json |
[production] |