4151-4200 of 10000 results (101ms)
2024-04-04 ยง
15:47 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1191.eqiad.wmnet with reason: Maintenance [production]
15:47 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174 (T360332)', diff saved to https://phabricator.wikimedia.org/P59522 and previous config saved to /var/cache/conftool/dbconfig/20240404-154707-arnaudb.json [production]
15:45 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2213 (re)pooling @ 25%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59521 and previous config saved to /var/cache/conftool/dbconfig/20240404-154549-arnaudb.json [production]
15:45 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2207 (re)pooling @ 25%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59520 and previous config saved to /var/cache/conftool/dbconfig/20240404-154535-arnaudb.json [production]
15:45 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.debug (exit_code=0) for Netbox circuit ID 108 [production]
15:45 <ayounsi@cumin1002> START - Cookbook sre.network.debug for Netbox circuit ID 108 [production]
15:38 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1206', diff saved to https://phabricator.wikimedia.org/P59519 and previous config saved to /var/cache/conftool/dbconfig/20240404-153850-marostegui.json [production]
15:36 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1203 (T355609)', diff saved to https://phabricator.wikimedia.org/P59518 and previous config saved to /var/cache/conftool/dbconfig/20240404-153626-marostegui.json [production]
15:36 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db2214.codfw.wmnet with reason: depooled, see T361851 [production]
15:36 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1203.eqiad.wmnet with reason: Maintenance [production]
15:36 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db2214.codfw.wmnet with reason: depooled, see T361851 [production]
15:36 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1203.eqiad.wmnet with reason: Maintenance [production]
15:36 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193 (T355609)', diff saved to https://phabricator.wikimedia.org/P59517 and previous config saved to /var/cache/conftool/dbconfig/20240404-153603-marostegui.json [production]
15:32 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P59516 and previous config saved to /var/cache/conftool/dbconfig/20240404-153200-arnaudb.json [production]
15:30 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2213 (re)pooling @ 16%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59515 and previous config saved to /var/cache/conftool/dbconfig/20240404-153043-arnaudb.json [production]
15:30 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2207 (re)pooling @ 16%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59514 and previous config saved to /var/cache/conftool/dbconfig/20240404-153030-arnaudb.json [production]
15:30 <sukhe@cumin2002> dbctl commit (dc=all): 'depool db2214', diff saved to https://phabricator.wikimedia.org/P59513 and previous config saved to /var/cache/conftool/dbconfig/20240404-153023-sukhe.json [production]
15:23 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1206', diff saved to https://phabricator.wikimedia.org/P59512 and previous config saved to /var/cache/conftool/dbconfig/20240404-152341-marostegui.json [production]
15:23 <jgiannelos@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply [production]
15:22 <jgiannelos@deploy1002> helmfile [eqiad] START helmfile.d/services/mobileapps: apply [production]
15:22 <logmsgbot> dreamyjazz Deployed security patch for T361296 [production]
15:20 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P59511 and previous config saved to /var/cache/conftool/dbconfig/20240404-152056-marostegui.json [production]
15:18 <jgiannelos@deploy1002> helmfile [codfw] DONE helmfile.d/services/mobileapps: apply [production]
15:16 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P59510 and previous config saved to /var/cache/conftool/dbconfig/20240404-151653-arnaudb.json [production]
15:16 <jgiannelos@deploy1002> helmfile [codfw] START helmfile.d/services/mobileapps: apply [production]
15:16 <jgiannelos@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
15:16 <jgiannelos@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
15:15 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2113 (re)pooling @ 100%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59509 and previous config saved to /var/cache/conftool/dbconfig/20240404-151547-arnaudb.json [production]
15:15 <jgiannelos@deploy1002> helmfile [codfw] START helmfile.d/services/mobileapps: apply [production]
15:15 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2213 (re)pooling @ 8%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59508 and previous config saved to /var/cache/conftool/dbconfig/20240404-151537-arnaudb.json [production]
15:15 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2207 (re)pooling @ 8%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59507 and previous config saved to /var/cache/conftool/dbconfig/20240404-151524-arnaudb.json [production]
15:11 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2107 (re)pooling @ 100%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59506 and previous config saved to /var/cache/conftool/dbconfig/20240404-151123-arnaudb.json [production]
15:10 <herron> beginning rolling hardware upgrades on titan200[12] T361229 [production]
15:08 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1206 (T356166)', diff saved to https://phabricator.wikimedia.org/P59505 and previous config saved to /var/cache/conftool/dbconfig/20240404-150833-marostegui.json [production]
15:08 <jgiannelos@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
15:08 <jgiannelos@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
15:05 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P59504 and previous config saved to /var/cache/conftool/dbconfig/20240404-150549-marostegui.json [production]
15:03 <logmsgbot> dreamyjazz Deployed security patch for T361295 [production]
15:01 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1174 (T360332)', diff saved to https://phabricator.wikimedia.org/P59503 and previous config saved to /var/cache/conftool/dbconfig/20240404-150145-arnaudb.json [production]
15:00 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2113 (re)pooling @ 70%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59502 and previous config saved to /var/cache/conftool/dbconfig/20240404-150041-arnaudb.json [production]
15:00 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2213 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59501 and previous config saved to /var/cache/conftool/dbconfig/20240404-150032-arnaudb.json [production]
15:00 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2207 (re)pooling @ 4%: Post clone repool (dst)', diff saved to https://phabricator.wikimedia.org/P59500 and previous config saved to /var/cache/conftool/dbconfig/20240404-150017-arnaudb.json [production]
14:57 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1174 (T360332)', diff saved to https://phabricator.wikimedia.org/P59499 and previous config saved to /var/cache/conftool/dbconfig/20240404-145714-arnaudb.json [production]
14:57 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1174.eqiad.wmnet with reason: Maintenance [production]
14:56 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1174.eqiad.wmnet with reason: Maintenance [production]
14:56 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
14:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2107 (re)pooling @ 70%: Post clone repool (src)', diff saved to https://phabricator.wikimedia.org/P59498 and previous config saved to /var/cache/conftool/dbconfig/20240404-145617-arnaudb.json [production]
14:56 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
14:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1170 (T360332)', diff saved to https://phabricator.wikimedia.org/P59497 and previous config saved to /var/cache/conftool/dbconfig/20240404-145613-arnaudb.json [production]
14:50 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193 (T355609)', diff saved to https://phabricator.wikimedia.org/P59496 and previous config saved to /var/cache/conftool/dbconfig/20240404-145041-marostegui.json [production]