701-750 of 10000 results (90ms)
2025-08-05 ยง
17:23 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P80832 and previous config saved to /var/cache/conftool/dbconfig/20250805-172327-fceratto.json [production]
17:19 <bblack@cumin1002> DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Jobo out of all services on: 2395 hosts [production]
17:16 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:15 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:15 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:14 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
17:14 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:14 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
17:14 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:14 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
17:14 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
17:14 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
17:14 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
17:14 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:14 <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:14 <bking@cumin2002> END (ERROR) - Cookbook sre.wdqs.data-transfer (exit_code=97) (T386098, transfer newly-reloaded data) xfer wikidata_main from wdqs1022.eqiad.wmnet -> wdqs2007.codfw.wmnet w/ force delete existing files, repooling both afterwards [production]
17:12 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
17:12 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
17:12 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
17:11 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
17:11 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
17:11 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
17:10 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
17:10 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
17:10 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
17:10 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
17:10 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
17:10 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
17:09 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
17:09 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
17:09 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
17:09 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
17:08 <swfrench@deploy1003> swfrench: Migrate debug and cli images to xhprof - T401152 synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
17:08 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
17:08 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P80831 and previous config saved to /var/cache/conftool/dbconfig/20250805-170820-fceratto.json [production]
17:08 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
17:08 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
17:07 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
17:07 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
17:07 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
17:07 <swfrench@deploy1003> Started scap sync-world: Migrate debug and cli images to xhprof - T401152 [production]
17:05 <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1175620|Profiler: Add php-xhprof support besides php-tideways_xhprof (T401152)]] (duration: 11m 15s) [production]