651-700 of 10000 results (84ms)
2024-05-09 ยง
09:52 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157 (T364299)', diff saved to https://phabricator.wikimedia.org/P62220 and previous config saved to /var/cache/conftool/dbconfig/20240509-095249-marostegui.json [production]
09:52 <jforrester@deploy1002> Finished scap: Backport for [[gerrit:1027194|Disable ParserMigration on commonswiki (T364228)]] (duration: 16m 17s) [production]
09:45 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1167.eqiad.wmnet with reason: host reimage [production]
09:44 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to Unable to send diff to phaste and previous config saved to /var/cache/conftool/dbconfig/20240509-094431-ladsgroup.json [production]
09:43 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db1167.eqiad.wmnet with reason: host reimage [production]
09:39 <jforrester@deploy1002> lucaswerkmeister-wmde and jforrester: Continuing with sync [production]
09:38 <jforrester@deploy1002> lucaswerkmeister-wmde and jforrester: Backport for [[gerrit:1027194|Disable ParserMigration on commonswiki (T364228)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
09:37 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P62219 and previous config saved to /var/cache/conftool/dbconfig/20240509-093742-marostegui.json [production]
09:36 <jforrester@deploy1002> Started scap: Backport for [[gerrit:1027194|Disable ParserMigration on commonswiki (T364228)]] [production]
09:31 <jynus@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1171.eqiad.wmnet with reason: upgrade to 10.6 [production]
09:31 <jynus@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1171.eqiad.wmnet with reason: upgrade to 10.6 [production]
09:31 <jynus@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1150.eqiad.wmnet with reason: upgrade to 10.6 [production]
09:31 <jynus@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db1150.eqiad.wmnet with reason: upgrade to 10.6 [production]
09:29 <marostegui@cumin1002> START - Cookbook sre.hosts.reimage for host db1167.eqiad.wmnet with OS bookworm [production]
09:29 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P62218 and previous config saved to /var/cache/conftool/dbconfig/20240509-092921-ladsgroup.json [production]
09:27 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1167', diff saved to https://phabricator.wikimedia.org/P62217 and previous config saved to /var/cache/conftool/dbconfig/20240509-092757-root.json [production]
09:22 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P62216 and previous config saved to /var/cache/conftool/dbconfig/20240509-092234-marostegui.json [production]
09:14 <marostegui@cumin1002> dbctl commit (dc=all): 'es2020 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P62215 and previous config saved to /var/cache/conftool/dbconfig/20240509-091445-root.json [production]
09:14 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T352010)', diff saved to https://phabricator.wikimedia.org/P62214 and previous config saved to /var/cache/conftool/dbconfig/20240509-091413-ladsgroup.json [production]
09:07 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2157 (T364299)', diff saved to https://phabricator.wikimedia.org/P62213 and previous config saved to /var/cache/conftool/dbconfig/20240509-090726-marostegui.json [production]
09:04 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/editor-analytics: apply [production]
08:59 <marostegui@cumin1002> dbctl commit (dc=all): 'es2020 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P62212 and previous config saved to /var/cache/conftool/dbconfig/20240509-085939-root.json [production]
08:54 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/editor-analytics: apply [production]
08:53 <jynus> deploy new grants for es6, es7 backups T363812 [production]
08:44 <marostegui@cumin1002> dbctl commit (dc=all): 'es2020 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P62211 and previous config saved to /var/cache/conftool/dbconfig/20240509-084433-root.json [production]
08:37 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2157 (T364299)', diff saved to https://phabricator.wikimedia.org/P62210 and previous config saved to /var/cache/conftool/dbconfig/20240509-083705-marostegui.json [production]
08:36 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
08:36 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
08:36 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2128 (T364299)', diff saved to https://phabricator.wikimedia.org/P62209 and previous config saved to /var/cache/conftool/dbconfig/20240509-083643-marostegui.json [production]
08:30 <godog> set batphone oncall for May 9th only for EMEA, not Americas - T350192 [production]
08:29 <marostegui@cumin1002> dbctl commit (dc=all): 'es2020 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P62208 and previous config saved to /var/cache/conftool/dbconfig/20240509-082927-root.json [production]
08:21 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P62207 and previous config saved to /var/cache/conftool/dbconfig/20240509-082135-marostegui.json [production]
08:14 <marostegui@cumin1002> dbctl commit (dc=all): 'es2020 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P62206 and previous config saved to /var/cache/conftool/dbconfig/20240509-081422-root.json [production]
08:13 <godog> set batphone oncall for May 9th - T350192 [production]
08:09 <marostegui@cumin1002> dbctl commit (dc=all): 'db1180 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P62205 and previous config saved to /var/cache/conftool/dbconfig/20240509-080936-root.json [production]
08:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P62204 and previous config saved to /var/cache/conftool/dbconfig/20240509-080627-marostegui.json [production]
08:05 <marostegui@cumin1002> dbctl commit (dc=all): 'db1231 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P62203 and previous config saved to /var/cache/conftool/dbconfig/20240509-080549-root.json [production]
08:02 <zabe@deploy1002> Finished scap: Backport for [[gerrit:1029257|Fix error when marking a new page for translations (T364522)]] (duration: 19m 28s) [production]
07:59 <marostegui@cumin1002> dbctl commit (dc=all): 'es2020 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P62202 and previous config saved to /var/cache/conftool/dbconfig/20240509-075914-root.json [production]
07:54 <marostegui@cumin1002> dbctl commit (dc=all): 'db1180 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P62201 and previous config saved to /var/cache/conftool/dbconfig/20240509-075429-root.json [production]
07:51 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2128 (T364299)', diff saved to https://phabricator.wikimedia.org/P62200 and previous config saved to /var/cache/conftool/dbconfig/20240509-075118-marostegui.json [production]
07:50 <marostegui@cumin1002> dbctl commit (dc=all): 'db1231 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P62199 and previous config saved to /var/cache/conftool/dbconfig/20240509-075043-root.json [production]
07:50 <zabe@deploy1002> zabe and abi: Continuing with sync [production]
07:45 <zabe@deploy1002> zabe and abi: Backport for [[gerrit:1029257|Fix error when marking a new page for translations (T364522)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
07:44 <marostegui@cumin1002> dbctl commit (dc=all): 'es2020 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P62198 and previous config saved to /var/cache/conftool/dbconfig/20240509-074408-root.json [production]
07:43 <marostegui@cumin1002> dbctl commit (dc=all): 'Fully repool db1172', diff saved to https://phabricator.wikimedia.org/P62197 and previous config saved to /var/cache/conftool/dbconfig/20240509-074355-marostegui.json [production]
07:43 <zabe@deploy1002> Started scap: Backport for [[gerrit:1029257|Fix error when marking a new page for translations (T364522)]] [production]
07:42 <zabe@deploy1002> Finished scap: Backport for [[gerrit:1029200|Move wgGroupsAddToSelf and wgGroupsRemoveFromSelf to core-Permissions]] (duration: 17m 37s) [production]
07:39 <marostegui@cumin1002> dbctl commit (dc=all): 'db1180 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P62196 and previous config saved to /var/cache/conftool/dbconfig/20240509-073922-root.json [production]
07:35 <marostegui@cumin1002> dbctl commit (dc=all): 'db1231 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P62195 and previous config saved to /var/cache/conftool/dbconfig/20240509-073537-root.json [production]