2251-2300 of 10000 results (74ms)
2023-01-12 ยง
13:53 <jelto@cumin1001> START - Cookbook sre.gitlab.upgrade [production]
13:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P43132 and previous config saved to /var/cache/conftool/dbconfig/20230112-135143-marostegui.json [production]
13:36 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P43131 and previous config saved to /var/cache/conftool/dbconfig/20230112-133636-marostegui.json [production]
13:30 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: sync [production]
13:29 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/thumbor: sync [production]
13:28 <hnowlan@deploy1002> helmfile [eqiad] DONE helmfile.d/services/thumbor: sync [production]
13:28 <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:879277|Remove obsolete MWMinimalScriptInit and MEDIAWIKI_MAINT_INIT_ONLY.]] (duration: 21m 44s) [production]
13:26 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: sync [production]
13:26 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/services/thumbor: sync [production]
13:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1196 (T321391)', diff saved to https://phabricator.wikimedia.org/P43130 and previous config saved to /var/cache/conftool/dbconfig/20230112-132130-marostegui.json [production]
13:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1196 (T321391)', diff saved to https://phabricator.wikimedia.org/P43129 and previous config saved to /var/cache/conftool/dbconfig/20230112-131908-marostegui.json [production]
13:19 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1196.eqiad.wmnet with reason: Maintenance [production]
13:18 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1196.eqiad.wmnet with reason: Maintenance [production]
13:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186 (T321391)', diff saved to https://phabricator.wikimedia.org/P43128 and previous config saved to /var/cache/conftool/dbconfig/20230112-131847-marostegui.json [production]
13:16 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/thumbor: sync [production]
13:13 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: sync [production]
13:10 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/thumbor: sync [production]
13:08 <ladsgroup@deploy1002> ladsgroup and daniel: Backport for [[gerrit:879277|Remove obsolete MWMinimalScriptInit and MEDIAWIKI_MAINT_INIT_ONLY.]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet [production]
13:06 <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:879277|Remove obsolete MWMinimalScriptInit and MEDIAWIKI_MAINT_INIT_ONLY.]] [production]
13:05 <btullis@cumin1001> END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) [production]
13:05 <btullis@cumin1001> Added views for new wiki: gorwiktionary T326138 [production]
13:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186', diff saved to https://phabricator.wikimedia.org/P43127 and previous config saved to /var/cache/conftool/dbconfig/20230112-130341-marostegui.json [production]
12:58 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: sync [production]
12:56 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/thumbor: sync [production]
12:51 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: sync [production]
12:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186', diff saved to https://phabricator.wikimedia.org/P43125 and previous config saved to /var/cache/conftool/dbconfig/20230112-124834-marostegui.json [production]
12:41 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/thumbor: sync [production]
12:41 <btullis@cumin1001> START - Cookbook sre.wikireplicas.add-wiki [production]
12:33 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1186 (T321391)', diff saved to https://phabricator.wikimedia.org/P43123 and previous config saved to /var/cache/conftool/dbconfig/20230112-123328-marostegui.json [production]
12:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1186 (T321391)', diff saved to https://phabricator.wikimedia.org/P43122 and previous config saved to /var/cache/conftool/dbconfig/20230112-123106-marostegui.json [production]
12:31 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1186.eqiad.wmnet with reason: Maintenance [production]
12:30 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1186.eqiad.wmnet with reason: Maintenance [production]
12:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T321391)', diff saved to https://phabricator.wikimedia.org/P43121 and previous config saved to /var/cache/conftool/dbconfig/20230112-123045-marostegui.json [production]
12:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P43120 and previous config saved to /var/cache/conftool/dbconfig/20230112-121538-marostegui.json [production]
12:13 <XioNoX> repool esams [production]
12:10 <jayme@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
12:09 <jayme@deploy1002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
12:09 <jayme@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
12:09 <jayme@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
12:08 <jayme@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
12:08 <jayme@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
12:08 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
12:08 <jayme@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P43119 and previous config saved to /var/cache/conftool/dbconfig/20230112-120032-marostegui.json [production]
11:54 <XioNoX> re-seating cr2-esams fpc0 linecard - T318783 [production]
11:52 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: sync [production]
11:45 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T321391)', diff saved to https://phabricator.wikimedia.org/P43116 and previous config saved to /var/cache/conftool/dbconfig/20230112-114524-marostegui.json [production]
11:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1184 (T321391)', diff saved to https://phabricator.wikimedia.org/P43115 and previous config saved to /var/cache/conftool/dbconfig/20230112-114302-marostegui.json [production]
11:42 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
11:42 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]