3201-3250 of 10000 results (41ms)
2021-12-13 ยง
19:11 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs1017.eqiad.wmnet with OS bullseye [production]
19:05 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:55 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs1019.eqiad.wmnet with OS bullseye [production]
18:53 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:52 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host lvs1020.eqiad.wmnet with OS bullseye [production]
18:51 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs1018.eqiad.wmnet with OS bullseye [production]
18:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host lvs1017.eqiad.wmnet with OS bullseye [production]
18:47 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:45 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs1020.eqiad.wmnet with OS bullseye [production]
18:34 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs1017.eqiad.wmnet with OS bullseye [production]
18:31 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:29 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host lvs1020.eqiad.wmnet with OS bullseye [production]
18:28 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host lvs1019.eqiad.wmnet with OS bullseye [production]
18:28 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host lvs1018.eqiad.wmnet with OS bullseye [production]
18:25 <dancy> dancy@deploy1002 rebuilt and synchronized wikiversions files: all wikis to 1.38.0-wmf.12 T293953 [production]
18:25 <dancy@deploy1002> rebuilt and synchronized wikiversions files: all wikis to 1.38.0-wmf.12 refs T293954 [production]
18:24 <hnowlan> joining restbase2025-c to cassandra cluster [production]
18:24 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:23 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:21 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.12/includes/: Backport: [[gerrit:746909|DeprecationHelper: avoid closures (T297236)]] (duration: 01m 02s) [production]
18:14 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host lvs1017.eqiad.wmnet with OS bullseye [production]
18:13 <jgiannelos@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . [production]
18:13 <cmjohnson@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host lvs1017.eqiad.wmnet with OS bullseye [production]
18:12 <jgiannelos@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . [production]
18:10 <jgiannelos@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . [production]
18:09 <jbond> upload cas_6.4.4-1+wmf10u2 [production]
18:05 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host lvs1017.eqiad.wmnet with OS bullseye [production]
18:01 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
18:00 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:54 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:54 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:51 <accraze@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
17:49 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.9/extensions/WikibaseLexeme: Backport: [[gerrit:746908|Add form:hasGrammaticalFeature() method (T297478)]] (no-op because Lexeme Lua is not yet enabled in prod) (duration: 00m 57s) [production]
17:47 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:47 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.9/extensions/WikibaseLexeme: Backport: [[gerrit:746907|Add lexeme:getLemma(), sense:getGloss(), form:getRepresentation() (T297024)]] (no-op because Lexeme Lua is not yet enabled in prod) (duration: 00m 57s) [production]
17:47 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:45 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.9/extensions/WikibaseLexeme: Backport: [[gerrit:746906|Remove most of mw.wikibase.lexeme Lua module (T297404)]] (no-op because Lexeme Lua is not yet enabled in prod) (duration: 00m 58s) [production]
17:43 <accraze@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . [production]
17:41 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on ganeti2021.codfw.wmnet with reason: Temporarily remove node from Ganeti for reimage [production]
17:41 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on ganeti2021.codfw.wmnet with reason: Temporarily remove node from Ganeti for reimage [production]
17:40 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:39 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:37 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
17:37 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
17:34 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
17:34 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
17:00 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:55 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
16:14 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]