1201-1250 of 10000 results (35ms)
2021-09-29 ยง
17:13 <ladsgroup@deploy1002> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:724776|Fully enable change dispatching via jobs on test wikis]], Part I (duration: 01m 07s) [production]
16:52 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:47 <pt1979@cumin2002> END (PASS) - Cookbook sre.experimental.reimage (exit_code=0) for host thumbor2006.codfw.wmnet [production]
16:47 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:43 <akosiaris> start hbal -L -G row_B -X on ganeti01.svc.codfw.wmnet . Rows C and D are fine [production]
16:42 <akosiaris> start hbal -L -G row_A -X on ganeti01.svc.codfw.wmnet [production]
16:40 <akosiaris> migrate kubemaster2001 off ganeti2007 and to ganeti2008 due to memory starvation on ganeti2007 [production]
16:37 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:34 <pt1979@cumin2002> START - Cookbook sre.experimental.reimage for host thumbor2006.codfw.wmnet [production]
16:33 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
16:25 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.2/extensions/WikimediaBadges/: Backport: [[gerrit:724561|Handle missing items in WikibaseClientSiteLinksForItemHandler (T291953)]] (duration: 01m 08s) [production]
16:24 <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.38.0-wmf.1/extensions/WikimediaBadges/: Backport: [[gerrit:724560|Handle missing items in WikibaseClientSiteLinksForItemHandler (T291953)]] (duration: 01m 10s) [production]
15:58 <pt1979@cumin2002> END (FAIL) - Cookbook sre.experimental.reimage (exit_code=99) for host thumbor2006.codfw.wmnet [production]
15:53 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:50 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:45 <Amir1> disabled cron dispatching for mediawikiwiki [production]
15:44 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:724765|Enable change dispatching via jobs in wikidatawiki (T48643)]] (duration: 01m 08s) [production]
15:44 <pt1979@cumin2002> START - Cookbook sre.experimental.reimage for host thumbor2006.codfw.wmnet [production]
15:39 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be2035.codfw.wmnet [production]
15:39 <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.2/extensions/Wikibase/client: Backport: [[gerrit:724558|Track time until dispatched recent changes are inserted (T291962)]] (duration: 01m 10s) [production]
15:24 <pt1979@cumin2002> END (PASS) - Cookbook sre.experimental.reimage (exit_code=0) for host thumbor2006.codfw.wmnet [production]
15:22 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host ms-be2035.codfw.wmnet [production]
15:21 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2026.codfw.wmnet [production]
15:12 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2026.codfw.wmnet [production]
15:02 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . [production]
14:51 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2025.codfw.wmnet [production]
14:45 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2025.codfw.wmnet [production]
14:38 <jmm@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:35 <jmm@cumin2002> START - Cookbook sre.dns.netbox [production]
14:25 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts testvm2001.codfw.wmnet [production]
14:17 <jmm@cumin2002> START - Cookbook sre.hosts.decommission for hosts testvm2001.codfw.wmnet [production]
14:08 <dcausse@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . [production]
14:07 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
14:07 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
14:04 <pt1979@cumin2002> START - Cookbook sre.experimental.reimage for host thumbor2006.codfw.wmnet [production]
14:01 <dcausse@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . [production]
13:38 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:38 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. [production]
13:34 <volans@cumin2002> END (PASS) - Cookbook sre.experimental.reimage (exit_code=0) for host sretest1001.eqiad.wmnet [production]
13:31 <dcausse@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . [production]
13:11 <otto@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-analytics' for release 'production' . [production]
13:11 <otto@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'eventgate-analytics' for release 'canary' . [production]
13:09 <volans@cumin2002> START - Cookbook sre.experimental.reimage for host sretest1001.eqiad.wmnet [production]
13:09 <otto@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'eventgate-analytics' for release 'canary' . [production]
13:09 <otto@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'eventgate-analytics' for release 'production' . [production]
13:08 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:08 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. [production]
13:04 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
13:04 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
12:09 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]