4601-4650 of 10000 results (46ms)
2021-09-13 §
09:18 <elukey> upgrade rsyslog* on ml-serve* nodes to 8.1901.0-1+wmf2 [production]
09:16 <godog> swift eqiad-prod: add weight to ms-be10[64-67] - T290546 [production]
09:11 <moritzm> reimaging sretest1002 [production]
09:11 <elukey> upload rsyslog* 8.1901.0-1+wmf2 to buster-wikimedia component/rsyslog-k8s - T277739 [production]
08:57 <arturo> cleared grid queues error states (T290844) [tools]
08:55 <arturo> repooling sgeexec-0907 (T290798) [tools]
08:16 <godog> bump +100G prometheus/ops codfw [production]
08:14 <arturo> rebooting sgeexec-0907 (T290798) [tools]
08:12 <arturo> depool sgeexec-0907 (T290798) [tools]
2021-09-12 §
21:04 <wm-bot> <lucaswerkmeister> deployed 4da7f64c4b (updates without downtime) [tools.lexeme-forms]
20:02 <wm-bot> <lucaswerkmeister> deployed f21554ab71 (refactoring, noop) [tools.lexeme-forms]
18:33 <vgutierrez> restart varnish-fe on cp3061, cp3063 and cp3065 [production]
18:29 <vgutierrez> restart varnish on cp3055 [production]
18:26 <vgutierrez> restart varnish on cp3057 [production]
17:28 <bstorm> truncated 38 GB files qaus.err and purawiki.err T278199 [tools.khanamalumat]
17:28 <bstorm> truncated 38 GB files qaus.err and purawiki.err [tools.khanamalumat]
17:07 <bstorm> truncated 77G Worker2.out T288276 [tools.iabot]
17:06 <bstorm> truncating 45G reflinks.err file T288276 [tools.rubin16]
17:04 <bstorm> truncated 58gb error.log file T288276 [tools.magnus-toolserver]
15:05 <wm-bot> <lucaswerkmeister> deployed a4b05045d6 (Croatian nouns) [tools.lexeme-forms]
04:53 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
04:52 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
2021-09-11 §
19:13 <wm-bot> <bd808> Restart for fcgi container crash [tools.sal]
19:02 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 27814b8eaacb5ba2fee1b6167a36ea14356a1ecf: testwiki: Fully remove securepoll-related groups (T290808) (duration: 00m 57s) [production]
18:34 <urbanecm> [urbanecm@mwmaint2002 ~]$ mwscript emptyUserGroup.php --wiki=testwiki {electionadmin,electcomm} # T290808 [production]
18:31 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 908bbf35235ea4129795dfbf4c0e646440152e18: Revert "test: Add electcomm and electionadmin groups" (T290808) (duration: 00m 58s) [production]
08:51 <majavah> depool tools-sgeexec-0907 [tools]
08:41 <majavah> silence deployment-prep alerts yet again [metricsinfra]
2021-09-10 §
23:26 <bstorm> cleared error state for tools-sgeexec-0907.tools.eqiad.wmflabs [tools]
21:52 <James_F> Created experimental integration-agent-docker-1021 for T252071 [releng]
21:48 <James_F> Deleting CI agent integration-agent-docker-1001 for T252071 [releng]
21:44 <James_F> Pulling oldest CI agent integration-agent-docker-1001 from rotation so it can be replaced by a bullseye one for T252071 [releng]
21:28 <legoktm@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'shellbox-syntaxhighlight' for release 'main' . [production]
21:27 <legoktm@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'shellbox-syntaxhighlight' for release 'main' . [production]
21:23 <James_F> Zuul: [integration/config] Add shellcheck job for scripts defined in jjb as an experimental job [releng]
21:21 <legoktm@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'shellbox-syntaxhighlight' for release 'main' . [production]
20:46 <jhuneidi@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'blubberoid' for release 'production' . [production]
20:44 <jhuneidi@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'blubberoid' for release 'production' . [production]
20:42 <jhuneidi@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . [production]
18:34 <volans@cumin1001> END (FAIL) - Cookbook sre.experimental.reimage (exit_code=99) for host sretest1001.eqiad.wmnet [production]
18:08 <volans@cumin1001> START - Cookbook sre.experimental.reimage for host sretest1001.eqiad.wmnet [production]
17:41 <James_F> Zuul: [cloud/toolforge/jobs-framework-emailer] Add basic tox CI [releng]
17:16 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on puppetmaster2005.codfw.wmnet with reason: REIMAGE [production]
17:14 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on puppetmaster2005.codfw.wmnet with reason: REIMAGE [production]
16:42 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on puppetmaster2004.codfw.wmnet with reason: REIMAGE [production]
16:40 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on puppetmaster2004.codfw.wmnet with reason: REIMAGE [production]
16:14 <volans@cumin1001> END (FAIL) - Cookbook sre.experimental.reimage (exit_code=99) for host sretest1001.eqiad.wmnet [production]
16:03 <volans@cumin1001> START - Cookbook sre.experimental.reimage for host sretest1001.eqiad.wmnet [production]
15:39 <volans@cumin1001> END (FAIL) - Cookbook sre.experimental.reimage (exit_code=99) for host sretest1001.eqiad.wmnet [production]
15:27 <volans@cumin1001> START - Cookbook sre.experimental.reimage for host sretest1001.eqiad.wmnet [production]