3851-3900 of 10000 results (105ms)
2023-07-13 §
00:50 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase10[17,22,23,24,29,32].eqiad.wmnet: Applying JVM update - eevans@cumin1001 [production]
00:47 <eileen> config revision changed from ccc33b1e to e3e5a11d - renabled jobs [production]
00:33 <eileen> civicrm upgraded from 1bfc3b17 to 882e2310 [production]
00:17 <eileen> drush @wmff civicrm-upgrade-db [production]
00:08 <eileen> config revision changed from 6ac88ea8 to ccc33b1e (I pushed the upgrade code out) [production]
2023-07-12 §
23:59 <eileen> config revision changed from c543419d to ccc33b1e [production]
23:57 <eileen> config revision changed from 6ac88ea8 to c543419d [production]
23:47 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase10[16,19,20,21,28,31].eqiad.wmnet: Applying JVM update - eevans@cumin1001 [production]
23:27 <bking@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
23:07 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase10[16,19,20,21,28,31].eqiad.wmnet: Applying JVM update - eevans@cumin1001 [production]
22:18 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase20[12,17,18,23,26,27].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
21:49 <bking@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
21:43 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase20[12,17,18,23,26,27].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
21:33 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase20[16,20,22,25].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
21:32 <rzl@deploy1002> helmfile [eqiad] DONE helmfile.d/services/opentelemetry-collector: apply [production]
21:32 <rzl@deploy1002> helmfile [eqiad] START helmfile.d/services/opentelemetry-collector: apply [production]
21:09 <TheresNoTime> close UTC late backport window [production]
21:09 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase20[16,20,22,25].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
21:08 <samtar@deploy1002> Finished scap: Backport for [[gerrit:937513|ruwikibooks: Add NS104 to wgNamespacesToBeSearchedDefault (T341708)]] (duration: 08m 10s) [production]
21:02 <samtar@deploy1002> stang and samtar: Backport for [[gerrit:937513|ruwikibooks: Add NS104 to wgNamespacesToBeSearchedDefault (T341708)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
21:00 <rzl@deploy1002> helmfile [codfw] DONE helmfile.d/services/opentelemetry-collector: apply [production]
21:00 <samtar@deploy1002> Started scap: Backport for [[gerrit:937513|ruwikibooks: Add NS104 to wgNamespacesToBeSearchedDefault (T341708)]] [production]
21:00 <rzl@deploy1002> helmfile [codfw] START helmfile.d/services/opentelemetry-collector: apply [production]
20:59 <eevans@cumin1001> END (FAIL) - Cookbook sre.cassandra.roll-restart (exit_code=99) for nodes matching restbase20[15-16,20,22,25].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
20:59 <samtar@deploy1002> Finished scap: Backport for [[gerrit:937527|Fix mediawiki.special_diff_interactions configuration]] (duration: 08m 47s) [production]
20:52 <samtar@deploy1002> samtar and urbanecm: Backport for [[gerrit:937527|Fix mediawiki.special_diff_interactions configuration]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
20:50 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase20[15-16,20,22,25].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
20:50 <samtar@deploy1002> Started scap: Backport for [[gerrit:937527|Fix mediawiki.special_diff_interactions configuration]] [production]
20:49 <samtar@deploy1002> Finished scap: Backport for [[gerrit:937096|log additional events on Special:Diff|MobileDiff (T326212)]] (duration: 26m 41s) [production]
20:48 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase20[14,21,24].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
20:29 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase20[14,21,24].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
20:29 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase20[13,19].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
20:24 <jiji@deploy1002> helmfile [staging] DONE helmfile.d/services/thumbor: apply [production]
20:24 <samtar@deploy1002> jsn and samtar: Backport for [[gerrit:937096|log additional events on Special:Diff|MobileDiff (T326212)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet [production]
20:23 <jiji@deploy1002> helmfile [staging] START helmfile.d/services/thumbor: apply [production]
20:22 <samtar@deploy1002> Started scap: Backport for [[gerrit:937096|log additional events on Special:Diff|MobileDiff (T326212)]] [production]
20:21 <samtar@deploy1002> Finished scap: Backport for [[gerrit:937478|Fix Error: Module "./ext.pageTriage.defaultTagsOptions.js" is not loaded (T340112)]] (duration: 09m 27s) [production]
20:20 <jiji@deploy1002> helmfile [staging] DONE helmfile.d/services/thumbor: apply [production]
20:20 <jiji@deploy1002> helmfile [staging] START helmfile.d/services/thumbor: apply [production]
20:20 <jiji@deploy1002> helmfile [staging] DONE helmfile.d/services/thumbor: apply [production]
20:20 <jiji@deploy1002> helmfile [staging] START helmfile.d/services/thumbor: apply [production]
20:17 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase20[13,19].codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
20:13 <samtar@deploy1002> samtar: Backport for [[gerrit:937478|Fix Error: Module "./ext.pageTriage.defaultTagsOptions.js" is not loaded (T340112)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
20:11 <samtar@deploy1002> Started scap: Backport for [[gerrit:937478|Fix Error: Module "./ext.pageTriage.defaultTagsOptions.js" is not loaded (T340112)]] [production]
20:10 <samtar@deploy1002> Finished scap: Backport for [[gerrit:937511|[ruwiki] Add permissions to 'editor' usergroup (T341707)]] (duration: 08m 04s) [production]
20:08 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching restbase2013.codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
20:04 <samtar@deploy1002> samtar: Backport for [[gerrit:937511|[ruwiki] Add permissions to 'editor' usergroup (T341707)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
20:02 <samtar@deploy1002> Started scap: Backport for [[gerrit:937511|[ruwiki] Add permissions to 'editor' usergroup (T341707)]] [production]
19:57 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching restbase2013.codfw.wmnet: Applying JVM update - eevans@cumin1001 [production]
18:39 <dduvall@deploy1002> Synchronized php: group1 wikis to 1.41.0-wmf.17 refs T340245 (duration: 06m 16s) [production]