9551-9600 of 10000 results (96ms)
2023-09-05 §
05:30 <kartik@deploy1002> helmfile [codfw] START helmfile.d/services/machinetranslation: apply [production]
05:25 <kartik@deploy1002> helmfile [staging] DONE helmfile.d/services/machinetranslation: apply [production]
05:22 <kartik@deploy1002> helmfile [staging] START helmfile.d/services/machinetranslation: apply [production]
03:59 <mwpresync@deploy1002> Finished scap: testwikis wikis to 1.41.0-wmf.25 refs T343727 (duration: 56m 29s) [production]
03:02 <mwpresync@deploy1002> Started scap: testwikis wikis to 1.41.0-wmf.25 refs T343727 [production]
2023-09-04 §
16:14 <aikochou@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
16:07 <topranks> setting port 1/1/5 to speed 100G on cr2-codfw [production]
16:06 <topranks> setting port 1/1/5 to speed 100G on cr1-codfw [production]
16:05 <aikochou@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
15:46 <jdrewniak@deploy1002> Synchronized portals: Wikimedia Portals Update: [[gerrit:954709| Bumping portals to master (T128546)]] (duration: 06m 14s) [production]
15:40 <jdrewniak@deploy1002> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:954709| Bumping portals to master (T128546)]] (duration: 07m 01s) [production]
14:57 <moritzm> installing json-c security updates [production]
14:53 <ayounsi@cumin1001> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
14:47 <ayounsi@cumin1001> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
14:45 <ayounsi@cumin1001> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary [production]
14:44 <ayounsi@cumin1001> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
14:31 <godog> bounce prometheus@k8s-aux [production]
14:29 <aikochou@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
13:58 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
13:58 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
13:58 <elukey@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
13:56 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
13:55 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
13:54 <elukey@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
13:53 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host irc1001.wikimedia.org [production]
13:52 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
13:50 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:50 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS entries for codfw cr<-> ssw links. - cmooney@cumin1001" [production]
13:50 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
13:49 <elukey@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
13:49 <cmooney@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS entries for codfw cr<-> ssw links. - cmooney@cumin1001" [production]
13:49 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host irc1001.wikimedia.org [production]
13:48 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
13:48 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
13:48 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
13:47 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
13:46 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
13:45 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
13:45 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS entries for codfw cr<-> ssw links. - cmooney@cumin1001" [production]
13:41 <elukey@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
13:24 <cmooney@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS entries for codfw cr<-> ssw links. - cmooney@cumin1001" [production]
13:18 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
13:15 <cmooney@cumin1001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
13:12 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
12:52 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1002.eqiad.wmnet with OS bullseye [production]
12:46 <hnowlan> staggered restarting restbase service on A:restbase [production]
12:30 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 149665 [production]
12:30 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 149665 [production]
12:29 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 138884 [production]
12:28 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 138884 [production]