4101-4150 of 10000 results (93ms)
2024-05-15 ยง
13:44 <dcausse@deploy1002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
13:44 <eevans@deploy1002> helmfile [staging] START helmfile.d/services/echostore: apply [production]
13:43 <jsn@deploy1002> jsn: Continuing with sync [production]
13:42 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" [production]
13:42 <jsn@deploy1002> jsn: Backport for [[gerrit:1026972|extension-list: Add AutoModerator (T364034)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:41 <moritzm> installing libpgjava security updates [production]
13:40 <elukey@puppetmaster1001> conftool action : set/pooled=yes; selector: name=thanos-fe1001.eqiad.wmnet [production]
13:40 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-host for host db1209.eqiad.wmnet [production]
13:39 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host db1203.eqiad.wmnet [production]
13:34 <elukey> depool thanos-fe1001 and move envoy to PKI TLS cert [production]
13:34 <elukey@puppetmaster1001> conftool action : set/pooled=no; selector: name=thanos-fe1001.eqiad.wmnet [production]
13:32 <brouberol@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:32 <brouberol@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
13:27 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:27 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:27 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:27 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:26 <arnaudb@cumin1002> END (ERROR) - Cookbook sre.mysql.reboot_sanitaria (exit_code=97) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:26 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:25 <arnaudb@cumin1002> END (FAIL) - Cookbook sre.mysql.reboot_sanitaria (exit_code=99) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:25 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:25 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-main2006.codfw.wmnet with reason: host reimage [production]
13:22 <jhancock@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-main2006.codfw.wmnet with reason: host reimage [production]
13:18 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:18 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:17 <arnaudb@cumin1002> END (FAIL) - Cookbook sre.mysql.reboot_sanitaria (exit_code=99) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:17 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:16 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:16 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:15 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:15 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:12 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:12 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:11 <brouberol@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:11 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:11 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:11 <brouberol@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
13:10 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:10 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:09 <jsn@deploy1002> Started scap: Backport for [[gerrit:1026972|extension-list: Add AutoModerator (T364034)]] [production]
13:09 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:09 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:07 <vgutierrez> uploaded golang-github-florianl-go-tc 0.4.4-0.20240511074908-d584238bf6cb to apt.wm.o (bookworm-wikimedia) [production]
13:04 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host kafka-main2006.codfw.wmnet with OS bullseye [production]
13:03 <arnaudb@cumin1002> END (FAIL) - Cookbook sre.mysql.copy (exit_code=99) Will create a clone of db2114.codfw.wmnet onto db1125.eqiad.wmnet [production]
13:03 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.reboot_sanitaria (exit_code=0) Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:03 <arnaudb@cumin1002> START - Cookbook sre.mysql.reboot_sanitaria Will restart a pool of Sanitarium MariaDB instances and/or hosts. [production]
13:02 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host kafka-main2006.codfw.wmnet with OS bullseye [production]
13:01 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host kafka-main2006.codfw.wmnet with OS bullseye [production]
13:01 <jayme@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]