3551-3600 of 10000 results (114ms)
2023-05-01 §
11:20 <zabe@deploy1002> sync-world aborted: Backport for [[gerrit:913915|Start writing to af_actor/afh_actor in group0 wikis (T334295)]] (duration: 02m 33s) [production]
11:17 <zabe@deploy1002> Started scap: Backport for [[gerrit:913915|Start writing to af_actor/afh_actor in group0 wikis (T334295)]] [production]
2023-04-30 §
14:17 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db2184.codfw.wmnet with reason: Host down T335640 [production]
14:17 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on db2184.codfw.wmnet with reason: Host down T335640 [production]
08:06 <elukey> powercycle ores1002 (mgmt console tty not usable, host frozen) [production]
2023-04-29 §
23:03 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1132.eqiad.wmnet with reason: Maint [production]
23:03 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on db1132.eqiad.wmnet with reason: Maint [production]
22:54 <rzl@cumin2002> dbctl commit (dc=all): 'Depool db1132', diff saved to https://phabricator.wikimedia.org/P47290 and previous config saved to /var/cache/conftool/dbconfig/20230429-225457-rzl.json [production]
2023-04-28 §
22:46 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
22:46 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS entries for new frack nodes - pt1979@cumin2002" [production]
22:33 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS entries for new frack nodes - pt1979@cumin2002" [production]
22:31 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
21:53 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on gerrit1001.wikimedia.org with reason: setup [production]
21:53 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on gerrit1001.wikimedia.org with reason: setup [production]
20:25 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on gerrit2002.wikimedia.org with reason: setup [production]
20:25 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on gerrit2002.wikimedia.org with reason: setup [production]
20:24 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on gerrit1003.wikimedia.org with reason: setup [production]
20:24 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on gerrit1003.wikimedia.org with reason: setup [production]
20:24 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on gerrit1001.wikimedia.org with reason: setup [production]
20:24 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on gerrit1001.wikimedia.org with reason: setup [production]
19:20 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
19:20 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
19:16 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
19:16 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
19:10 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
19:10 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
19:07 <otto@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
19:07 <otto@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
17:50 <htriedman@deploy1002> Finished deploy [airflow-dags/platform_eng@d56b7fb]: (no justification provided) (duration: 00m 10s) [production]
17:50 <htriedman@deploy1002> Started deploy [airflow-dags/platform_eng@d56b7fb]: (no justification provided) [production]
15:39 <elukey@deploy1002> helmfile [ml-staging-codfw] 'sync' command on namespace 'ores-legacy' for release 'main' . [production]
15:23 <jynus> update schema for backup1-codfw (mediabackups) T327157 [production]
15:07 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 2519 [production]
15:01 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 2519 [production]
14:57 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['stat1004'] [production]
14:57 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['stat1004'] [production]
14:51 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['stat1004'] [production]
14:50 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['stat1004'] [production]
13:21 <vgutierrez> import haproxy 2.7.7 on apt.wm.o thirdparty/haproxy27 for bullseye [production]
12:36 <akosiaris@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
12:35 <akosiaris@deploy1002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
12:35 <akosiaris@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
12:34 <akosiaris@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
12:31 <akosiaris@deploy1002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
12:30 <akosiaris@deploy1002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
12:29 <akosiaris@deploy1002> helmfile [staging-eqiad] DONE helmfile.d/admin 'sync'. [production]
12:29 <akosiaris@deploy1002> helmfile [staging-eqiad] START helmfile.d/admin 'sync'. [production]
12:08 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add new server sretest1003 - jclark@cumin1001" [production]
12:06 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
10:43 <klausman@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ml-cache2003.codfw.wmnet with OS bullseye [production]