| 2024-09-25
      
      ยง | 
    
  | 11:20 | <akosiaris@cumin1002> | END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) depool zotero in eqiad: maintenance | [production] | 
            
  | 11:17 | <gmodena@deploy1003> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich-next: apply | [production] | 
            
  | 11:17 | <gmodena@deploy1003> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich-next: apply | [production] | 
            
  | 11:15 | <akosiaris@cumin1002> | END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) zotero.discovery.wmnet on all recursors | [production] | 
            
  | 11:15 | <akosiaris@cumin1002> | START - Cookbook sre.dns.wipe-cache zotero.discovery.wmnet on all recursors | [production] | 
            
  | 11:15 | <akosiaris@cumin1002> | START - Cookbook sre.discovery.service-route depool zotero in eqiad: maintenance | [production] | 
            
  | 11:15 | <gmodena@deploy1003> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich-next: apply | [production] | 
            
  | 11:15 | <gmodena@deploy1003> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich-next: apply | [production] | 
            
  | 11:13 | <akosiaris@cumin1002> | END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool zotero in codfw: maintenance | [production] | 
            
  | 11:11 | <jmm@cumin2002> | END (FAIL) - Cookbook sre.puppet.migrate-host (exit_code=99) for host cloudcephosd1036.eqiad.wmnet | [production] | 
            
  | 11:10 | <jynus> | running UPDATE into viwiki db2218 (s7) T375507 | [production] | 
            
  | 11:08 | <akosiaris@cumin1002> | END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) zotero.discovery.wmnet on all recursors | [production] | 
            
  | 11:08 | <akosiaris@cumin1002> | START - Cookbook sre.dns.wipe-cache zotero.discovery.wmnet on all recursors | [production] | 
            
  | 11:08 | <akosiaris@cumin1002> | START - Cookbook sre.discovery.service-route pool zotero in codfw: maintenance | [production] | 
            
  | 11:05 | <mvolz@deploy1003> | helmfile [codfw] DONE helmfile.d/services/zotero: apply | [production] | 
            
  | 11:04 | <mvolz@deploy1003> | helmfile [codfw] START helmfile.d/services/zotero: apply | [production] | 
            
  | 11:04 | <jmm@cumin2002> | START - Cookbook sre.puppet.migrate-host for host cloudcephosd1036.eqiad.wmnet | [production] | 
            
  | 11:03 | <jmm@cumin2002> | END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host cloudcephosd1037.eqiad.wmnet | [production] | 
            
  | 10:54 | <akosiaris@cumin1002> | END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) depool zotero in codfw: maintenance | [production] | 
            
  | 10:53 | <jmm@cumin2002> | START - Cookbook sre.puppet.migrate-host for host cloudcephosd1037.eqiad.wmnet | [production] | 
            
  | 10:49 | <akosiaris@cumin1002> | END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) zotero.discovery.wmnet on all recursors | [production] | 
            
  | 10:49 | <akosiaris@cumin1002> | START - Cookbook sre.dns.wipe-cache zotero.discovery.wmnet on all recursors | [production] | 
            
  | 10:49 | <akosiaris@cumin1002> | START - Cookbook sre.discovery.service-route depool zotero in codfw: maintenance | [production] | 
            
  | 10:46 | <sfaci@deploy1003> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mpic-next: apply | [production] | 
            
  | 10:46 | <sfaci@deploy1003> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mpic-next: apply | [production] | 
            
  | 10:43 | <akosiaris@cumin1002> | END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool zotero in eqiad: maintenance | [production] | 
            
  | 10:38 | <akosiaris@cumin1002> | START - Cookbook sre.discovery.service-route pool zotero in eqiad: maintenance | [production] | 
            
  | 10:33 | <arturo> | [codfw1dev] cleanup unused security groups (T375604) | [admin] | 
            
  | 10:25 | <jmm@cumin2002> | END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host cloudcephosd1025.eqiad.wmnet | [production] | 
            
  | 10:24 | <mvolz@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/zotero: apply | [production] | 
            
  | 10:24 | <mvolz@deploy1003> | helmfile [eqiad] START helmfile.d/services/zotero: apply | [production] | 
            
  | 10:19 | <mvolz@deploy1003> | helmfile [staging] DONE helmfile.d/services/zotero: apply | [production] | 
            
  | 10:19 | <mvolz@deploy1003> | helmfile [staging] START helmfile.d/services/zotero: apply | [production] | 
            
  | 10:06 | <jmm@cumin2002> | START - Cookbook sre.puppet.migrate-host for host cloudcephosd1025.eqiad.wmnet | [production] | 
            
  | 09:54 | <aborrero@cloudcumin1001> | END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch | [admin] | 
            
  | 09:54 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch | [admin] | 
            
  | 09:53 | <aborrero@cloudcumin1001> | END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch | [admin] | 
            
  | 09:52 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch | [admin] | 
            
  | 09:52 | <aborrero@cloudcumin1001> | END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch | [admin] | 
            
  | 09:48 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch | [admin] | 
            
  | 09:48 | <arturo> | [codfw1dev] restart rabbitmq on all cloudcontrol servers | [admin] | 
            
  | 09:48 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan+apply for main branch | [admin] | 
            
  | 09:46 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch | [admin] | 
            
  | 09:46 | <aborrero@cloudcumin1001> | END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) | [admin] | 
            
  | 09:44 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.restart_openstack | [admin] | 
            
  | 09:43 | <btullis> | root@an-test-worker1002:/tmp# find *_resources -type f -mtime +60 -exec rm {} \; | [analytics] | 
            
  | 09:39 | <moritzm> | installing distro-info-data updates from bullseye/bookworm point updates | [production] | 
            
  | 09:35 | <jmm@cumin2002> | END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host cloudcephosd1038.eqiad.wmnet | [production] | 
            
  | 09:35 | <arturo> | [codfw1dev] deletre a bunch of tests and seemingly unused projects | [admin] | 
            
  | 09:25 | <jmm@cumin2002> | START - Cookbook sre.puppet.migrate-host for host cloudcephosd1038.eqiad.wmnet | [production] |