| 2024-07-24
      
      ยง | 
    
  | 16:43 | <jhancock@cumin2002> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding frack servers to codfw - jhancock@cumin2002" | [production] | 
            
  | 16:38 | <jhancock@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 16:33 | <sukhe> | sudo cumin -b1 -s120 'O:wikidough' 'systemctl restart anycast-healthchecker.service' | [production] | 
            
  | 16:02 | <Southparkfan> | moved sessionstorage/kask from sessionstorage04 to sessionstorage06 T370461 | [releng] | 
            
  | 16:02 | <Southparkfan> | moved sessionstorage/kask from sessionstorage04 to sessionstorage06 T370461 | [deployment-prep] | 
            
  | 16:01 | <aborrero@cloudcumin1001> | END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 16:01 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 16:01 | <aborrero@cloudcumin1001> | END (ERROR) - Cookbook wmcs.openstack.tofu (exit_code=97) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 | [admin] | 
            
  | 16:01 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 | [admin] | 
            
  | 15:59 | <aborrero@cloudcumin1001> | END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 | [admin] | 
            
  | 15:59 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/24 | [admin] | 
            
  | 15:57 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:57 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:56 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:56 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:49 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:49 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:46 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:45 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:43 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:43 | <hnowlan@deploy1002> | helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply | [production] | 
            
  | 15:43 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:42 | <hnowlan@deploy1002> | helmfile [eqiad] START helmfile.d/services/shellbox-video: apply | [production] | 
            
  | 15:39 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:38 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:37 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:37 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:34 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:34 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:31 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:31 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:30 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs1020.eqiad.wmnet | [production] | 
            
  | 15:29 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:29 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:28 | <aborrero@cloudcumin1001> | END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:28 | <aborrero@cloudcumin1001> | START - Cookbook wmcs.openstack.tofu running tofu plan for https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/23 | [admin] | 
            
  | 15:24 | <sukhe@cumin1002> | START - Cookbook sre.hosts.reboot-single for host lvs1020.eqiad.wmnet | [production] | 
            
  | 15:20 | <jhancock@cumin2002> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host pc2017.codfw.wmnet with OS bookworm | [production] | 
            
  | 15:10 | <jhancock@cumin2002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on pc2017.codfw.wmnet with reason: host reimage | [production] | 
            
  | 15:07 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs3010.esams.wmnet | [production] | 
            
  | 15:07 | <jhancock@cumin2002> | START - Cookbook sre.hosts.downtime for 2:00:00 on pc2017.codfw.wmnet with reason: host reimage | [production] | 
            
  | 15:04 | <jhancock@cumin2002> | START - Cookbook sre.hosts.reimage for host pc2017.codfw.wmnet with OS bookworm | [production] | 
            
  | 15:01 | <sukhe@cumin1002> | START - Cookbook sre.hosts.reboot-single for host lvs3010.esams.wmnet | [production] | 
            
  | 14:54 | <jhancock@cumin2002> | END (FAIL) - Cookbook sre.hosts.reimage (exit_code=1) for host pc2017.codfw.wmnet with OS bookworm | [production] | 
            
  | 14:54 | <jhancock@cumin2002> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" | [production] | 
            
  | 14:52 | <jhancock@cumin2002> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" | [production] | 
            
  | 14:52 | <logmsgbot> | lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1056515|Revert "TranslatablePage: Split translatable page id cache into multiple shards"]], [[gerrit:1056516|Revert "TranslatablePage: Split translatable page id cache into multiple shards"]] (duration: 09m 37s) | [production] | 
            
  | 14:47 | <logmsgbot> | lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, trainbranchbot: Continuing with sync | [production] | 
            
  | 14:44 | <logmsgbot> | lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde, trainbranchbot: Backport for [[gerrit:1056515|Revert "TranslatablePage: Split translatable page id cache into multiple shards"]], [[gerrit:1056516|Revert "TranslatablePage: Split translatable page id cache into multiple shards"]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 14:42 | <logmsgbot> | lucaswerkmeister-wmde@deploy1002 Started scap sync-world: Backport for [[gerrit:1056515|Revert "TranslatablePage: Split translatable page id cache into multiple shards"]], [[gerrit:1056516|Revert "TranslatablePage: Split translatable page id cache into multiple shards"]] | [production] |