| 2024-08-15
      
      ยง | 
    
  | 16:39 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:38 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:38 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:33 | <wmbot~dcaro@urcuchillay> | END (FAIL) - Cookbook wmcs.ceph.wait_for_rebalance (exit_code=99) | [admin] | 
            
  | 16:32 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:32 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:31 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:31 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:30 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:30 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:29 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:29 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:29 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:28 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:28 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:28 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:27 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 16:27 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 16:27 | <andrew@cloudcumin1001> | END (ERROR) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=97) | [admin] | 
            
  | 16:27 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 15:55 | <jayme@cumin1002> | END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host kafka-main2007.codfw.wmnet with OS bullseye | [production] | 
            
  | 15:53 | <SandraEbele_> | reran druid_load_geoeditors_monthly, cassandra_load_editors_by_country_monthly, and druid_load_edit_hourly airflow dags with run_id scheduled__2024-06-01T00:00:00+00:00 as part of down stream tasks after rerunning mediawiki_history_denormalize for 2024-06 snapshot. | [production] | 
            
  | 15:53 | <SandraEbele_> | reran druid_load_geoeditors_monthly, cassandra_load_editors_by_country_monthly, and druid_load_edit_hourly airflow dags with run_id scheduled__2024-06-01T00:00:00+00:00 as part of down stream tasks after rerunning mediawiki_history_denormalize for 2024-06 snapshot. | [analytics] | 
            
  | 15:52 | <sukhe> | sudo cumin -b1 -s60 "A:dnsbox" "run-puppet-agent --enable 'merging CR 1053929 T369366'": T369366 | [production] | 
            
  | 15:51 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 15:48 | <sukhe@cumin1002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 15:45 | <sukhe> | running authdns-update again | [production] | 
            
  | 15:43 | <sukhe> | running authdns-update | [production] | 
            
  | 15:31 | <ebernhardson@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 15:31 | <ebernhardson@deploy1003> | helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 15:30 | <ebernhardson@deploy1003> | helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 15:30 | <ebernhardson@deploy1003> | helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 15:27 | <jayme@cumin1002> | START - Cookbook sre.hosts.reimage for host kafka-main2008.codfw.wmnet with OS bullseye | [production] | 
            
  | 15:21 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: show site None [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:21 | <sukhe@cumin1002> | START - Cookbook sre.dns.admin DNS admin: show site None [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:21 | <sukhe> | running authdns-update | [production] | 
            
  | 15:20 | <sukhe@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=dns4004.wikimedia.org [reason: moving ahead with admin_state migration] | [production] | 
            
  | 15:10 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool site esams [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:09 | <sukhe@cumin1002> | START - Cookbook sre.dns.admin DNS admin: pool site esams [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:09 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: show site None [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:09 | <sukhe@cumin1002> | START - Cookbook sre.dns.admin DNS admin: show site None [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:04 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: show site None [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:03 | <sukhe@cumin1002> | START - Cookbook sre.dns.admin DNS admin: show site None [reason: no reason specified, no task ID specified] | [production] | 
            
  | 15:02 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site esams [reason: testing on dns4004, no task ID specified] | [production] | 
            
  | 15:01 | <sukhe@cumin1002> | START - Cookbook sre.dns.admin DNS admin: depool site esams [reason: testing on dns4004, no task ID specified] | [production] | 
            
  | 15:01 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool site magru [reason: testing on dns4004, no task ID specified] | [production] | 
            
  | 15:00 | <sukhe@cumin1002> | START - Cookbook sre.dns.admin DNS admin: pool site magru [reason: testing on dns4004, no task ID specified] | [production] | 
            
  | 14:57 | <klausman@deploy1003> | helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . | [production] | 
            
  | 14:53 | <klausman@deploy1003> | helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . | [production] | 
            
  | 14:49 | <klausman@deploy1003> | helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . | [production] |