| 2025-09-24
      
      ยง | 
    
  | 23:11 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host an-worker1210.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED | [production] | 
            
  | 23:11 | <jclark@cumin1002> | END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host an-worker1210.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED | [production] | 
            
  | 23:11 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host an-worker1211.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED | [production] | 
            
  | 23:11 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host an-worker1210.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED | [production] | 
            
  | 23:10 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host an-worker1209.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED | [production] | 
            
  | 23:08 | <jclark@cumin1002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 23:08 | <jclark@cumin1002> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns an-worker  - jclark@cumin1002" | [production] | 
            
  | 23:08 | <jclark@cumin1002> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns an-worker  - jclark@cumin1002" | [production] | 
            
  | 23:03 | <jclark@cumin1002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 22:53 | <krinkle@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1191135|Disable wmgUseMdotRouting on Wikivoyage and Wikiversity (group1) (T403510)]] (duration: 11m 53s) | [production] | 
            
  | 22:48 | <krinkle@deploy1003> | krinkle: Continuing with sync | [production] | 
            
  | 22:48 | <krinkle@deploy1003> | krinkle: Backport for [[gerrit:1191135|Disable wmgUseMdotRouting on Wikivoyage and Wikiversity (group1) (T403510)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 22:41 | <krinkle@deploy1003> | Started scap sync-world: Backport for [[gerrit:1191135|Disable wmgUseMdotRouting on Wikivoyage and Wikiversity (group1) (T403510)]] | [production] | 
            
  | 21:39 | <eileen> | civicrm upgraded from 9e1ffed5 to 7acd20cd | [production] | 
            
  | 21:39 | <dmartin@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:39 | <dmartin@deploy1003> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:38 | <dmartin@deploy1003> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:38 | <dmartin@deploy1003> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:37 | <dmartin@deploy1003> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:37 | <dmartin@deploy1003> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:31 | <dmartin@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:30 | <dmartin@deploy1003> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:30 | <dmartin@deploy1003> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:28 | <dmartin@deploy1003> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:28 | <dmartin@deploy1003> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:27 | <dmartin@deploy1003> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:23 | <dmartin@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:23 | <dmartin@deploy1003> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:22 | <dmartin@deploy1003> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:22 | <dmartin@deploy1003> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:21 | <dmartin@deploy1003> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:21 | <dmartin@deploy1003> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:17 | <dmartin@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:17 | <jhathaway> | uploaded spicerack_11.8.0 to apt.wikimedia.org bullseye-wikimedia,bookworm-wikimedia | [production] | 
            
  | 21:17 | <dmartin@deploy1003> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:16 | <dmartin@deploy1003> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:16 | <dmartin@deploy1003> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:15 | <dmartin@deploy1003> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:14 | <dmartin@deploy1003> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:10 | <dmartin@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:09 | <dmartin@deploy1003> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:09 | <dmartin@deploy1003> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:08 | <dmartin@deploy1003> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:07 | <dmartin@deploy1003> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:06 | <dmartin@deploy1003> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 21:05 | <brett@dns1004> | END - running authdns-update | [production] | 
            
  | 21:04 | <brett@dns1004> | START - running authdns-update | [production] | 
            
  | 20:56 | <mutante> | phab1004/phab2002 - unlink /srv/phab/phabricator/bin/move_project (broken symbolic link to ../scripts/move_beneath.php) T342275 | [production] | 
            
  | 20:42 | <ejegg> | fundraising civicrm upgraded from 5e5445a9 to 9e1ffed5 | [production] | 
            
  | 20:37 | <cmooney@cumin1003> | END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2009.codfw.wmnet with OS bookworm | [production] |