| 2023-06-27
      
      ยง | 
    
  | 23:58 | <denisse@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host xhgui2002.codfw.wmnet | [production] | 
            
  | 23:58 | <denisse@cumin1001> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host xhgui2002.codfw.wmnet with OS bookworm | [production] | 
            
  | 23:49 | <denisse@cumin1001> | END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host xhgui1002.eqiad.wmnet | [production] | 
            
  | 23:49 | <denisse@cumin1001> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host xhgui1002.eqiad.wmnet with OS bookworm | [production] | 
            
  | 23:43 | <denisse@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on xhgui2002.codfw.wmnet with reason: host reimage | [production] | 
            
  | 23:40 | <denisse@cumin1001> | START - Cookbook sre.hosts.downtime for 2:00:00 on xhgui2002.codfw.wmnet with reason: host reimage | [production] | 
            
  | 23:34 | <denisse@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on xhgui1002.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 23:31 | <denisse@cumin1001> | START - Cookbook sre.hosts.downtime for 2:00:00 on xhgui1002.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 23:23 | <denisse@cumin1001> | START - Cookbook sre.hosts.reimage for host xhgui2002.codfw.wmnet with OS bookworm | [production] | 
            
  | 23:23 | <denisse@cumin1001> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM xhgui2002.codfw.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:22 | <denisse@cumin1001> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM xhgui2002.codfw.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:22 | <denisse@cumin1001> | END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) xhgui2002.codfw.wmnet on all recursors | [production] | 
            
  | 23:22 | <denisse@cumin1001> | START - Cookbook sre.dns.wipe-cache xhgui2002.codfw.wmnet on all recursors | [production] | 
            
  | 23:22 | <denisse@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 23:22 | <denisse@cumin1001> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM xhgui2002.codfw.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:21 | <denisse@cumin1001> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM xhgui2002.codfw.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:20 | <denisse@cumin1001> | START - Cookbook sre.hosts.reimage for host xhgui1002.eqiad.wmnet with OS bookworm | [production] | 
            
  | 23:20 | <denisse@cumin1001> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM xhgui1002.eqiad.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:19 | <denisse@cumin1001> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM xhgui1002.eqiad.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:19 | <denisse@cumin1001> | END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) xhgui1002.eqiad.wmnet on all recursors | [production] | 
            
  | 23:19 | <denisse@cumin1001> | START - Cookbook sre.dns.wipe-cache xhgui1002.eqiad.wmnet on all recursors | [production] | 
            
  | 23:19 | <denisse@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 23:19 | <denisse@cumin1001> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM xhgui1002.eqiad.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:18 | <denisse@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 23:18 | <denisse@cumin1001> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM xhgui1002.eqiad.wmnet - denisse@cumin1001" | [production] | 
            
  | 23:18 | <denisse@cumin1001> | START - Cookbook sre.ganeti.makevm for new host xhgui2002.codfw.wmnet | [production] | 
            
  | 23:16 | <denisse@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 23:16 | <denisse@cumin1001> | START - Cookbook sre.ganeti.makevm for new host xhgui1002.eqiad.wmnet | [production] | 
            
  | 22:43 | <andrew@deploy1002> | Finished deploy [horizon/deploy@9d02cd6]: installing (but not registering) magnum-ui (duration: 01m 27s) | [production] | 
            
  | 22:42 | <andrew@deploy1002> | Started deploy [horizon/deploy@9d02cd6]: installing (but not registering) magnum-ui | [production] | 
            
  | 21:50 | <mutante> | prometheus4002 - sudo a2dismod access_compat ; sudo systemctl restart apach2 ; sudo apachectl configtest -> Syntax OK :) - to proof it works without the access_compat module T258686 | [production] | 
            
  | 21:45 | <mutante> | prometheus* - puppet and partially manaul restart of apaches after deploying gerrit:932443 | [production] | 
            
  | 20:50 | <TheresNoTime> | close UTC late backport window | [production] | 
            
  | 20:48 | <samtar@deploy1002> | Finished scap: Backport for [[gerrit:933626|Title: Fix exists() assertion in toPageRecord() (T340568)]] (duration: 06m 52s) | [production] | 
            
  | 20:43 | <samtar@deploy1002> | matmarex and samtar: Backport for [[gerrit:933626|Title: Fix exists() assertion in toPageRecord() (T340568)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet | [production] | 
            
  | 20:41 | <samtar@deploy1002> | Started scap: Backport for [[gerrit:933626|Title: Fix exists() assertion in toPageRecord() (T340568)]] | [production] | 
            
  | 20:20 | <samtar@deploy1002> | Finished scap: Backport for [[gerrit:927653|Remove most DiscussionTools feature configs (T322497)]], [[gerrit:932270|Remove references to auth-api.php (T204193)]] (duration: 06m 53s) | [production] | 
            
  | 20:17 | <jforrester@deploy1002> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 20:16 | <jforrester@deploy1002> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 20:15 | <samtar@deploy1002> | reedy and esanders and samtar: Backport for [[gerrit:927653|Remove most DiscussionTools feature configs (T322497)]], [[gerrit:932270|Remove references to auth-api.php (T204193)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet | [production] | 
            
  | 20:13 | <samtar@deploy1002> | Started scap: Backport for [[gerrit:927653|Remove most DiscussionTools feature configs (T322497)]], [[gerrit:932270|Remove references to auth-api.php (T204193)]] | [production] | 
            
  | 20:13 | <btullis@deploy1002> | helmfile [staging] DONE helmfile.d/services/datahub: sync on main | [production] | 
            
  | 20:10 | <samtar@deploy1002> | Finished scap: Backport for [[gerrit:933460|Remove unused config $wgVisualEditorAllowLossySwitching (T339871)]], [[gerrit:927632|Remove wgDiscussionToolsEnable config (T322497)]] (duration: 07m 35s) | [production] | 
            
  | 20:04 | <samtar@deploy1002> | esanders and samtar and matmarex: Backport for [[gerrit:933460|Remove unused config $wgVisualEditorAllowLossySwitching (T339871)]], [[gerrit:927632|Remove wgDiscussionToolsEnable config (T322497)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet | [production] | 
            
  | 20:03 | <samtar@deploy1002> | Started scap: Backport for [[gerrit:933460|Remove unused config $wgVisualEditorAllowLossySwitching (T339871)]], [[gerrit:927632|Remove wgDiscussionToolsEnable config (T322497)]] | [production] | 
            
  | 20:00 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 19:59 | <brennen@deploy1002> | Finished deploy [phabricator/deployment@a25a737]: deploy latest state to phab1004 (duration: 00m 38s) | [production] | 
            
  | 19:59 | <brennen@deploy1002> | Started deploy [phabricator/deployment@a25a737]: deploy latest state to phab1004 | [production] | 
            
  | 19:57 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on phab1004.eqiad.wmnet with reason: patch application | [production] | 
            
  | 19:57 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime for 0:30:00 on phab1004.eqiad.wmnet with reason: patch application | [production] |