| 2023-06-27
      
      ยง | 
    
  | 19:56 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on phab2002.codfw.wmnet with reason: patch application | [production] | 
            
  | 19:56 | <dzahn@cumin1001> | START - Cookbook sre.hosts.downtime for 0:30:00 on phab2002.codfw.wmnet with reason: patch application | [production] | 
            
  | 19:55 | <kindrobot@deploy1002> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 19:54 | <kindrobot@deploy1002> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 19:53 | <kindrobot@deploy1002> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 19:51 | <btullis@deploy1002> | helmfile [staging] DONE helmfile.d/services/datahub: sync on main | [production] | 
            
  | 19:39 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 19:33 | <brennen@deploy1002> | Finished scap: Backport for [[gerrit:933425|Drop redundant targets (T340499)]] (duration: 07m 51s) | [production] | 
            
  | 19:27 | <brennen@deploy1002> | brennen: Backport for [[gerrit:933425|Drop redundant targets (T340499)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet | [production] | 
            
  | 19:25 | <brennen@deploy1002> | Started scap: Backport for [[gerrit:933425|Drop redundant targets (T340499)]] | [production] | 
            
  | 19:23 | <btullis@deploy1002> | helmfile [staging] DONE helmfile.d/services/datahub: sync on main | [production] | 
            
  | 19:11 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 18:47 | <sukhe> | upgrade dns6001 to gdnsd 3.99.0~alpha2 | [production] | 
            
  | 18:41 | <brennen@deploy1002> | rebuilt and synchronized wikiversions files: group0 wikis to 1.41.0-wmf.15  refs T340243 | [production] | 
            
  | 18:40 | <btullis@deploy1002> | helmfile [staging] DONE helmfile.d/services/datahub: sync on main | [production] | 
            
  | 18:31 | <brennen@deploy1002> | Finished scap: Backport for [[gerrit:933154|Display the language button on pages without languages (T315036)]] (duration: 08m 53s) | [production] | 
            
  | 18:29 | <jhathaway> | puppet re-enabled, enjoy! | [production] | 
            
  | 18:28 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 18:27 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 18:26 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 18:25 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 18:25 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 18:24 | <brennen@deploy1002> | abi and brennen: Backport for [[gerrit:933154|Display the language button on pages without languages (T315036)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet | [production] | 
            
  | 18:22 | <brennen@deploy1002> | Started scap: Backport for [[gerrit:933154|Display the language button on pages without languages (T315036)]] | [production] | 
            
  | 18:18 | <jhathaway> | disabling puppet to test stdlib upgrade patch | [production] | 
            
  | 17:45 | <gmodena@deploy1002> | helmfile [staging] DONE helmfile.d/services/mw-page-content-change-enrich: apply | [production] | 
            
  | 17:45 | <gmodena@deploy1002> | helmfile [staging] START helmfile.d/services/mw-page-content-change-enrich: apply | [production] | 
            
  | 17:45 | <gmodena@deploy1002> | helmfile [staging] DONE helmfile.d/services/mw-page-content-change-enrich: apply | [production] | 
            
  | 17:45 | <gmodena@deploy1002> | helmfile [staging] START helmfile.d/services/mw-page-content-change-enrich: apply | [production] | 
            
  | 17:44 | <gmodena@deploy1002> | helmfile [staging] DONE helmfile.d/services/mw-page-content-change-enrich: apply | [production] | 
            
  | 17:44 | <gmodena@deploy1002> | helmfile [staging] START helmfile.d/services/mw-page-content-change-enrich: apply | [production] | 
            
  | 17:22 | <brennen@deploy1002> | Pruned MediaWiki: 1.41.0-wmf.12 (duration: 02m 05s) | [production] | 
            
  | 17:20 | <brennen@deploy1002> | Finished scap: testwikis wikis to 1.41.0-wmf.15  refs T340243 (duration: 42m 56s) | [production] | 
            
  | 16:49 | <mutante> | webperf1003/2003 restarted apache after deploying gerrit:932441 | [production] | 
            
  | 16:37 | <brennen@deploy1002> | Started scap: testwikis wikis to 1.41.0-wmf.15  refs T340243 | [production] | 
            
  | 16:36 | <brennen> | train 1.41.0-wmf.15: re-running scap stage-train (T340243) | [production] | 
            
  | 16:03 | <btullis@deploy1002> | helmfile [staging] DONE helmfile.d/services/datahub: sync on main | [production] | 
            
  | 15:51 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 15:36 | <jbond> | puppet-merge fixed again | [production] | 
            
  | 15:35 | <root@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:34 | <root@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:34 | <jmm@cumin2002> | END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti-test2002.codfw.wmnet | [production] | 
            
  | 15:34 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2002.codfw.wmnet | [production] | 
            
  | 15:33 | <root@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:33 | <root@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:32 | <root@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:32 | <root@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:24 | <root@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:24 | <root@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2001.codfw.wmnet | [production] | 
            
  | 15:24 | <jbond> | puppet-merge temprrarily broken | [production] |