| 
      
        2024-09-09
      
      ยง
     | 
  
    
  | 21:01 | 
  <kindrobot@deploy1003> | 
  Finished scap sync-world: Backport for [[gerrit:1071281|Release donate link to pilot wikis (French Wikipedia and Wikifunctions) (T373585)]], [[gerrit:1071202|Fix typo in browser vendor prefix (T374180)]] (duration: 12m 39s) | 
  [production] | 
            
  | 20:57 | 
  <kindrobot@deploy1003> | 
  jforrester, toyofuku, kindrobot: Continuing with sync | 
  [production] | 
            
  | 20:51 | 
  <kindrobot@deploy1003> | 
  jforrester, toyofuku, kindrobot: Backport for [[gerrit:1071281|Release donate link to pilot wikis (French Wikipedia and Wikifunctions) (T373585)]], [[gerrit:1071202|Fix typo in browser vendor prefix (T374180)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 20:49 | 
  <kindrobot@deploy1003> | 
  Started scap sync-world: Backport for [[gerrit:1071281|Release donate link to pilot wikis (French Wikipedia and Wikifunctions) (T373585)]], [[gerrit:1071202|Fix typo in browser vendor prefix (T374180)]] | 
  [production] | 
            
  | 19:33 | 
  <jforrester@deploy1003> | 
  Finished scap sync-world: Backport for [[gerrit:1071690|ZObjectStructureValidator::validate: use set_time_limit() to limit in the case of run-away JsonSchema (T374241)]] (duration: 08m 19s) | 
  [production] | 
            
  | 19:28 | 
  <jforrester@deploy1003> | 
  jforrester: Continuing with sync | 
  [production] | 
            
  | 19:27 | 
  <jforrester@deploy1003> | 
  jforrester: Backport for [[gerrit:1071690|ZObjectStructureValidator::validate: use set_time_limit() to limit in the case of run-away JsonSchema (T374241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 19:25 | 
  <jforrester@deploy1003> | 
  Started scap sync-world: Backport for [[gerrit:1071690|ZObjectStructureValidator::validate: use set_time_limit() to limit in the case of run-away JsonSchema (T374241)]] | 
  [production] | 
            
  | 18:16 | 
  <ejegg> | 
  payments-wiki upgraded from 4f3500f7 to 672c9fb6 | 
  [production] | 
            
  | 17:44 | 
  <jforrester@deploy1003> | 
  Finished scap sync-world: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] (duration: 12m 05s) | 
  [production] | 
            
  | 17:39 | 
  <jforrester@deploy1003> | 
  jforrester: Continuing with sync | 
  [production] | 
            
  | 17:33 | 
  <jforrester@deploy1003> | 
  jforrester: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 17:31 | 
  <jforrester@deploy1003> | 
  Started scap sync-world: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] | 
  [production] | 
            
  | 17:15 | 
  <ebernhardson@deploy1003> | 
  helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 17:15 | 
  <ebernhardson@deploy1003> | 
  helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 17:08 | 
  <ebernhardson@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 17:07 | 
  <ebernhardson@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 17:05 | 
  <pt1979@cumin2002> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 17:05 | 
  <pt1979@cumin2002> | 
  END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add nes frack firewalls - pt1979@cumin2002" | 
  [production] | 
            
  | 17:05 | 
  <pt1979@cumin2002> | 
  START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add nes frack firewalls - pt1979@cumin2002" | 
  [production] | 
            
  | 17:02 | 
  <ebernhardson@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 17:02 | 
  <ebernhardson@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 17:01 | 
  <pt1979@cumin2002> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 16:54 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Manuel Merz (WMDE) out of all services on: 1552 hosts | 
  [production] | 
            
  | 16:53 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Manuel Merz (WMDE) out of all services on: 1552 hosts | 
  [production] | 
            
  | 16:52 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Manuel Merz (WMDE) out of all services on: 677 hosts | 
  [production] | 
            
  | 16:52 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Manuel Merz (WMDE) out of all services on: 677 hosts | 
  [production] | 
            
  | 16:46 | 
  <ejegg> | 
  payments-wiki upgraded from e47e61cb to 4f3500f7 | 
  [production] | 
            
  | 16:36 | 
  <dcaro> | 
  cleaned up dns leaks | 
  [admin] | 
            
  | 16:26 | 
  <hnowlan@deploy1003> | 
  Finished scap sync-world: Backport for [[gerrit:1071628|Enable async uploads on test2wiki (T356241)]] (duration: 11m 11s) | 
  [production] | 
            
  | 16:23 | 
  <wmbot~raymondndibe@wmf3402> | 
  END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component cert-manager | 
  [tools] | 
            
  | 16:20 | 
  <hnowlan@deploy1003> | 
  hnowlan: Continuing with sync | 
  [production] | 
            
  | 16:20 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 2 hosts: mw[2428-2429].codfw.wmnet | 
  [production] | 
            
  | 16:20 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.debmonitor.remove-hosts for 2 hosts: mw[2428-2429].codfw.wmnet | 
  [production] | 
            
  | 16:19 | 
  <hnowlan@deploy1003> | 
  hnowlan: Backport for [[gerrit:1071628|Enable async uploads on test2wiki (T356241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 16:17 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.k8s.renumber-node (exit_code=0) Renumbering for host wikikube-worker2106.codfw.wmnet | 
  [production] | 
            
  | 16:17 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2106.codfw.wmnet | 
  [production] | 
            
  | 16:17 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2106.codfw.wmnet | 
  [production] | 
            
  | 16:17 | 
  <elukey@cumin1002> | 
  END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2001.codfw.wmnet with OS bookworm | 
  [production] | 
            
  | 16:16 | 
  <wmbot~raymondndibe@wmf3402> | 
  START - Cookbook wmcs.toolforge.component.deploy for component cert-manager | 
  [tools] | 
            
  | 16:15 | 
  <hnowlan@deploy1003> | 
  Started scap sync-world: Backport for [[gerrit:1071628|Enable async uploads on test2wiki (T356241)]] | 
  [production] | 
            
  | 16:14 | 
  <wmbot~raymondndibe@wmf3402> | 
  END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component cert-manager | 
  [toolsbeta] | 
            
  | 16:13 | 
  <elukey@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host sretest2001.codfw.wmnet with OS bookworm | 
  [production] | 
            
  | 16:13 | 
  <elukey@cumin1002> | 
  END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2001.codfw.wmnet with OS bookworm | 
  [production] | 
            
  | 16:10 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2106.codfw.wmnet with OS bullseye | 
  [production] | 
            
  | 16:09 | 
  <wmbot~raymondndibe@wmf3402> | 
  START - Cookbook wmcs.toolforge.component.deploy for component cert-manager | 
  [toolsbeta] | 
            
  | 16:09 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.k8s.renumber-node (exit_code=0) Renumbering for host wikikube-worker2105.codfw.wmnet | 
  [production] | 
            
  | 16:09 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2105.codfw.wmnet | 
  [production] | 
            
  | 16:09 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2105.codfw.wmnet | 
  [production] | 
            
  | 16:09 | 
  <hnowlan@cumin1002> | 
  END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2095.codfw.wmnet | 
  [production] |