| 
      
        2024-11-27
      
      ยง
     | 
  
    
  | 14:19 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of durum7001.magru.wmnet to drbd | 
  [production] | 
            
  | 14:13 | 
  <urbanecm@deploy2002> | 
  urbanecm, cscott, nmw03: Continuing with sync | 
  [production] | 
            
  | 14:09 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.ganeti.changedisk for changing disk type of durum7001.magru.wmnet to drbd | 
  [production] | 
            
  | 14:08 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of ncredir7001.magru.wmnet to drbd | 
  [production] | 
            
  | 14:08 | 
  <urbanecm@deploy2002> | 
  urbanecm, cscott, nmw03: Backport for [[gerrit:1098076|Enable ParserMigration compact indicator on all wikis (T363484)]], [[gerrit:1093405|Deploy Parsoid Read Views to de/ru wikivoyage and dagwiki (T375394 T380401)]], [[gerrit:1098019|Updated wordmark for Azerbaijani Wikiquote (T380974)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 14:03 | 
  <urbanecm@deploy2002> | 
  Started scap sync-world: Backport for [[gerrit:1098076|Enable ParserMigration compact indicator on all wikis (T363484)]], [[gerrit:1093405|Deploy Parsoid Read Views to de/ru wikivoyage and dagwiki (T375394 T380401)]], [[gerrit:1098019|Updated wordmark for Azerbaijani Wikiquote (T380974)]] | 
  [production] | 
            
  | 13:59 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.ganeti.changedisk for changing disk type of ncredir7001.magru.wmnet to drbd | 
  [production] | 
            
  | 13:55 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of install7001.wikimedia.org to drbd | 
  [production] | 
            
  | 13:45 | 
  <moritzm> | 
  installing php8.2 security updates | 
  [production] | 
            
  | 13:40 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.ganeti.changedisk for changing disk type of install7001.wikimedia.org to drbd | 
  [production] | 
            
  | 13:40 | 
  <aborrero@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) on host 'cloudvirt1061.eqiad.wmnet' | 
  [admin] | 
            
  | 13:39 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of netflow7001.magru.wmnet to drbd | 
  [production] | 
            
  | 13:38 | 
  <mszabo@deploy2002> | 
  Finished scap sync-world: Backport for [[gerrit:1098506|private: Add stub for wgReportIncidentZendeskSubjectLine (T380868)]], [[gerrit:1098480|Configure IRS Zendesk integration (T380908)]], [[gerrit:1093389|Configure instrument for the Incident Reporting System (T372823)]] (duration: 13m 53s) | 
  [production] | 
            
  | 13:31 | 
  <mszabo@deploy2002> | 
  mszabo: Continuing with sync | 
  [production] | 
            
  | 13:30 | 
  <mszabo@deploy2002> | 
  mszabo: Backport for [[gerrit:1098506|private: Add stub for wgReportIncidentZendeskSubjectLine (T380868)]], [[gerrit:1098480|Configure IRS Zendesk integration (T380908)]], [[gerrit:1093389|Configure instrument for the Incident Reporting System (T372823)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 13:28 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.ganeti.changedisk for changing disk type of netflow7001.magru.wmnet to drbd | 
  [production] | 
            
  | 13:27 | 
  <moritzm> | 
  rebalance magru02 following switch of VMs back to DRBD T376737 | 
  [production] | 
            
  | 13:26 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of durum7002.magru.wmnet to drbd | 
  [production] | 
            
  | 13:26 | 
  <aborrero@cloudcumin1001> | 
  START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt1061.eqiad.wmnet' | 
  [admin] | 
            
  | 13:24 | 
  <mszabo@deploy2002> | 
  Started scap sync-world: Backport for [[gerrit:1098506|private: Add stub for wgReportIncidentZendeskSubjectLine (T380868)]], [[gerrit:1098480|Configure IRS Zendesk integration (T380908)]], [[gerrit:1093389|Configure instrument for the Incident Reporting System (T372823)]] | 
  [production] | 
            
  | 13:20 | 
  <jclark@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wdqs1026.eqiad.wmnet with OS bullseye | 
  [production] | 
            
  | 13:20 | 
  <jclark@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wdqs1025.eqiad.wmnet with OS bullseye | 
  [production] | 
            
  | 13:20 | 
  <jclark@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wdqs1027.eqiad.wmnet with OS bullseye | 
  [production] | 
            
  | 13:16 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.ganeti.changedisk for changing disk type of durum7002.magru.wmnet to drbd | 
  [production] | 
            
  | 13:15 | 
  <kartik@deploy2002> | 
  helmfile [ml-staging-codfw] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . | 
  [production] | 
            
  | 13:15 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of doh7002.wikimedia.org to drbd | 
  [production] | 
            
  | 13:05 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.ganeti.changedisk for changing disk type of doh7002.wikimedia.org to drbd | 
  [production] | 
            
  | 13:05 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of prometheus7001.magru.wmnet to drbd | 
  [production] | 
            
  | 12:56 | 
  <jiji@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on kafka-main[1002,1007].eqiad.wmnet with reason: Hardware refresh | 
  [production] | 
            
  | 12:56 | 
  <jiji@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on kafka-main[1002,1007].eqiad.wmnet with reason: Hardware refresh | 
  [production] | 
            
  | 12:50 | 
  <moritzm> | 
  installing ghostscript security updates | 
  [production] | 
            
  | 12:39 | 
  <kartik@deploy2002> | 
  helmfile [ml-staging-codfw] 'sync' command on namespace 'recommendation-api-ng' for release 'main' . | 
  [production] | 
            
  | 12:38 | 
  <effie> | 
  start replacing kafka-main1002 with kafka-main1007 - T363214 | 
  [production] | 
            
  | 12:24 | 
  <mvolz@deploy2002> | 
  helmfile [staging] DONE helmfile.d/services/citoid: apply | 
  [production] | 
            
  | 12:24 | 
  <mvolz@deploy2002> | 
  helmfile [staging] START helmfile.d/services/citoid: apply | 
  [production] | 
            
  | 12:24 | 
  <kart_> | 
  Updated cxserver to 2024-11-20-121713-production (T377966, T357950) | 
  [production] | 
            
  | 12:22 | 
  <kartik@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 12:22 | 
  <kartik@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 12:20 | 
  <kartik@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 12:20 | 
  <kartik@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 12:18 | 
  <moritzm> | 
  installing python-cryptography security updates | 
  [production] | 
            
  | 12:14 | 
  <kartik@deploy2002> | 
  helmfile [staging] DONE helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 12:13 | 
  <kartik@deploy2002> | 
  helmfile [staging] START helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 12:12 | 
  <moritzm> | 
  installing openssl security updates | 
  [production] | 
            
  | 12:08 | 
  <mvolz@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/zotero: apply | 
  [production] | 
            
  | 12:07 | 
  <mvolz@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/zotero: apply | 
  [production] | 
            
  | 12:06 | 
  <mvolz@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/zotero: apply | 
  [production] | 
            
  | 12:06 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.ganeti.changedisk for changing disk type of prometheus7001.magru.wmnet to drbd | 
  [production] | 
            
  | 12:06 | 
  <mvolz@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/zotero: apply | 
  [production] | 
            
  | 12:05 | 
  <mvolz@deploy2002> | 
  helmfile [staging] DONE helmfile.d/services/zotero: apply | 
  [production] |