| 
      
        2023-10-23
      
      ยง
     | 
  
    
  | 13:16 | 
  <urbanecm@deploy2002> | 
  Finished scap: Backport for [[gerrit:967429|wikidatawiki: Switch property for determining Lexeme language code (T348923)]] (duration: 12m 50s) | 
  [production] | 
            
  | 13:11 | 
  <urbanecm@deploy2002> | 
  migr and urbanecm: Continuing with sync | 
  [production] | 
            
  | 13:05 | 
  <urbanecm@deploy2002> | 
  migr and urbanecm: Backport for [[gerrit:967429|wikidatawiki: Switch property for determining Lexeme language code (T348923)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 13:04 | 
  <moritzm> | 
  installing libxpm security updates on buster | 
  [production] | 
            
  | 13:04 | 
  <urbanecm@deploy2002> | 
  Started scap: Backport for [[gerrit:967429|wikidatawiki: Switch property for determining Lexeme language code (T348923)]] | 
  [production] | 
            
  | 12:41 | 
  <jayme@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 12:40 | 
  <jayme> | 
  switched mw-web (mw-on-k8s) to certmanager certificates - T300033 | 
  [production] | 
            
  | 12:40 | 
  <jayme@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 12:40 | 
  <jayme@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 12:39 | 
  <jayme@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 12:33 | 
  <moritzm> | 
  installing libx11 security updates | 
  [production] | 
            
  | 12:16 | 
  <arnaudb@cumin1001> | 
  END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1131.eqiad.wmnet onto db1231.eqiad.wmnet | 
  [production] | 
            
  | 11:49 | 
  <jnuche@deploy2002> | 
  Finished deploy [releng/jenkins-deploy@054e07d] (releasing): (no justification provided) (duration: 00m 42s) | 
  [production] | 
            
  | 11:49 | 
  <jnuche@deploy2002> | 
  Started deploy [releng/jenkins-deploy@054e07d] (releasing): (no justification provided) | 
  [production] | 
            
  | 11:49 | 
  <moritzm> | 
  added Balthazar to pwstore | 
  [production] | 
            
  | 11:33 | 
  <btullis@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on an-airflow1007.eqiad.wmnet with reason: Server not yet in productin use | 
  [production] | 
            
  | 11:33 | 
  <btullis@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on an-airflow1007.eqiad.wmnet with reason: Server not yet in productin use | 
  [production] | 
            
  | 10:51 | 
  <brouberol@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts kafka-jumbo1001.eqiad.wmnet | 
  [production] | 
            
  | 10:51 | 
  <brouberol@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 10:51 | 
  <brouberol@cumin1001> | 
  END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-jumbo1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin1001" | 
  [production] | 
            
  | 10:51 | 
  <arnaudb@cumin1001> | 
  START - Cookbook sre.mysql.clone of db1131.eqiad.wmnet onto db1231.eqiad.wmnet | 
  [production] | 
            
  | 10:50 | 
  <arnaudb@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1131 T344036', diff saved to https://phabricator.wikimedia.org/P53025 and previous config saved to /var/cache/conftool/dbconfig/20231023-105036-arnaudb.json | 
  [production] | 
            
  | 10:50 | 
  <brouberol@cumin1001> | 
  START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-jumbo1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin1001" | 
  [production] | 
            
  | 10:41 | 
  <jayme> | 
  switched mw-debug (mw-on-k8s) to certmanager certificates - T300033 | 
  [production] | 
            
  | 10:40 | 
  <btullis@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-test-client1002.eqiad.wmnet | 
  [production] | 
            
  | 10:40 | 
  <brouberol@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 10:37 | 
  <arnaudb@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1131.eqiad.wmnet with reason: provisionning - T344036 | 
  [production] | 
            
  | 10:37 | 
  <arnaudb@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1131.eqiad.wmnet with reason: provisionning - T344036 | 
  [production] | 
            
  | 10:37 | 
  <arnaudb@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1231.eqiad.wmnet with reason: provisionning - T344036 | 
  [production] | 
            
  | 10:37 | 
  <arnaudb@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1231.eqiad.wmnet with reason: provisionning - T344036 | 
  [production] | 
            
  | 10:36 | 
  <btullis@cumin1001> | 
  START - Cookbook sre.hosts.reboot-single for host an-test-client1002.eqiad.wmnet | 
  [production] | 
            
  | 10:35 | 
  <jayme@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 10:34 | 
  <jayme@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 10:34 | 
  <jayme@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 10:34 | 
  <jayme@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 10:32 | 
  <brouberol@cumin1001> | 
  START - Cookbook sre.hosts.decommission for hosts kafka-jumbo1001.eqiad.wmnet | 
  [production] | 
            
  | 10:32 | 
  <arnaudb@cumin1001> | 
  dbctl commit (dc=all): 'Provision db1231 depooled as a candidate master for s6', diff saved to https://phabricator.wikimedia.org/P53024 and previous config saved to /var/cache/conftool/dbconfig/20231023-103202-arnaudb.json | 
  [production] | 
            
  | 10:31 | 
  <jayme@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 10:29 | 
  <jbond@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 10:29 | 
  <jbond@cumin1001> | 
  END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: test - jbond@cumin1001" | 
  [production] | 
            
  | 10:28 | 
  <jbond@cumin1001> | 
  START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: test - jbond@cumin1001" | 
  [production] | 
            
  | 10:26 | 
  <jbond@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 10:26 | 
  <jbond@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 10:26 | 
  <jbond@cumin1001> | 
  END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: test - jbond@cumin1001" | 
  [production] | 
            
  | 10:25 | 
  <jbond@cumin1001> | 
  START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: test - jbond@cumin1001" | 
  [production] | 
            
  | 10:23 | 
  <jbond@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 10:20 | 
  <jbond@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 10:19 | 
  <jbond@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 10:13 | 
  <brouberol@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts kafka-jumbo1002.eqiad.wmnet | 
  [production] | 
            
  | 10:13 | 
  <brouberol@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] |