| 
      
        2023-11-17
      
      ยง
     | 
  
    
  | 11:16 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] [main] DONE helmfile.d/services/mw-jobrunner : sync | 
  [production] | 
            
  | 11:16 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] [canary] START helmfile.d/services/mw-jobrunner : sync | 
  [production] | 
            
  | 11:16 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] [main] START helmfile.d/services/mw-jobrunner : sync | 
  [production] | 
            
  | 11:16 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] [main] DONE helmfile.d/services/mw-jobrunner : sync | 
  [production] | 
            
  | 11:15 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] [canary] DONE helmfile.d/services/mw-jobrunner : sync | 
  [production] | 
            
  | 11:15 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] [main] START helmfile.d/services/mw-jobrunner : sync | 
  [production] | 
            
  | 11:15 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] [canary] START helmfile.d/services/mw-jobrunner : sync | 
  [production] | 
            
  | 11:15 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply | 
  [production] | 
            
  | 11:14 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply | 
  [production] | 
            
  | 11:14 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-api-ext: apply | 
  [production] | 
            
  | 11:14 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-api-ext: apply | 
  [production] | 
            
  | 11:14 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply | 
  [production] | 
            
  | 11:13 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-api-int: apply | 
  [production] | 
            
  | 11:13 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-api-int: apply | 
  [production] | 
            
  | 11:13 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-api-int: apply | 
  [production] | 
            
  | 11:12 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 11:12 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 11:12 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 11:11 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 11:11 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:11 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:11 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:10 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:10 | 
  <jynus> | 
  running schema change on backup1-eqiad (mediabackups) T191804 | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-wikifunctions: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-misc: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-api-ext: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-api-int: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-api-ext: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-api-int: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:08 | 
  <cgoubert@deploy2002> | 
  helmfile [codfw] START helmfile.d/services/mw-debug: apply | 
  [production] | 
            
  | 11:07 | 
  <claime> | 
  Redeploying mw-on-k8s for T350430 | 
  [production] | 
            
  | 10:54 | 
  <klausman@cumin1001> | 
  END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host ml-serve1008.eqiad.wmnet | 
  [production] | 
            
  | 10:53 | 
  <kharlan@deploy2002> | 
  helmfile [eqiad] DONE helmfile.d/services/ipoid: apply | 
  [production] | 
            
  | 10:52 | 
  <kharlan@deploy2002> | 
  helmfile [eqiad] START helmfile.d/services/ipoid: apply | 
  [production] | 
            
  | 10:51 | 
  <klausman@cumin1001> | 
  START - Cookbook sre.puppet.migrate-host for host ml-serve1008.eqiad.wmnet | 
  [production] | 
            
  | 10:31 | 
  <klausman@cumin1001> | 
  END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host ml-serve-ctrl1001.eqiad.wmnet | 
  [production] | 
            
  | 10:29 | 
  <arnaudb@cumin1001> | 
  dbctl commit (dc=all): 'Depooling db1148 (T348183)', diff saved to https://phabricator.wikimedia.org/P53542 and previous config saved to /var/cache/conftool/dbconfig/20231117-102952-arnaudb.json | 
  [production] | 
            
  | 10:29 | 
  <arnaudb@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1148.eqiad.wmnet with reason: Maintenance | 
  [production] | 
            
  | 10:29 | 
  <arnaudb@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1148.eqiad.wmnet with reason: Maintenance | 
  [production] | 
            
  | 10:29 | 
  <arnaudb@cumin1001> | 
  dbctl commit (dc=all): 'Repooling after maintenance db1147 (T348183)', diff saved to https://phabricator.wikimedia.org/P53541 and previous config saved to /var/cache/conftool/dbconfig/20231117-102931-arnaudb.json | 
  [production] |