| 2023-06-02
      
      § | 
    
  | 13:24 | <otto@deploy1002> | helmfile [staging-eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 13:24 | <otto@deploy1002> | helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 13:24 | <otto@deploy1002> | helmfile [staging-codfw] START helmfile.d/admin 'apply'. | [production] | 
            
  | 13:22 | <otto@deploy1002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 13:22 | <otto@deploy1002> | helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 12:03 | <moritzm> | installing at-spi2-core bugfix updates from Bullseye point release | [production] | 
            
  | 09:35 | <moritzm> | installing texlive-security updates on buster | [production] | 
            
  | 09:18 | <akosiaris> | update kubernetes-node to 1.23.14-2 on all P:kubernetes::node hosts (88 in total) T337836. Reload systemd for unit changes to take effect | [production] | 
            
  | 08:52 | <jmm@cumin2002> | END (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: cp5016.eqsin.wmnet | [production] | 
            
  | 08:52 | <jmm@cumin2002> | START - Cookbook sre.debmonitor.remove-hosts for 1 hosts: cp5016.eqsin.wmnet | [production] | 
            
  | 08:52 | <jmm@cumin2002> | END (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: cp5015.eqsin.wmnet | [production] | 
            
  | 08:51 | <jmm@cumin2002> | START - Cookbook sre.debmonitor.remove-hosts for 1 hosts: cp5015.eqsin.wmnet | [production] | 
            
  | 08:51 | <jmm@cumin2002> | END (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: cp5014.eqsin.wmnet | [production] | 
            
  | 08:51 | <jmm@cumin2002> | START - Cookbook sre.debmonitor.remove-hosts for 1 hosts: cp5014.eqsin.wmnet | [production] | 
            
  | 08:51 | <jmm@cumin2002> | END (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 1 hosts: cp5013.eqsin.wmnet | [production] | 
            
  | 08:51 | <jmm@cumin2002> | START - Cookbook sre.debmonitor.remove-hosts for 1 hosts: cp5013.eqsin.wmnet | [production] | 
            
  | 08:51 | <jmm@cumin2002> | END (PASS) - Cookbook sre.debmonitor.remove-hosts (exit_code=0) for 0 hosts: | [production] | 
            
  | 08:51 | <jmm@cumin2002> | START - Cookbook sre.debmonitor.remove-hosts for 0 hosts: | [production] | 
            
  | 08:42 | <moritzm> | installing traceroute bugfix updates from Bullseye point release | [production] | 
            
  | 07:53 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast6002.wikimedia.org | [production] | 
            
  | 07:47 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host bast6002.wikimedia.org | [production] | 
            
  | 07:42 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast3006.wikimedia.org | [production] | 
            
  | 07:36 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host bast3006.wikimedia.org | [production] | 
            
  | 07:30 | <mvernon@cumin2002> | END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:eqiad or A:codfw and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) | [production] | 
            
  | 07:28 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast1003.wikimedia.org | [production] | 
            
  | 07:22 | <mvernon@cumin2002> | START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:eqiad or A:codfw and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) | [production] | 
            
  | 07:21 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host bast1003.wikimedia.org | [production] | 
            
  | 01:53 | <ejegg> | fundraising python tools upgraded from 759d4c89 to 2ca83336 | [production] | 
            
  | 01:22 | <cstone> | civicrm upgraded from 3819d6d1 to bcc8fccc | [production] | 
            
  
    | 2023-06-01
      
      § | 
    
  | 21:06 | <samtar@deploy1002> | Finished scap: Backport for [[gerrit:925858|Remove deleted config wgVectorStickyHeaderEdit (T337955)]] (duration: 08m 30s) | [production] | 
            
  | 20:59 | <samtar@deploy1002> | esanders and samtar: Backport for [[gerrit:925858|Remove deleted config wgVectorStickyHeaderEdit (T337955)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet | [production] | 
            
  | 20:57 | <samtar@deploy1002> | Started scap: Backport for [[gerrit:925858|Remove deleted config wgVectorStickyHeaderEdit (T337955)]] | [production] | 
            
  | 20:54 | <samtar@deploy1002> | Finished scap: Backport for [[gerrit:925792|Remove config and AB test code for edit buttons in sticky header (T337955)]] (duration: 10m 29s) | [production] | 
            
  | 20:45 | <samtar@deploy1002> | samtar and ksarabia: Backport for [[gerrit:925792|Remove config and AB test code for edit buttons in sticky header (T337955)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet | [production] | 
            
  | 20:44 | <samtar@deploy1002> | Started scap: Backport for [[gerrit:925792|Remove config and AB test code for edit buttons in sticky header (T337955)]] | [production] | 
            
  | 20:21 | <samtar@deploy1002> | Finished scap: Backport for [[gerrit:917863|Deploy Research Incentive survey on enwiki (T336092)]] (duration: 07m 56s) | [production] | 
            
  | 20:15 | <samtar@deploy1002> | dani and samtar: Backport for [[gerrit:917863|Deploy Research Incentive survey on enwiki (T336092)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet | [production] | 
            
  | 20:13 | <samtar@deploy1002> | Started scap: Backport for [[gerrit:917863|Deploy Research Incentive survey on enwiki (T336092)]] | [production] | 
            
  | 20:12 | <samtar@deploy1002> | Finished scap: Backport for [[gerrit:886370|Always collapse by default the CheckUserHelper on loginwiki (T328726)]] (duration: 08m 20s) | [production] | 
            
  | 20:05 | <samtar@deploy1002> | samtar and dreamyjazz: Backport for [[gerrit:886370|Always collapse by default the CheckUserHelper on loginwiki (T328726)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet | [production] | 
            
  | 20:03 | <samtar@deploy1002> | Started scap: Backport for [[gerrit:886370|Always collapse by default the CheckUserHelper on loginwiki (T328726)]] | [production] | 
            
  | 19:51 | <ejegg> | fundraising python tools upgraded from 72570bdd to 759d4c89 | [production] | 
            
  | 19:12 | <mforns@deploy1002> | Finished deploy [airflow-dags/analytics@21e7354]: (no justification provided) (duration: 02m 42s) | [production] | 
            
  | 19:11 | <mforns@deploy1002> | Started deploy [airflow-dags/analytics@21e7354]: (no justification provided) | [production] | 
            
  | 19:11 | <bblack@deploy1002> | Unlocked for deployment [ALL REPOSITORIES]: temporary lock for LVS/pybal upgrade work (duration: 03m 27s) | [production] | 
            
  | 19:09 | <bblack> | lvs1* (eqiad): upgrade pybal to 1.15.13 - T334703 | [production] | 
            
  | 19:08 | <bblack@deploy1002> | Locking from deployment [ALL REPOSITORIES]: temporary lock for LVS/pybal upgrade work | [production] | 
            
  | 18:45 | <bblack> | lvs6* (drmrs): upgrade pybal to 1.15.13 - T334703 | [production] | 
            
  | 18:33 | <bblack> | lvs3* (esams): upgrade pybal to 1.15.13 - T334703 | [production] | 
            
  | 18:32 | <dduvall@deploy1002> | rebuilt and synchronized wikiversions files: group2 wikis to 1.41.0-wmf.11  refs T337525 | [production] |