| 
      
        2023-07-03
      
      §
     | 
  
    
  | 08:48 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging David.pujol out of all services on: 760 hosts | 
  [production] | 
            
  | 08:48 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging David.pujol out of all services on: 760 hosts | 
  [production] | 
            
  | 08:48 | 
  <btullis@deploy1002> | 
  helmfile [staging] START helmfile.d/services/datahub: apply on main | 
  [production] | 
            
  | 08:47 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Dasm out of all services on: 760 hosts | 
  [production] | 
            
  | 08:47 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Dasm out of all services on: 760 hosts | 
  [production] | 
            
  | 08:47 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Dasm out of all services on: 1271 hosts | 
  [production] | 
            
  | 08:46 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Dasm out of all services on: 1271 hosts | 
  [production] | 
            
  | 08:46 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Dasm out of all services on: 20 hosts | 
  [production] | 
            
  | 08:46 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Dasm out of all services on: 20 hosts | 
  [production] | 
            
  | 08:45 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Aranyap out of all services on: 20 hosts | 
  [production] | 
            
  | 08:45 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Aranyap out of all services on: 20 hosts | 
  [production] | 
            
  | 08:45 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Aranyap out of all services on: 1271 hosts | 
  [production] | 
            
  | 08:45 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Aranyap out of all services on: 1271 hosts | 
  [production] | 
            
  | 08:44 | 
  <jmm@cumin2002> | 
  END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Aranyap out of all services on: 760 hosts | 
  [production] | 
            
  | 08:44 | 
  <jmm@cumin2002> | 
  START - Cookbook sre.idm.logout Logging Aranyap out of all services on: 760 hosts | 
  [production] | 
            
  | 08:44 | 
  <btullis@deploy1002> | 
  helmfile [staging] DONE helmfile.d/services/datahub: sync on main | 
  [production] | 
            
  | 08:33 | 
  <btullis@deploy1002> | 
  helmfile [staging] START helmfile.d/services/datahub: apply on main | 
  [production] | 
            
  | 07:54 | 
  <elukey@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/toolhub: sync | 
  [production] | 
            
  | 07:54 | 
  <elukey@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/toolhub: sync | 
  [production] | 
            
  | 07:32 | 
  <taavi@deploy1002> | 
  Finished scap: Backport for [[gerrit:932642|Update plwiki autopromote per consensus (T340397)]] (duration: 07m 48s) | 
  [production] | 
            
  | 07:25 | 
  <taavi@deploy1002> | 
  msz2001 and taavi: Backport for [[gerrit:932642|Update plwiki autopromote per consensus (T340397)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet | 
  [production] | 
            
  | 07:24 | 
  <taavi@deploy1002> | 
  Started scap: Backport for [[gerrit:932642|Update plwiki autopromote per consensus (T340397)]] | 
  [production] | 
            
  | 07:22 | 
  <taavi@deploy1002> | 
  Finished scap: Backport for [[gerrit:934723|Enable edit-in-sequence in Italian Wikisource (T340847)]] (duration: 18m 21s) | 
  [production] | 
            
  | 07:13 | 
  <taavi@deploy1002> | 
  soda and taavi: Backport for [[gerrit:934723|Enable edit-in-sequence in Italian Wikisource (T340847)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet | 
  [production] | 
            
  | 07:04 | 
  <taavi@deploy1002> | 
  Started scap: Backport for [[gerrit:934723|Enable edit-in-sequence in Italian Wikisource (T340847)]] | 
  [production] | 
            
  
    | 
      
        2023-06-30
      
      §
     | 
  
    
  | 22:34 | 
  <bking@cumin1001> | 
  END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) | 
  [production] | 
            
  | 22:20 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2013.* | 
  [production] | 
            
  | 22:20 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2014.* | 
  [production] | 
            
  | 22:20 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2015.* | 
  [production] | 
            
  | 22:19 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2016.* | 
  [production] | 
            
  | 22:19 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2017.* | 
  [production] | 
            
  | 22:19 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2018.* | 
  [production] | 
            
  | 22:19 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2019.* | 
  [production] | 
            
  | 22:19 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2020.* | 
  [production] | 
            
  | 22:19 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2021.* | 
  [production] | 
            
  | 22:19 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2022.* | 
  [production] | 
            
  | 22:09 | 
  <bking@deploy1002> | 
  Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 00m 47s) | 
  [production] | 
            
  | 22:08 | 
  <bking@deploy1002> | 
  Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 | 
  [production] | 
            
  | 22:08 | 
  <bking@deploy1002> | 
  deploy aborted: 0.3.124 (duration: 00m 00s) | 
  [production] | 
            
  | 22:08 | 
  <bking@deploy1002> | 
  Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 | 
  [production] | 
            
  | 22:00 | 
  <bking@deploy1002> | 
  Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 00m 05s) | 
  [production] | 
            
  | 22:00 | 
  <bking@deploy1002> | 
  Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 | 
  [production] | 
            
  | 21:58 | 
  <bking@deploy1002> | 
  Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 00m 05s) | 
  [production] | 
            
  | 21:58 | 
  <bking@deploy1002> | 
  Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 | 
  [production] | 
            
  | 21:07 | 
  <jhathaway> | 
  debugging a cert issue on pki1001.eqiad | 
  [production] | 
            
  | 21:03 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1004.wikimedia.org with reason: security release | 
  [production] | 
            
  | 21:00 | 
  <bking@cumin1001> | 
  START - Cookbook sre.wdqs.data-transfer | 
  [production] |