| 2023-07-12
      
      ยง | 
    
  | 13:34 | <bking@cumin1001> | START - Cookbook sre.wdqs.data-transfer | [production] | 
            
  | 13:31 | <btullis@cumin1001> | END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) for Druid analytics cluster: Roll restart of Druid jvm daemons. | [production] | 
            
  | 13:31 | <lucaswerkmeister-wmde@deploy1002> | daimona and lucaswerkmeister-wmde: Backport for [[gerrit:937123|Add new campaign_events.event_answers_status column (T341142)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet | [production] | 
            
  | 13:30 | <lucaswerkmeister-wmde@deploy1002> | Started scap: Backport for [[gerrit:937123|Add new campaign_events.event_answers_status column (T341142)]] | [production] | 
            
  | 13:29 | <lucaswerkmeister-wmde@deploy1002> | Finished scap: Backport for [[gerrit:937121|Html: Support more attr types in getTextInputAttributes() (T341566)]] (duration: 07m 40s) | [production] | 
            
  | 13:28 | <bking@deploy1002> | Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 00m 47s) | [production] | 
            
  | 13:27 | <bking@deploy1002> | Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 | [production] | 
            
  | 13:27 | <bking@deploy1002> | Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 02m 30s) | [production] | 
            
  | 13:24 | <bking@deploy1002> | Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 | [production] | 
            
  | 13:24 | <bking@deploy1002> | Finished deploy [wdqs/wdqs@dff41b7]: 0.3.124 (duration: 02m 09s) | [production] | 
            
  | 13:23 | <lucaswerkmeister-wmde@deploy1002> | lucaswerkmeister-wmde: Backport for [[gerrit:937121|Html: Support more attr types in getTextInputAttributes() (T341566)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet | [production] | 
            
  | 13:22 | <bking@deploy1002> | Started deploy [wdqs/wdqs@dff41b7]: 0.3.124 | [production] | 
            
  | 13:21 | <lucaswerkmeister-wmde@deploy1002> | Started scap: Backport for [[gerrit:937121|Html: Support more attr types in getTextInputAttributes() (T341566)]] | [production] | 
            
  | 13:06 | <moritzm> | installing node-tough-cookie security updates | [production] | 
            
  | 12:54 | <moritzm> | rebalance ganeti codfw/D following reboots | [production] | 
            
  | 12:52 | <moritzm> | imported wikidiff2 1.14.1-0+wmf1+buster1+icu67u1 to component/icu67 T340087 T329491 | [production] | 
            
  | 12:44 | <btullis@cumin1001> | START - Cookbook sre.druid.roll-restart-workers for Druid analytics cluster: Roll restart of Druid jvm daemons. | [production] | 
            
  | 12:30 | <akosiaris> | upgrade wikidiff2 1.13.0-1+wmf1+buster1 -> 1.14.1-0+wmf1+buster1 on mw-canary hosts T340087 | [production] | 
            
  | 12:11 | <mvernon@cumin2002> | conftool action : set/pooled=no; selector: name=thanos-fe1002.eqiad.wmnet,service=thanos-web | [production] | 
            
  | 11:50 | <moritzm> | installing apache2 security updates on Bullseye | [production] | 
            
  | 11:48 | <moritzm> | installing wireshark security updates | [production] | 
            
  | 11:44 | <moritzm> | rebalance ganeti codfw/C following reboots | [production] | 
            
  | 11:43 | <hnowlan> | rebuilding fluent-bit image to include wmf-certificates | [production] | 
            
  | 11:33 | <ariel@cumin1001> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host htmldumper1001.eqiad.wmnet | [production] | 
            
  | 11:26 | <ariel@cumin1001> | START - Cookbook sre.hosts.reboot-single for host htmldumper1001.eqiad.wmnet | [production] | 
            
  | 11:00 | <ladsgroup@deploy1002> | Finished scap: Backport for [[gerrit:937122|fix: add request headers properly (T319170)]] (duration: 10m 20s) | [production] | 
            
  | 10:51 | <ladsgroup@deploy1002> | ladsgroup: Backport for [[gerrit:937122|fix: add request headers properly (T319170)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet | [production] | 
            
  | 10:50 | <ladsgroup@deploy1002> | Started scap: Backport for [[gerrit:937122|fix: add request headers properly (T319170)]] | [production] | 
            
  | 10:49 | <ladsgroup@deploy1002> | Finished scap: Backport for [[gerrit:937108|Externallinks: Keep domain wildcard if path is not specified (T326251)]] (duration: 08m 09s) | [production] | 
            
  | 10:42 | <ladsgroup@deploy1002> | ladsgroup: Backport for [[gerrit:937108|Externallinks: Keep domain wildcard if path is not specified (T326251)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet | [production] | 
            
  | 10:40 | <ladsgroup@deploy1002> | Started scap: Backport for [[gerrit:937108|Externallinks: Keep domain wildcard if path is not specified (T326251)]] | [production] | 
            
  | 09:35 | <isaranto@deploy1002> | helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . | [production] | 
            
  | 09:35 | <isaranto@deploy1002> | helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . | [production] | 
            
  | 09:34 | <isaranto@deploy1002> | helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . | [production] | 
            
  | 09:33 | <isaranto@deploy1002> | helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . | [production] | 
            
  | 08:55 | <moritzm> | move secondary instances away from ganeti2014 T341546 | [production] | 
            
  | 07:29 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.network.tls (exit_code=0) | [production] | 
            
  | 07:29 | <ayounsi@cumin1001> | START - Cookbook sre.network.tls | [production] | 
            
  | 07:26 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.network.tls (exit_code=0) | [production] | 
            
  | 07:26 | <ayounsi@cumin1001> | START - Cookbook sre.network.tls | [production] | 
            
  | 06:55 | <ayounsi@cumin1001> | END (FAIL) - Cookbook sre.network.tls (exit_code=99) | [production] | 
            
  | 06:55 | <ayounsi@cumin1001> | START - Cookbook sre.network.tls | [production] | 
            
  | 06:47 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.network.tls (exit_code=0) | [production] | 
            
  | 06:47 | <ayounsi@cumin1001> | START - Cookbook sre.network.tls | [production] | 
            
  | 06:45 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.network.tls (exit_code=0) | [production] | 
            
  | 06:45 | <ayounsi@cumin1001> | START - Cookbook sre.network.tls | [production] | 
            
  | 06:37 | <ayounsi@cumin1001> | END (FAIL) - Cookbook sre.network.tls (exit_code=99) | [production] | 
            
  | 06:37 | <ayounsi@cumin1001> | START - Cookbook sre.network.tls | [production] | 
            
  | 06:18 | <ayounsi@cumin1001> | END (FAIL) - Cookbook sre.network.tls (exit_code=99) | [production] | 
            
  | 06:18 | <ayounsi@cumin1001> | START - Cookbook sre.network.tls | [production] |