| 2025-10-27
      
      ยง | 
    
  | 12:13 | <slyngshede@cumin1003> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 12:08 | <slyngshede@cumin1003> | START - Cookbook sre.hosts.decommission for hosts idp-test1004.wikimedia.org | [production] | 
            
  | 11:43 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/ferretdb-growthbook: apply | [production] | 
            
  | 11:43 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/ferretdb-growthbook: apply | [production] | 
            
  | 11:39 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply | [production] | 
            
  | 11:39 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply | [production] | 
            
  | 11:37 | <zabe@deploy2002> | Finished scap sync-world: Backport for [[gerrit:1196112|Using Hadoop for MostTranscludedPages on testwiki (T309738)]] (duration: 11m 02s) | [production] | 
            
  | 11:31 | <zabe@deploy2002> | zabe: Continuing with sync | [production] | 
            
  | 11:30 | <zabe@deploy2002> | zabe: Backport for [[gerrit:1196112|Using Hadoop for MostTranscludedPages on testwiki (T309738)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 11:26 | <zabe@deploy2002> | Started scap sync-world: Backport for [[gerrit:1196112|Using Hadoop for MostTranscludedPages on testwiki (T309738)]] | [production] | 
            
  | 11:17 | <zabe@deploy2002> | Finished scap sync-world: Backport for [[gerrit:1198907|Correctly check if value is not false]] (duration: 12m 08s) | [production] | 
            
  | 11:09 | <zabe@deploy2002> | zabe: Continuing with sync | [production] | 
            
  | 11:08 | <zabe@deploy2002> | zabe: Backport for [[gerrit:1198907|Correctly check if value is not false]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 11:04 | <zabe@deploy2002> | Started scap sync-world: Backport for [[gerrit:1198907|Correctly check if value is not false]] | [production] | 
            
  | 11:00 | <slyngshede@cumin1003> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts idp2004.wikimedia.org | [production] | 
            
  | 10:59 | <slyngshede@cumin1003> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 10:59 | <slyngshede@cumin1003> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: idp2004.wikimedia.org decommissioned, removing all IPs except the asset tag one - slyngshede@cumin1003" | [production] | 
            
  | 10:59 | <slyngshede@cumin1003> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: idp2004.wikimedia.org decommissioned, removing all IPs except the asset tag one - slyngshede@cumin1003" | [production] | 
            
  | 10:53 | <slyngshede@cumin1003> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 10:48 | <slyngshede@cumin1003> | START - Cookbook sre.hosts.decommission for hosts idp2004.wikimedia.org | [production] | 
            
  | 10:34 | <slyngshede@cumin1003> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts idp1004.wikimedia.org | [production] | 
            
  | 10:34 | <slyngshede@cumin1003> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 10:34 | <slyngshede@cumin1003> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: idp1004.wikimedia.org decommissioned, removing all IPs except the asset tag one - slyngshede@cumin1003" | [production] | 
            
  | 10:33 | <slyngshede@cumin1003> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: idp1004.wikimedia.org decommissioned, removing all IPs except the asset tag one - slyngshede@cumin1003" | [production] | 
            
  | 10:29 | <slyngshede@cumin1003> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 10:24 | <slyngshede@cumin1003> | START - Cookbook sre.hosts.decommission for hosts idp1004.wikimedia.org | [production] | 
            
  | 09:57 | <cgoubert@deploy2002> | helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:57 | <cgoubert@deploy2002> | helmfile [codfw] START helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:57 | <cgoubert@deploy2002> | helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:56 | <cgoubert@deploy2002> | helmfile [eqiad] START helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:56 | <cgoubert@deploy2002> | helmfile [staging] DONE helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:56 | <cgoubert@deploy2002> | helmfile [staging] START helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:56 | <cgoubert@deploy2002> | helmfile [staging] DONE helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:56 | <cgoubert@deploy2002> | helmfile [staging] START helmfile.d/services/rest-gateway: apply | [production] | 
            
  | 09:40 | <slyngshede@cumin1003> | END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) for hosts idp1004.wikimedia.org | [production] | 
            
  | 09:39 | <slyngshede@cumin1003> | START - Cookbook sre.hosts.decommission for hosts idp1004.wikimedia.org | [production] | 
            
  | 09:37 | <fceratto@deploy2002> | helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . | [production] | 
            
  | 09:34 | <cmooney@cumin1003> | END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for ssw1-d8-eqiad | [production] | 
            
  | 09:34 | <cmooney@cumin1003> | START - Cookbook sre.hosts.remove-downtime for ssw1-d8-eqiad | [production] | 
            
  | 09:25 | <ammarpad@deploy2002> | mwscript-k8s job started: extensions/Translate/scripts/moveTranslatableBundle.php --wiki=metawiki 'Lingua Libre/SignIt' SignIt Ammarpad --reason 'requested at [[:phab:T408314]]'  # T408314 | [production] | 
            
  | 08:58 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/ferretdb-growthbook: apply | [production] | 
            
  | 08:57 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/ferretdb-growthbook: apply | [production] | 
            
  | 08:37 | <dcausse> | closing UTC morning backport window | [production] | 
            
  | 08:36 | <dcausse@deploy2002> | Finished scap sync-world: Backport for [[gerrit:1198295|Revert^2 "cirrus: enable completion search with defaultsort A/B test"]] (duration: 30m 01s) | [production] | 
            
  | 08:26 | <dcausse@deploy2002> | dcausse: Continuing with sync | [production] | 
            
  | 08:10 | <dcausse@deploy2002> | dcausse: Backport for [[gerrit:1198295|Revert^2 "cirrus: enable completion search with defaultsort A/B test"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 08:06 | <dcausse@deploy2002> | Started scap sync-world: Backport for [[gerrit:1198295|Revert^2 "cirrus: enable completion search with defaultsort A/B test"]] | [production] | 
            
  | 08:01 | <dcausse@deploy2002> | Finished scap sync-world: Backport for [[gerrit:1198404|ext.xLab: Implement UnenrolledExperiment#setStream()]], [[gerrit:1198413|ext.xLab: Implement OverriddenExperiment#setStream()]], [[gerrit:1198529|CompletionSuggester: fix index id format check (T404858)]] (duration: 46m 13s) | [production] | 
            
  | 07:55 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/ferretdb-growthbook: apply | [production] | 
            
  | 07:55 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/ferretdb-growthbook: apply | [production] |