| 2022-05-16
      
      ยง | 
    
  | 22:14 | <jhathaway@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on mx2001.wikimedia.org with reason: exim debugging | [production] | 
            
  | 22:14 | <jhathaway@cumin1001> | START - Cookbook sre.hosts.downtime for 1:00:00 on mx2001.wikimedia.org with reason: exim debugging | [production] | 
            
  | 21:47 | <dzahn@cumin2002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 21:47 | <robh> | ganeti4002 rebooting for firmware update via T307997 | [production] | 
            
  | 21:44 | <dzahn@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 21:31 | <dzahn@cumin2002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 21:26 | <dzahn@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 21:14 | <dzahn@cumin2002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 21:08 | <dzahn@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 21:07 | <cstone> | civicrm revision changed from 6d85f1cc to d45afdfc | [production] | 
            
  | 21:05 | <mutante> | gerrit2002 (in setup) - rebooting | [production] | 
            
  | 20:46 | <mwdebug-deploy@deploy1002> | helmfile [codfw] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:45 | <mwdebug-deploy@deploy1002> | helmfile [codfw] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:45 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:44 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:41 | <catrope@deploy1002> | Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:792141|Revert "cirrus: Turn on AB test of wbsearchentities profiles" (T306644)]] (duration: 00m 51s) | [production] | 
            
  | 20:36 | <catrope@deploy1002> | Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:792197|yiwiktionary: Add localized mobile wordmark (T308411)]] and [[gerrit:792196|hewiktionary: Add localized mobile wordmark (T308411)]] (duration: 00m 50s) | [production] | 
            
  | 20:34 | <catrope@deploy1002> | Synchronized static/images/mobile/copyright/wiktionary-wordmark-yi.svg: Config: [[gerrit:792197|yiwiktionary: Add localized mobile wordmark (T308411)]] (duration: 00m 49s) | [production] | 
            
  | 20:33 | <catrope@deploy1002> | Synchronized static/images/mobile/copyright/wiktionary-wordmark-he.svg: Config: [[gerrit:792196|hewiktionary: Add localized mobile wordmark (T308411)]] (duration: 00m 50s) | [production] | 
            
  | 20:31 | <catrope@deploy1002> | Synchronized wmf-config/logos.php: Config: [[gerrit:792192|yiwiktionary: Update desktop logo (T308411)]] (duration: 00m 51s) | [production] | 
            
  | 20:29 | <mwdebug-deploy@deploy1002> | helmfile [codfw] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:29 | <catrope@deploy1002> | Synchronized static/images/project-logos/: Config: [[gerrit:792192|yiwiktionary: Update desktop logo (T308411)]] (duration: 00m 51s) | [production] | 
            
  | 20:28 | <mwdebug-deploy@deploy1002> | helmfile [codfw] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:28 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:27 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:20 | <catrope@deploy1002> | Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:791725|thwikibooks: Enable import (T308374)]] (duration: 00m 51s) | [production] | 
            
  | 20:14 | <catrope@deploy1002> | Synchronized wmf-config: Config: [[gerrit:792149|GrowthExperiments: Update campaigns benefit list config (T305659)]] (duration: 00m 51s) | [production] | 
            
  | 20:12 | <mwdebug-deploy@deploy1002> | helmfile [codfw] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:11 | <mwdebug-deploy@deploy1002> | helmfile [codfw] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:11 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 20:10 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:44 | <mwdebug-deploy@deploy1002> | helmfile [codfw] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:43 | <mwdebug-deploy@deploy1002> | helmfile [codfw] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:43 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:42 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:42 | <ladsgroup@deploy1002> | Synchronized php-1.39.0-wmf.10/includes/api/ApiQueryBacklinksprop.php: Backport: [[gerrit:792140|ApiQueryBacklinksprop: Make sure the index setting exists (T306673)]] (duration: 00m 50s) | [production] | 
            
  | 18:12 | <mwdebug-deploy@deploy1002> | helmfile [codfw] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:11 | <mwdebug-deploy@deploy1002> | helmfile [codfw] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:11 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 18:10 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 17:25 | <mutante> | ACKIng again all unhandled CRIT alerts on hosts with "dev" in their name - (imho dev hosts should not have prod CRIT alerts?) | [production] | 
            
  | 15:59 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts netbox-dev2001.wikimedia.org | [production] | 
            
  | 15:59 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 15:54 | <mwdebug-deploy@deploy1002> | helmfile [codfw] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 15:50 | <ayounsi@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 15:50 | <mwdebug-deploy@deploy1002> | helmfile [codfw] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 15:50 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply | [production] | 
            
  | 15:49 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] START helmfile.d/services/mwdebug: apply | [production] | 
            
  | 15:47 | <ayounsi@cumin1001> | START - Cookbook sre.hosts.decommission for hosts netbox-dev2001.wikimedia.org | [production] | 
            
  | 15:46 | <jdrewniak@deploy1002> | Synchronized portals: Wikimedia Portals Update: [[gerrit:792229| Bumping portals to master (T128546)]] (duration: 00m 51s) | [production] |