| 2021-08-18
      
      § | 
    
  | 16:28 | <cmjohnson@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 16:24 | <cmjohnson@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 15:48 | <cmjohnson@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 15:46 | <ejegg> | updated matching gift employers list on payments-wiki | [production] | 
            
  | 15:43 | <cmjohnson@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 14:50 | <jgiannelos@deploy1002> | helmfile [staging] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . | [production] | 
            
  | 14:26 | <effie> | enable puppet on alert* | [production] | 
            
  | 14:11 | <effie> | disable puppet on alerts* to avoid alert flood due to 713494 | [production] | 
            
  | 14:01 | <mwdebug-deploy@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | [production] | 
            
  | 13:57 | <jiji@deploy1002> | Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:713619|ProductionServices: change rdb* servers in eqiad and codfw (T280582)]] (duration: 01m 51s) | [production] | 
            
  | 13:57 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | [production] | 
            
  | 13:41 | <godog> | bounce logstash on logstash100[89] | [production] | 
            
  | 13:33 | <jgiannelos@deploy1002> | helmfile [staging] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . | [production] | 
            
  | 13:24 | <effie> | mw2383 is depooled - T286463 | [production] | 
            
  | 13:01 | <kormat> | Deploying wmfmariadbpy 0.7.2 T289139 | [production] | 
            
  | 13:01 | <kormat> | uploaded wmfmariadbpy 0.7.2 to apt.wm.o | [production] | 
            
  | 11:38 | <zpapierski@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . | [production] | 
            
  | 11:36 | <zpapierski@deploy1002> | helmfile [eqiad] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . | [production] | 
            
  | 11:35 | <zpapierski@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . | [production] | 
            
  | 11:12 | <jiji@cumin1001> | conftool action : set/pooled=inactive; selector: name=mw2383.codfw.wmnet | [production] | 
            
  | 11:03 | <zpapierski@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . | [production] | 
            
  | 10:47 | <effie> | pooling mw2383 - T286463 | [production] | 
            
  | 10:41 | <zpapierski@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . | [production] | 
            
  | 10:18 | <hnowlan@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on maps1004.eqiad.wmnet with reason: Awaiting decommissioning | [production] | 
            
  | 10:18 | <hnowlan@cumin1001> | START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on maps1004.eqiad.wmnet with reason: Awaiting decommissioning | [production] | 
            
  | 10:13 | <jiji@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2383.codfw.wmnet with reason: REIMAGE | [production] | 
            
  | 10:10 | <jiji@cumin1001> | START - Cookbook sre.hosts.downtime for 2:00:00 on mw2383.codfw.wmnet with reason: REIMAGE | [production] | 
            
  | 09:36 | <joal@deploy1002> | Finished deploy [analytics/refinery@88c6618] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@88c6618] (duration: 05m 48s) | [production] | 
            
  | 09:30 | <joal@deploy1002> | Started deploy [analytics/refinery@88c6618] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@88c6618] | [production] | 
            
  | 09:30 | <joal@deploy1002> | Finished deploy [analytics/refinery@88c6618] (thin): Regular analytics weekly train THIN [analytics/refinery@88c6618] (duration: 00m 07s) | [production] | 
            
  | 09:30 | <joal@deploy1002> | Started deploy [analytics/refinery@88c6618] (thin): Regular analytics weekly train THIN [analytics/refinery@88c6618] | [production] | 
            
  | 09:29 | <joal@deploy1002> | Finished deploy [analytics/refinery@88c6618]: Regular analytics weekly train [analytics/refinery@88c6618] (duration: 32m 29s) | [production] | 
            
  | 08:57 | <joal@deploy1002> | Started deploy [analytics/refinery@88c6618]: Regular analytics weekly train [analytics/refinery@88c6618] | [production] | 
            
  | 04:38 | <marostegui> | Drop user2 from s6 - T289051 | [production] | 
            
  | 02:03 | <rzl@cumin2001> | conftool action : get/pooled; selector: service=docker-registry | [production] | 
            
  | 00:39 | <dpifke@deploy1002> | Finished deploy [performance/navtiming@88f12a0]: Revert CpuBenchmark again (T281243) (duration: 00m 05s) | [production] | 
            
  | 00:39 | <dpifke@deploy1002> | Started deploy [performance/navtiming@88f12a0]: Revert CpuBenchmark again (T281243) | [production] | 
            
  | 00:38 | <dpifke@deploy1002> | Finished deploy [performance/navtiming@88f12a0]: Re-deploy fixed CpuBenchmark (T281243) (duration: 00m 06s) | [production] | 
            
  | 00:38 | <dpifke@deploy1002> | Started deploy [performance/navtiming@88f12a0]: Re-deploy fixed CpuBenchmark (T281243) | [production] | 
            
  
    | 2021-08-17
      
      § | 
    
  | 23:40 | <mwdebug-deploy@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | [production] | 
            
  | 23:33 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | [production] | 
            
  | 23:32 | <ebernhardson@deploy1002> | Synchronized php-1.37.0-wmf.19/extensions/CirrusSearch/maintenance/UpdateSuggesterIndex.php: T288233: Work around cache failure for wikitech (duration: 01m 28s) | [production] | 
            
  | 23:05 | <tzatziki> | resetting email for vanished user | [production] | 
            
  | 21:51 | <mwdebug-deploy@deploy1002> | helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | [production] | 
            
  | 21:44 | <mwdebug-deploy@deploy1002> | helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . | [production] | 
            
  | 21:44 | <urbanecm> | Deploy security patch for T289063 | [production] | 
            
  | 20:30 | <brennen> | running scap pull on mw2383 | [production] | 
            
  | 20:29 | <brennen@deploy1002> | Pruned MediaWiki: 1.37.0-wmf.16 (duration: 02m 01s) | [production] | 
            
  | 20:20 | <brennen@deploy1002> | Pruned MediaWiki: 1.37.0-wmf.15 (duration: 06m 51s) | [production] | 
            
  | 20:14 | <brennen> | pruning 1.37.0-wmf.15 and .16 (T281160) | [production] |