| 2019-11-06
      
      § | 
    
  | 09:10 | <onimisionipe@deploy1001> | Finished deploy [wdqs/wdqs@2cb2dde]: T233213 (duration: 11m 38s) | [production] | 
            
  | 08:58 | <onimisionipe@deploy1001> | Started deploy [wdqs/wdqs@2cb2dde]: T233213 | [production] | 
            
  | 08:51 | <jynus> | upgrading wmf-mariadb101-client on cumin hosts | [production] | 
            
  | 08:51 | <moritzm> | upgrading remaining mwdebug* servers to PHP 7.2.24  T237239 | [production] | 
            
  | 08:33 | <jynus> | upgrading db2102 mariadb (test-s1) | [production] | 
            
  | 07:48 | <onimisionipe@deploy1001> | Finished deploy [wdqs/wdqs@2cb2dde]: T233213 (duration: 11m 38s) | [production] | 
            
  | 07:37 | <onimisionipe@deploy1001> | Started deploy [wdqs/wdqs@2cb2dde]: T233213 | [production] | 
            
  | 02:59 | <vgutierrez> | Switch from nginx to ats-tls on cp5012 - T231627 | [production] | 
            
  | 00:07 | <mdholloway> | created table wikimedia_editor_tasks_edit_streak on x1/wikishared (T234956) | [production] | 
            
  
    | 2019-11-05
      
      § | 
    
  | 23:32 | <twentyafterfour@deploy1001> | rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.5  refs T233853 | [production] | 
            
  | 23:25 | <twentyafterfour@deploy1001> | Finished scap: testwikis wikis to 1.35.0-wmf.5  refs T233853 (duration: 24m 13s) | [production] | 
            
  | 23:01 | <twentyafterfour@deploy1001> | Started scap: testwikis wikis to 1.35.0-wmf.5  refs T233853 | [production] | 
            
  | 22:51 | <twentyafterfour@deploy1001> | scap failed: CalledProcessError Command 'cp -r "/tmp/scap_l10n_2905573311"/* "/srv/mediawiki-staging/php-1.35.0-wmf.5/cache/l10n"' returned non-zero exit status 1 (duration: 01m 26s) | [production] | 
            
  | 22:50 | <twentyafterfour@deploy1001> | Started scap: testwikis wikis to 1.35.0-wmf.5  refs T233853 | [production] | 
            
  | 22:39 | <twentyafterfour@deploy1001> | scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_2076118383" --threads=30 --lang en  --quiet' returned non-zero exit status 1 (duration: 01m 26s) | [production] | 
            
  | 22:38 | <twentyafterfour@deploy1001> | Started scap: testwikis wikis to 1.35.0-wmf.5  refs T233853 | [production] | 
            
  | 22:17 | <twentyafterfour> | scap failed with error: A copy of your installation's LocalSettings.php must exist and be readable in the source directory. Use --conf to specify it. refs T233853 | [production] | 
            
  | 22:09 | <twentyafterfour@deploy1001> | scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_840646293" --threads=30 --lang en  --quiet' returned non-zero exit status 1 (duration: 04m 54s) | [production] | 
            
  | 22:04 | <twentyafterfour@deploy1001> | Started scap: testwikis wikis to 1.35.0-wmf.5  refs T233853 | [production] | 
            
  | 22:03 | <XioNoX> | remove 127.0.0.1/32 and ::1/128 from cr2-esams:lo0.0 | [production] | 
            
  | 21:58 | <XioNoX> | remove 127.0.0.1/32 and ::1/128 from cr3-esams:lo0.0 | [production] | 
            
  | 20:45 | <mutante> | shutting down cobalt (formerly gerrit server) | [production] | 
            
  | 20:44 | <dzahn@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) | [production] | 
            
  | 20:43 | <dzahn@cumin1001> | START - Cookbook sre.hosts.decommission | [production] | 
            
  | 20:33 | <XioNoX> | push fw policies to pfw3-eqiad - T236201 | [production] | 
            
  | 20:23 | <XioNoX> | push fw policies to pfw3-codfw - T236201 | [production] | 
            
  | 20:17 | <joal@deploy1001> | Finished deploy [analytics/refinery@ea631bd]: Analytics deploy for spark upgrade - forgotten patch (duration: 08m 21s) | [production] | 
            
  | 20:09 | <joal@deploy1001> | Started deploy [analytics/refinery@ea631bd]: Analytics deploy for spark upgrade - forgotten patch | [production] | 
            
  | 20:08 | <joal@deploy1001> | Finished deploy [analytics/refinery@8013a86]: Analytics deploy for spark upgrade (duration: 08m 49s) | [production] | 
            
  | 20:00 | <joal@deploy1001> | Started deploy [analytics/refinery@8013a86]: Analytics deploy for spark upgrade | [production] | 
            
  | 18:40 | <twentyafterfour> | MediaWiki train: start branching wmf/1.35.0-wmf.5 | [production] | 
            
  | 18:30 | <XioNoX> | fix typo on cr1-eqsin:lo0.0 v6 IP | [production] | 
            
  | 18:27 | <ejegg> | updated payments-wiki from 0de9d96208 to aac3d93f70 | [production] | 
            
  | 17:21 | <jynus> | restarting etherpad | [production] | 
            
  | 16:56 | <arturo> | deleted stretch-wikimedia/thirdparty/kubeadm-k8s and created buster-wikimedia/thirdparty/kubeadm-k8s | [production] | 
            
  | 16:24 | <papaul> | Replacing disk on db2120 | [production] | 
            
  | 15:37 | <jynus> | deploying schema change on x1 T234955 | [production] | 
            
  | 15:20 | <ema> | cp4027: upgrade trafficserver to 8.0.5-1wm10 | [production] | 
            
  | 14:37 | <jynus> | reducing consistency temporarilly on db1114 so it can catch up replication | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 11:58 | <aborrero@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 11:57 | <aborrero@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 11:57 | <aborrero@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 10:59 | <ema> | pool cp5012 with ATS backend T227432 | [production] |