| 
      
        2019-03-05
      
      §
     | 
  
    
  | 23:51 | 
  <ejegg> | 
  updated payments-wiki from 4f2935ad17 to f1a89d7045 | 
  [production] | 
            
  | 21:05 | 
  <godog> | 
  temporarily stop requests to k8s instance on prometheus2004 | 
  [production] | 
            
  | 21:00 | 
  <herron> | 
  restarted apache on grafana1001 | 
  [production] | 
            
  | 20:43 | 
  <herron> | 
  retarted apache on grafana1001 | 
  [production] | 
            
  | 19:56 | 
  <hashar@deploy1001> | 
  Synchronized php-1.33.0-wmf.20/extensions/LdapAuthentication/: Stop referring to the now-killed AuthPlugin class - T217692 (duration: 00m 57s) | 
  [production] | 
            
  | 17:44 | 
  <godog> | 
  bounce uwsgi on graphite1004 | 
  [production] | 
            
  | 17:25 | 
  <herron> | 
  restarting uwsgi-graphite-web on graphite1004 | 
  [production] | 
            
  | 16:54 | 
  <moritzm> | 
  imported logstash 1:5.6.14-1 to thirdparty/elastic56 | 
  [production] | 
            
  | 16:52 | 
  <herron> | 
  restarting uwsgi-graphite-web on graphite1004 | 
  [production] | 
            
  | 16:43 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 16:43 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 16:43 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging stable/eventgate-analytics -f eventgate-analytics-staging-values.yaml [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 16:20 | 
  <herron> | 
  restarting uwsgi-graphite-web on graphite1004 | 
  [production] | 
            
  | 15:53 | 
  <hashar@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.20 | 
  [production] | 
            
  | 15:35 | 
  <hashar@deploy1001> | 
  Finished scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 (duration: 51m 03s) | 
  [production] | 
            
  | 14:52 | 
  <gtirloni> | 
  reprepro added bdsync_0.10-1+deb9u1 T209527 | 
  [production] | 
            
  | 14:44 | 
  <hashar@deploy1001> | 
  Started scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 | 
  [production] | 
            
  | 14:42 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 14:42 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster eqiad completed | 
  [production] | 
            
  | 14:42 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade production stable/eventgate-analytics -f eventgate-analytics-eqiad-values.yaml [namespace: eventgate-analytics, clusters: eqiad] | 
  [production] | 
            
  | 14:41 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 14:41 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster codfw completed | 
  [production] | 
            
  | 14:41 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade production stable/eventgate-analytics -f eventgate-analytics-codfw-values.yaml [namespace: eventgate-analytics, clusters: codfw] | 
  [production] | 
            
  | 14:40 | 
  <jiji@cumin1001> | 
  conftool action : set/pooled=yes; selector: dc=codfw,service=citoid,cluster=scb,name=kubernetes.* | 
  [production] | 
            
  | 14:35 | 
  <hashar@deploy1001> | 
  scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="cawikibooks" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.BRPBtKvzZH" --verbose' returned non-zero exit status 1 (duration: 00m 20s) | 
  [production] | 
            
  | 14:35 | 
  <hashar@deploy1001> | 
  Started scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 | 
  [production] | 
            
  | 14:34 | 
  <jijiki> | 
  Rump up citoid traffic from k8s to 25% on codfw - T213194 | 
  [production] | 
            
  | 14:34 | 
  <hashar@deploy1001> | 
  scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="cawikibooks" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.ngh6XIMz8y" --verbose' returned non-zero exit status 1 (duration: 00m 21s) | 
  [production] | 
            
  | 14:33 | 
  <hashar@deploy1001> | 
  Started scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 | 
  [production] | 
            
  | 14:33 | 
  <jiji@cumin1001> | 
  conftool action : set/weight=5; selector: dc=codfw,service=citoid,cluster=scb,name=kubernetes.* | 
  [production] | 
            
  | 14:27 | 
  <hashar@deploy1001> | 
  scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="cawikibooks" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.JrfRQw0oDJ" --verbose' returned non-zero exit status 1 (duration: 00m 21s) | 
  [production] | 
            
  | 14:27 | 
  <hashar@deploy1001> | 
  Started scap: testwiki to php-1.33.0-wmf.20 and rebuild l10n cache # T206674 | 
  [production] | 
            
  | 14:25 | 
  <hashar@deploy1001> | 
  Pruned MediaWiki: 1.33.0-wmf.14 (duration: 09m 47s) | 
  [production] | 
            
  | 14:20 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 14:20 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster codfw completed | 
  [production] | 
            
  | 14:20 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster eqiad completed | 
  [production] | 
            
  | 14:20 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics  [namespace: eventgate-analytics, clusters: eqiad,codfw] | 
  [production] | 
            
  | 14:17 | 
  <hashar@deploy1001> | 
  scap failed: LockFailedError Failed to acquire lock "/var/lock/scap.operations_mediawiki-config.lock"; owner is "hashar"; reason is "Pruned MediaWiki: 1.33.0-wmf.14" (duration: 00m 00s) | 
  [production] | 
            
  | 14:14 | 
  <hashar> | 
  Applied wmf/1.33.0-wmf.20 local patches # T206674 | 
  [production] | 
            
  | 14:12 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1064 T217591 (duration: 01m 50s) | 
  [production] | 
            
  | 13:31 | 
  <hashar> | 
  Cutting branch wmf/1.33.0-wmf.20 # T206674 | 
  [production] | 
            
  | 13:16 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1064 T217591 (duration: 00m 48s) | 
  [production] | 
            
  | 13:14 | 
  <ema> | 
  lvs500[12]: upgrade linux to 4.9.144-3.1, reboot for L1TF kernel/microcode updates T203011 | 
  [production] | 
            
  | 13:07 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 12:58 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:487115|Set wgArticleCountMethod=any for zhwikiversity (T214946)]] (duration: 00m 49s) | 
  [production] | 
            
  | 12:45 | 
  <kartik@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert "Enable edittag for ExternalGuidance in CX and VE"  (duration: 00m 48s) | 
  [production] | 
            
  | 12:24 | 
  <kartik@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Revert gerrit:493155 (duration: 00m 49s) | 
  [production] | 
            
  | 11:59 | 
  <_joe_> | 
  upgrading scap everywhere to 3.9.2-1, T217611 | 
  [production] |