| 
      
        2019-03-12
      
      ยง
     | 
  
    
  | 15:26 | 
  <ayounsi@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=dns2001.wikimedia.org,service=pdns_recursor | 
  [production] | 
            
  | 15:23 | 
  <ayounsi@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=dns2001.wikimedia.org,service=pdns_^Ccursor | 
  [production] | 
            
  | 15:02 | 
  <ppchelko@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 15:02 | 
  <ppchelko@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 15:02 | 
  <ppchelko@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 15:02 | 
  <ppchelko@deploy1001> | 
  scap-helm eventgate-analytics upgrade -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 15:00 | 
  <ppchelko@deploy1001> | 
  scap-helm eventgate-analytics upgrade -n staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 14:26 | 
  <mutante> | 
  phab1002 - reboot | 
  [production] | 
            
  | 13:43 | 
  <marostegui> | 
  Upgrade MySQL and kernel on db2094 (inactive sanitarium) | 
  [production] | 
            
  | 13:27 | 
  <marostegui> | 
  Deploy schema change on s6 codfw, lag will be generated on s6 codfw | 
  [production] | 
            
  | 13:24 | 
  <zfilipin@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.21 | 
  [production] | 
            
  | 12:40 | 
  <arturo> | 
  T215605 include python-mwclient .deb in openstack-mitaka-jessie/jessie-wikimedia in install1002 | 
  [production] | 
            
  | 12:23 | 
  <jynus> | 
  testing snapshotting on db1117:3325 -> dbstore1001 T210292 | 
  [production] | 
            
  | 12:23 | 
  <zfilipin@deploy1001> | 
  Finished scap: testwiki to php-1.33.0-wmf.21 and rebuild l10n cache (duration: 34m 25s) | 
  [production] | 
            
  | 12:09 | 
  <moritzm> | 
  upgrading mw1238-mw1258 to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) | 
  [production] | 
            
  | 11:59 | 
  <mutante> | 
  analytics-tool1004 - start superset service | 
  [production] | 
            
  | 11:48 | 
  <zfilipin@deploy1001> | 
  Started scap: testwiki to php-1.33.0-wmf.21 and rebuild l10n cache | 
  [production] | 
            
  | 11:47 | 
  <zfilipin@deploy1001> | 
  Pruned MediaWiki: 1.33.0-wmf.17 [keeping static files] (duration: 01m 40s) | 
  [production] | 
            
  | 11:45 | 
  <zfilipin@deploy1001> | 
  Pruned MediaWiki: 1.33.0-wmf.18 [keeping static files] (duration: 01m 35s) | 
  [production] | 
            
  | 11:42 | 
  <arturo> | 
  T215605 include python-oath .deb in stretch-wikimedia thirdparty/oath | 
  [production] | 
            
  | 11:41 | 
  <zfilipin@deploy1001> | 
  Pruned MediaWiki: 1.33.0-wmf.16 (duration: 12m 41s) | 
  [production] | 
            
  | 11:39 | 
  <elukey> | 
  raise mysql's max_user_connection to 1000 for the Analytics user on labsdb1012 | 
  [production] | 
            
  | 11:36 | 
  <ema@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=cp1077.eqiad.wmnet | 
  [production] | 
            
  | 11:36 | 
  <ema> | 
  cp1077: repool varnish-be after service restart T217893 | 
  [production] | 
            
  | 11:35 | 
  <arturo> | 
  delete wrong stretch-wikimedia `thirdparty` component in install1002 | 
  [production] | 
            
  | 11:12 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 11:11 | 
  <kartik@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:495842]] Add campaign prefix for EG tag (T216123) (duration: 00m 49s) | 
  [production] | 
            
  | 11:11 | 
  <moritzm> | 
  upgrading API servers/job runners servers in eqiad to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) | 
  [production] | 
            
  | 10:32 | 
  <marostegui> | 
  Deploy schema change on db1082, lag will happen on s5 on labs | 
  [production] | 
            
  | 10:29 | 
  <gtirloni> | 
  re-enabled puppet on serpens and seaborgium | 
  [production] | 
            
  | 10:18 | 
  <gtirloni> | 
  updated slapd to version 2.4.47 on seaborgium (T217280) | 
  [production] | 
            
  | 10:17 | 
  <moritzm> | 
  upgrading API servers/job runners servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) | 
  [production] | 
            
  | 10:14 | 
  <gtirloni> | 
  upgrading seaborgium to slapd 2.4.47 | 
  [production] | 
            
  | 09:39 | 
  <jynus> | 
  stop db1114 and restart it empty | 
  [production] | 
            
  | 09:25 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1082 (duration: 00m 48s) | 
  [production] | 
            
  | 08:57 | 
  <elukey> | 
  restart memcached on mc1019 to apply new settings - T217731 | 
  [production] | 
            
  | 08:50 | 
  <ema> | 
  cp1077 depooled again T217893 | 
  [production] | 
            
  | 08:49 | 
  <ema@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=cp1077.eqiad.wmnet | 
  [production] | 
            
  | 08:48 | 
  <moritzm> | 
  upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712) | 
  [production] | 
            
  | 08:48 | 
  <ema> | 
  restart varnish-be on cp1077 T217893 | 
  [production] | 
            
  | 08:47 | 
  <moritzm> | 
  upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) | 
  [production] | 
            
  | 08:46 | 
  <ema> | 
  cp1077 repooled T217893 | 
  [production] | 
            
  | 08:46 | 
  <ema@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=cp1077.eqiad.wmnet | 
  [production] | 
            
  | 08:36 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1082 for schema change (duration: 00m 48s) | 
  [production] | 
            
  | 08:34 | 
  <jynus> | 
  deploy core replica events to db1118 | 
  [production] | 
            
  | 08:15 | 
  <ema> | 
  cp1099: ferm.service failed to resolve prometheus1003.eqiad.wmnet. ferm restarted T202966 | 
  [production] | 
            
  | 07:18 | 
  <marostegui> | 
  Deploy schema change on db2052 (s5 codfw master), this will generate lag on codfw T71127 T51199 | 
  [production] | 
            
  | 07:17 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1113 after schema change and upgrade (duration: 00m 49s) | 
  [production] | 
            
  | 07:09 | 
  <marostegui> | 
  Upgrade mysql and kernel on db1113 | 
  [production] | 
            
  | 06:40 | 
  <kart_> | 
  Finished manual run of unpublished ContentTranslation draft purge script (T217818) | 
  [production] |