| 
      
        2018-07-31
      
      ยง
     | 
  
    
  | 15:50 | 
  <ayounsi@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: dc=eqiad,cluster=aqs,service=aqs,name=aqs1004.eqiad.wmnet | 
  [production] | 
            
  | 15:46 | 
  <ayounsi@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: dc=eqiad,cluster=eventbus,service=eventbus,name=kafka1002.eqiad.wmnet | 
  [production] | 
            
  | 15:45 | 
  <ayounsi@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: dc=eqiad,cluster=prometheus,service=prometheus,name=prometheus1004.eqiad.wmnet | 
  [production] | 
            
  | 15:36 | 
  <ema> | 
  power-cycle cp3031, stuck rebooting | 
  [production] | 
            
  | 15:29 | 
  <_joe_> | 
  restarting apache on phab1001 | 
  [production] | 
            
  | 15:20 | 
  <godog> | 
  depool thumbor100[12] ahead of switch move - T183585 | 
  [production] | 
            
  | 15:01 | 
  <XioNoX> | 
  starting the eqiad row B servers move - T183585 | 
  [production] | 
            
  | 13:51 | 
  <moritzm> | 
  installing ffmpeg 3.2.12 security updates on video scalers | 
  [production] | 
            
  | 13:34 | 
  <akosiaris> | 
  repool mathoid eqiad cluster in discovery | 
  [production] | 
            
  | 13:18 | 
  <akosiaris> | 
  upgrade kubernetes1004 to 1.9.9 | 
  [production] | 
            
  | 13:14 | 
  <akosiaris> | 
  upgrade kubernetes1002 to 1.9.9 | 
  [production] | 
            
  | 13:14 | 
  <akosiaris> | 
  upgrade kubernetes1003 to 1.9.9 | 
  [production] | 
            
  | 13:03 | 
  <akosiaris> | 
  upgrade kubernetes1001 to 1.9.9 | 
  [production] | 
            
  | 12:58 | 
  <akosiaris> | 
  upgrade chlorine.eqiad.wmnet to kubernetes 1.9.9 | 
  [production] | 
            
  | 12:54 | 
  <akosiaris> | 
  upgrade argon.eqiad.wmnet to kubernetes 1.9.9 | 
  [production] | 
            
  | 12:45 | 
  <akosiaris> | 
  depool eqiad mathoid in discovery | 
  [production] | 
            
  | 12:29 | 
  <gehel@puppetmaster1001> | 
  conftool action : set/weight=15; selector: dc=eqiad,cluster=wdqs,name=wdqs1005.eqiad.wmnet | 
  [production] | 
            
  | 12:29 | 
  <gehel@puppetmaster1001> | 
  conftool action : set/weight=15; selector: dc=eqiad,cluster=wdqs,name=wdqs1004.eqiad.wmnet | 
  [production] | 
            
  | 12:29 | 
  <gehel> | 
  rebalance LVS weights to send less traffic to wdqs1003 - T200563 | 
  [production] | 
            
  | 12:13 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: Remove logging now in mw core (duration: 00m 48s) | 
  [production] | 
            
  | 12:11 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: wmg - for Sentry (duration: 00m 48s) | 
  [production] | 
            
  | 12:09 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: Remove wmg - for Sentry (duration: 00m 48s) | 
  [production] | 
            
  | 12:06 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: Load Sentry with wfLoadExtension (duration: 00m 48s) | 
  [production] | 
            
  | 12:05 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/liquidthreads.php: Remove lqt cruft, load with wfLoadExtension (duration: 00m 48s) | 
  [production] | 
            
  | 11:44 | 
  <gehel> | 
  restarting blazegraph on wdqs1003 - JVM unresponsive | 
  [production] | 
            
  | 11:38 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 11:37 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:448050|Introduce autopatrolled on bnwikisource (T199475)]] (duration: 00m 48s) | 
  [production] | 
            
  | 11:30 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:449391|Use HD logos for hewiktionary (T200470)]] (duration: 00m 48s) | 
  [production] | 
            
  | 11:24 | 
  <zfilipin@deploy1001> | 
  Synchronized static/images/project-logos/: SWAT: [[gerrit:449390|Upload HD logos for hewiktionary (T200470)]] (duration: 00m 48s) | 
  [production] | 
            
  | 11:16 | 
  <zfilipin@deploy1001> | 
  Synchronized static/images/project-logos: SWAT: [[gerrit:449389|Milestone logo for atjwiki (T200713)]] (duration: 00m 49s) | 
  [production] | 
            
  | 11:10 | 
  <zfilipin@deploy1001> | 
  Synchronized static/images/project-logos/hewikiquote.png: SWAT: [[gerrit:448154|Update hewikiquote logo (T200296)]] (duration: 00m 50s) | 
  [production] | 
            
  | 10:55 | 
  <_joe_> | 
  test log | 
  [production] | 
            
  | 10:40 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid finished | 
  [production] | 
            
  | 10:40 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid cluster codfw completed | 
  [production] | 
            
  | 10:39 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid cluster eqiad completed | 
  [production] | 
            
  | 10:39 | 
  <akosiaris@deploy1001> | 
  scap-helm mathoid upgrade production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] | 
  [production] | 
            
  | 09:46 | 
  <ema> | 
  reboot cp1045 (jessie) for kernel updates | 
  [production] | 
            
  | 09:29 | 
  <ema> | 
  upload varnishkafka 1.0.13-1 to stretch-wikimedia T200445 T186250 | 
  [production] | 
            
  | 08:44 | 
  <jynus> | 
  stop and reimage db1104 to stretch | 
  [production] | 
            
  | 08:17 | 
  <akosiaris> | 
  repool mathoid codfw in discovery, the kubernetes upgrade is done | 
  [production] | 
            
  | 08:08 | 
  <akosiaris> | 
  upgrade kubernetes2004 to 1.9.9 | 
  [production] | 
            
  | 08:05 | 
  <akosiaris> | 
  upgrade kubernetes2003 to 1.9.9 | 
  [production] | 
            
  | 08:00 | 
  <akosiaris> | 
  upgrade kubernetes2002 to 1.9.9 | 
  [production] | 
            
  | 07:55 | 
  <ema> | 
  reboot cp2018 for kernel updates | 
  [production] | 
            
  | 07:54 | 
  <moritzm> | 
  reboot cp1008 for kernel tests | 
  [production] | 
            
  | 07:53 | 
  <akosiaris> | 
  upgrade kubernetes2001 to 1.9.9 | 
  [production] | 
            
  | 07:50 | 
  <ema> | 
  reboot cp2025 for kernel updates | 
  [production] | 
            
  | 07:47 | 
  <akosiaris> | 
  upgrade acrux.codfw.wmnet to kubernetes 1.9.9 | 
  [production] | 
            
  | 07:44 | 
  <jynus> | 
  stopping db1118 mariadb and starting mysql there | 
  [production] | 
            
  | 07:36 | 
  <moritzm> | 
  reboot multatuli for kernel tests | 
  [production] |