| 2018-07-31
      
      ยง | 
    
  | 13:03 | <akosiaris> | upgrade kubernetes1001 to 1.9.9 | [production] | 
            
  | 12:58 | <akosiaris> | upgrade chlorine.eqiad.wmnet to kubernetes 1.9.9 | [production] | 
            
  | 12:54 | <akosiaris> | upgrade argon.eqiad.wmnet to kubernetes 1.9.9 | [production] | 
            
  | 12:45 | <akosiaris> | depool eqiad mathoid in discovery | [production] | 
            
  | 12:29 | <gehel@puppetmaster1001> | conftool action : set/weight=15; selector: dc=eqiad,cluster=wdqs,name=wdqs1005.eqiad.wmnet | [production] | 
            
  | 12:29 | <gehel@puppetmaster1001> | conftool action : set/weight=15; selector: dc=eqiad,cluster=wdqs,name=wdqs1004.eqiad.wmnet | [production] | 
            
  | 12:29 | <gehel> | rebalance LVS weights to send less traffic to wdqs1003 - T200563 | [production] | 
            
  | 12:13 | <reedy@deploy1001> | Synchronized wmf-config/CommonSettings.php: Remove logging now in mw core (duration: 00m 48s) | [production] | 
            
  | 12:11 | <reedy@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: wmg - for Sentry (duration: 00m 48s) | [production] | 
            
  | 12:09 | <reedy@deploy1001> | Synchronized wmf-config/CommonSettings.php: Remove wmg - for Sentry (duration: 00m 48s) | [production] | 
            
  | 12:06 | <reedy@deploy1001> | Synchronized wmf-config/CommonSettings.php: Load Sentry with wfLoadExtension (duration: 00m 48s) | [production] | 
            
  | 12:05 | <reedy@deploy1001> | Synchronized wmf-config/liquidthreads.php: Remove lqt cruft, load with wfLoadExtension (duration: 00m 48s) | [production] | 
            
  | 11:44 | <gehel> | restarting blazegraph on wdqs1003 - JVM unresponsive | [production] | 
            
  | 11:38 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 11:37 | <zfilipin@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:448050|Introduce autopatrolled on bnwikisource (T199475)]] (duration: 00m 48s) | [production] | 
            
  | 11:30 | <zfilipin@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:449391|Use HD logos for hewiktionary (T200470)]] (duration: 00m 48s) | [production] | 
            
  | 11:24 | <zfilipin@deploy1001> | Synchronized static/images/project-logos/: SWAT: [[gerrit:449390|Upload HD logos for hewiktionary (T200470)]] (duration: 00m 48s) | [production] | 
            
  | 11:16 | <zfilipin@deploy1001> | Synchronized static/images/project-logos: SWAT: [[gerrit:449389|Milestone logo for atjwiki (T200713)]] (duration: 00m 49s) | [production] | 
            
  | 11:10 | <zfilipin@deploy1001> | Synchronized static/images/project-logos/hewikiquote.png: SWAT: [[gerrit:448154|Update hewikiquote logo (T200296)]] (duration: 00m 50s) | [production] | 
            
  | 10:55 | <_joe_> | test log | [production] | 
            
  | 10:40 | <akosiaris@deploy1001> | scap-helm mathoid finished | [production] | 
            
  | 10:40 | <akosiaris@deploy1001> | scap-helm mathoid cluster codfw completed | [production] | 
            
  | 10:39 | <akosiaris@deploy1001> | scap-helm mathoid cluster eqiad completed | [production] | 
            
  | 10:39 | <akosiaris@deploy1001> | scap-helm mathoid upgrade production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] | [production] | 
            
  | 09:46 | <ema> | reboot cp1045 (jessie) for kernel updates | [production] | 
            
  | 09:29 | <ema> | upload varnishkafka 1.0.13-1 to stretch-wikimedia T200445 T186250 | [production] | 
            
  | 08:44 | <jynus> | stop and reimage db1104 to stretch | [production] | 
            
  | 08:17 | <akosiaris> | repool mathoid codfw in discovery, the kubernetes upgrade is done | [production] | 
            
  | 08:08 | <akosiaris> | upgrade kubernetes2004 to 1.9.9 | [production] | 
            
  | 08:05 | <akosiaris> | upgrade kubernetes2003 to 1.9.9 | [production] | 
            
  | 08:00 | <akosiaris> | upgrade kubernetes2002 to 1.9.9 | [production] | 
            
  | 07:55 | <ema> | reboot cp2018 for kernel updates | [production] | 
            
  | 07:54 | <moritzm> | reboot cp1008 for kernel tests | [production] | 
            
  | 07:53 | <akosiaris> | upgrade kubernetes2001 to 1.9.9 | [production] | 
            
  | 07:50 | <ema> | reboot cp2025 for kernel updates | [production] | 
            
  | 07:47 | <akosiaris> | upgrade acrux.codfw.wmnet to kubernetes 1.9.9 | [production] | 
            
  | 07:44 | <jynus> | stopping db1118 mariadb and starting mysql there | [production] | 
            
  | 07:36 | <moritzm> | reboot multatuli for kernel tests | [production] | 
            
  | 07:35 | <akosiaris> | upgrade acrab.codfw.wmnet to kubernetes 1.9.9 | [production] | 
            
  | 07:32 | <moritzm> | run decomission_appserver on terbium | [production] | 
            
  | 07:27 | <akosiaris> | backup kubetcd2001 etcd data before kubernetes upgrade | [production] | 
            
  | 07:24 | <akosiaris> | depool codfw mathoid from discovery for kubernetes upgrade | [production] | 
            
  | 07:10 | <moritzm> | installing mutt security updates | [production] | 
            
  | 07:05 | <marostegui> | Remove unused and old grants from codfw hosts - T146149#4456082 | [production] | 
            
  | 06:49 | <jynus> | dropping unnecesary tendril web users from tendril db backends | [production] | 
            
  | 06:45 | <jynus@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool row B es hosts, reduce db1089 load (duration: 00m 49s) | [production] | 
            
  | 06:11 | <moritzm> | installing ant security updates | [production] | 
            
  | 05:01 | <marostegui> | Deploy schema change on db1099:3318 T144010 T51190 T199368 | [production] | 
            
  | 05:00 | <marostegui> | Deploy schema change on dbstore1002:s8 T144010 T51190 T199368/script unload irssinotifier | [production] | 
            
  | 04:54 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool all hosts in row B - T183585 (duration: 00m 50s) | [production] |