| 
      
        2018-04-20
      
      §
     | 
  
    
  | 20:45 | 
  <andrewbogott> | 
  re-imaging labvirt1021 and 1022 as Jessie | 
  [production] | 
            
  | 20:23 | 
  <ejegg> | 
  updated fundraising python tools from 0c50f9e38f to 7c5c7a5f9e | 
  [production] | 
            
  | 18:23 | 
  <mutante> | 
  add LDAP user "tieu" to group "wmde" (T192256) | 
  [production] | 
            
  | 17:42 | 
  <imarlier@tin> | 
  Finished deploy [performance/coal@99db58f]: coal - update to submit via graphite.  Not yet active, requires puppet changes (duration: 00m 04s) | 
  [production] | 
            
  | 17:41 | 
  <imarlier@tin> | 
  Started deploy [performance/coal@99db58f]: coal - update to submit via graphite.  Not yet active, requires puppet changes | 
  [production] | 
            
  | 17:35 | 
  <no_justification> | 
  gerrit: update mysql-client and deps 5.5.59 -> 5.5.60 | 
  [production] | 
            
  | 17:28 | 
  <mutante> | 
  phabricator - restarted apache | 
  [production] | 
            
  | 17:26 | 
  <mutante> | 
  phabricator (phab1001) - upgrading Apache, openssl, mysql-common | 
  [production] | 
            
  | 17:17 | 
  <mutante> | 
  phab2001 - upgrading apache, openssl, mysql-common | 
  [production] | 
            
  | 17:04 | 
  <andrewbogott> | 
  rebooting labvirt1021 and 1022 | 
  [production] | 
            
  | 16:44 | 
  <dcausse@tin> | 
  Synchronized php-1.31.0-wmf.30/extensions/CirrusSearch/: T192609: Do not propagate Elastica doc modifications out of DataSender (duration: 01m 34s) | 
  [production] | 
            
  | 15:07 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2087 (duration: 01m 16s) | 
  [production] | 
            
  | 14:41 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2086, depool db2087 (duration: 01m 16s) | 
  [production] | 
            
  | 14:16 | 
  <andrew@tin> | 
  Synchronized dblists: Purging obsolete silver.dblist (duration: 01m 17s) | 
  [production] | 
            
  | 14:02 | 
  <moritzm> | 
  upgrading labweb* servers to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 14:00 | 
  <jynus> | 
  upgrade and restart db2086 | 
  [production] | 
            
  | 13:59 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Depool db2086 (duration: 01m 13s) | 
  [production] | 
            
  | 13:35 | 
  <anomie> | 
  (re-)creating `slots` table on all wikis, following up T190153 and T184446#4143097 | 
  [production] | 
            
  | 13:25 | 
  <moritzm> | 
  upgrading mysql (as shipped in Debian) on bohrium | 
  [production] | 
            
  | 13:00 | 
  <moritzm> | 
  installing zsh security updates on trusty servers | 
  [production] | 
            
  | 12:25 | 
  <moritzm> | 
  upgrading apache on auth* servers | 
  [production] | 
            
  | 12:18 | 
  <jynus> | 
  upgrading and restarting dbstore2002 | 
  [production] | 
            
  | 12:15 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 01m 17s) | 
  [production] | 
            
  | 12:06 | 
  <moritzm> | 
  installing apache security updates on video scalers | 
  [production] | 
            
  | 12:05 | 
  <moritzm> | 
  upgrading apache on einsteinium/icinga.wikimedia.org | 
  [production] | 
            
  | 11:53 | 
  <moritzm> | 
  installing apache security updates on netmon1002/2001 | 
  [production] | 
            
  | 11:27 | 
  <elukey> | 
  reimage analytics1068 to Debian Stretch - T192557 | 
  [production] | 
            
  | 11:06 | 
  <moritzm> | 
  installing tiff security updates on trusty | 
  [production] | 
            
  | 09:58 | 
  <godog> | 
  upload scap 3.8.0-2 - T192124 | 
  [production] | 
            
  | 09:51 | 
  <moritzm> | 
  upgrading deployment servers to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 09:41 | 
  <jynus> | 
  starting reimage of db2070 | 
  [production] | 
            
  | 09:41 | 
  <moritzm> | 
  upgrading mwdebug servers to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 09:33 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2071, depool db2070 (duration: 01m 16s) | 
  [production] | 
            
  | 09:12 | 
  <elukey> | 
  restart of mw apis showing ~50% cpu utilization as precaution before the weekend - mw[1224,1225,1228,1230,1231,1233-1235,1276-1283,1286,1312,1313,1315,1316,1341,1343,1344,1347,1348]* | 
  [production] | 
            
  | 09:06 | 
  <moritzm> | 
  upgrading video scalers in codfw to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 08:41 | 
  <moritzm> | 
  upgrading job runners in codfw to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 08:39 | 
  <marostegui> | 
  Going to sanitize gorwiki euwikisource romdwikimedia inhwiki on db1095 - T189112 T189466 T187774 T184375 | 
  [production] | 
            
  | 08:39 | 
  <elukey> | 
  restart hhvm on mw[1226,1232].eqiad.wmnet - high load | 
  [production] | 
            
  | 08:05 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Fully repool db1114 in API - T191996 (duration: 01m 16s) | 
  [production] | 
            
  | 07:57 | 
  <jynus> | 
  starting reimage of db2071 | 
  [production] | 
            
  | 07:52 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-codfw.php: Depool db2071 (duration: 01m 16s) | 
  [production] | 
            
  | 07:48 | 
  <moritzm> | 
  upgrading app servers in codfw to MEMC_VAL_COMPRESSION_ZLIB enabled HHVM build | 
  [production] | 
            
  | 07:40 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Give more API traffic to db1114 (duration: 01m 17s) | 
  [production] |