| 
      
        2018-05-03
      
      ยง
     | 
  
    
  | 18:48 | 
  <dzahn@neodymium> | 
  conftool action : set/pooled=inactive; selector: name=mw2229.codfw.wmnet | 
  [production] | 
            
  | 18:48 | 
  <dzahn@neodymium> | 
  conftool action : set/pooled=inactive; selector: name=mw2231.codfw.wmnet | 
  [production] | 
            
  | 18:44 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:430360|Enable ULS webfonts by default at Bengali Wikisource]] T193367 (duration: 01m 18s) | 
  [production] | 
            
  | 18:09 | 
  <mutante> | 
  mw2229,mw2231,mw2240 - wmf-auto-reimage with --new switch because their puppet cert wasn't found on puppetmaster, treated as new hosts that didnt exist before | 
  [production] | 
            
  | 17:59 | 
  <mutante> | 
  mw2254,mw2183,mw2184 - wmf-auto-reimage with stretch and raid/lvm | 
  [production] | 
            
  | 17:36 | 
  <ejegg> | 
  updated CiviCRM from 401344fb30 to 0fdef242a3 | 
  [production] | 
            
  | 16:34 | 
  <Jeff_Green> | 
  authdns-update to add frbast.wikimedia.org service alias | 
  [production] | 
            
  | 15:04 | 
  <imarlier@tin> | 
  Finished deploy [performance/coal@762d160]: verify coal is deploying properly after shutdown on graphite hosts (duration: 00m 14s) | 
  [production] | 
            
  | 15:04 | 
  <imarlier@tin> | 
  Started deploy [performance/coal@762d160]: verify coal is deploying properly after shutdown on graphite hosts | 
  [production] | 
            
  | 14:47 | 
  <marostegui> | 
  Manually set offline disk #1 on db1063 so it can be replaced | 
  [production] | 
            
  | 14:44 | 
  <imarlier@tin> | 
  Started deploy [performance/coal@bd7568a]: verify coal is deploying properly after shutdown on graphite hosts | 
  [production] | 
            
  | 14:13 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1056 (duration: 01m 17s) | 
  [production] | 
            
  | 14:00 | 
  <ema> | 
  cp3030 (text): upgrade varnish to 5.1.3-1wm8 T192368 | 
  [production] | 
            
  | 13:21 | 
  <addshore> | 
  swat done | 
  [production] | 
            
  | 13:19 | 
  <addshore@tin> | 
  Synchronized wmf-config/: [[gerrit:395488|Switch to extension.json for Wikidata.org]] (duration: 01m 19s) | 
  [production] | 
            
  | 13:13 | 
  <addshore@tin> | 
  Synchronized wmf-config/: [[gerrit:395486|Switch to extension.json for PropertySuggester]] (duration: 01m 35s) | 
  [production] | 
            
  | 13:05 | 
  <godog> | 
  stop and mask coal service on graphite hosts - T186774 | 
  [production] | 
            
  | 13:04 | 
  <gehel> | 
  rolling restart of wdqs for jvm upgrade | 
  [production] | 
            
  | 13:02 | 
  <marostegui> | 
  Deploy schema change on s4 codfw master db2051 with replication (this will generate lag on codfw) - T191519 T188299 T190148 | 
  [production] | 
            
  | 12:35 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Revert: Clarify that db1060 is running an alter table (duration: 01m 17s) | 
  [production] | 
            
  | 12:26 | 
  <moritzm> | 
  installing openjdk-8 security updates on stretch-based Hadoop workers | 
  [production] | 
            
  | 11:57 | 
  <moritzm> | 
  reimaging mw1301, mw1302 (job runners) to stretch | 
  [production] | 
            
  | 10:51 | 
  <moritzm> | 
  reimaging mw1319, mw1325, mw1326 (app servers) to stretch | 
  [production] | 
            
  | 09:29 | 
  <moritzm> | 
  reimaging mw1227, mw1231, mw1232 (API servers) to stretch | 
  [production] | 
            
  | 09:09 | 
  <gehel> | 
  rolling restart of elasticsearch completed - T191543 / T191236 | 
  [production] | 
            
  | 09:06 | 
  <moritzm> | 
  reimaging mw1300 (job runner) to stretch | 
  [production] | 
            
  | 08:33 | 
  <moritzm> | 
  installing Java security updates on wdqs* | 
  [production] | 
            
  | 08:30 | 
  <addshore@tin> | 
  Synchronized php-1.32.0-wmf.2/extensions/WikimediaEvents/WikimediaEventsHooks.php: T191500 [[gerrit:430380|Update campaign prefix for onBeforeInitializeWMDECampaign hook]] (duration: 01m 16s) | 
  [production] | 
            
  | 08:29 | 
  <addshore@tin> | 
  Synchronized php-1.32.0-wmf.1/extensions/WikimediaEvents/WikimediaEventsHooks.php: T191500 [[gerrit:430379|Update campaign prefix for onBeforeInitializeWMDECampaign hook]] (duration: 01m 17s) | 
  [production] | 
            
  | 08:27 | 
  <mobrovac@tin> | 
  Finished deploy [changeprop/deploy@7e86531]: Bug fix: Resubscribe to the proper list of topics on metadata change (duration: 01m 12s) | 
  [production] | 
            
  | 08:26 | 
  <mobrovac@tin> | 
  Started deploy [changeprop/deploy@7e86531]: Bug fix: Resubscribe to the proper list of topics on metadata change | 
  [production] | 
            
  | 08:18 | 
  <mobrovac@tin> | 
  Finished deploy [cpjobqueue/deploy@5c1dcb9]: Bug fix: Resubscribe to the proper list of topics on metadata change (duration: 00m 54s) | 
  [production] | 
            
  | 08:17 | 
  <mobrovac@tin> | 
  Started deploy [cpjobqueue/deploy@5c1dcb9]: Bug fix: Resubscribe to the proper list of topics on metadata change | 
  [production] | 
            
  | 08:13 | 
  <ema> | 
  cp-misc: upgrade varnish to 5.1.3-1wm8 T192368 | 
  [production] | 
            
  | 08:08 | 
  <godog> | 
  eqiad-prod: more weight to ms-be104[0-3] - T190081 | 
  [production] | 
            
  | 07:54 | 
  <moritzm> | 
  reimaging mw1284, mw1289, mw1290 (API servers) to stretch | 
  [production] | 
            
  | 07:39 | 
  <moritzm> | 
  reimaging mw1256, mw1257, mw1258 (app servers) to stretch | 
  [production] | 
            
  | 07:26 | 
  <marostegui> | 
  Drop table flaggedrevs from eswikibooks - T193676 | 
  [production] | 
            
  | 07:11 | 
  <moritzm> | 
  reimaging mwdebug1002 to stretch | 
  [production] | 
            
  | 05:59 | 
  <marostegui> | 
  Drop mostly empty flagged* tables from metawiki (s7) - T193390 | 
  [production] | 
            
  | 05:59 | 
  <marostegui> | 
  Drop mostly empty flagged* tables from metawiki (s7) - T193678 | 
  [production] | 
            
  | 05:57 | 
  <elukey> | 
  reimage analytics10[39,40] to Debian Stretch | 
  [production] | 
            
  | 05:31 | 
  <marostegui> | 
  Drop empty flagged* tables from eswiki (s7) - T193678 | 
  [production] | 
            
  | 05:28 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Clarify that db1060 is running an alter table (duration: 01m 15s) | 
  [production] | 
            
  | 05:22 | 
  <marostegui> | 
  Deploy schema change on db1060 with replication (this will generate lag on labs - s2) - T191519 T188299 T190148 | 
  [production] | 
            
  | 03:32 | 
  <dzahn@neodymium> | 
  conftool action : set/pooled=yes; selector: name=mw2182.codfw.wmnet | 
  [production] | 
            
  | 03:28 | 
  <dzahn@neodymium> | 
  conftool action : set/pooled=yes; selector: name=mw2181.codfw.wmnet | 
  [production] | 
            
  | 03:26 | 
  <dzahn@neodymium> | 
  conftool action : set/pooled=yes; selector: name=mw2180.codfw.wmnet | 
  [production] | 
            
  | 02:43 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.32.0-wmf.1) (duration: 07m 20s) | 
  [production] | 
            
  | 00:33 | 
  <krinkle@tin> | 
  Synchronized php-1.32.0-wmf.1/extensions/NavigationTiming/modules/: Ie77e77de3b8 (duration: 01m 18s) | 
  [production] |