| 
      
        2016-11-07
      
      ยง
     | 
  
    
  | 21:08 | 
  <Amir1> | 
  deploying c61b9c1 from ORES into canary nodes (T149730) | 
  [production] | 
            
  | 20:44 | 
  <urandom> | 
  T133395: restbase2001-b.codfw.wmnet: Performing user-defined compaction of la-169239-big-Data.db and la-172629-big-Data.db | 
  [production] | 
            
  | 20:38 | 
  <jynus> | 
  upgrading new labsdbs to mariadb 10.1.19 | 
  [production] | 
            
  | 20:32 | 
  <bblack> | 
  repooling cp4018 (done experimenting) | 
  [production] | 
            
  | 20:21 | 
  <mutante> | 
  projectcom.wikimedia.org created in DNS (T143138) | 
  [production] | 
            
  | 20:14 | 
  <godog> | 
  cmjohnson1 is performing work on LVS in row D, there might be flaps | 
  [production] | 
            
  | 19:58 | 
  <ejegg> | 
  updated payments-wiki from ed98772ead6365d58356294ddd46bc4312204b1d to c1fa73c649986be89a6a038b9e19f6b3ea19e537 | 
  [production] | 
            
  | 19:42 | 
  <ejegg> | 
  updated civicrm from bdc2786ddaf09e9f412f97406e3cffb13fcc96ab to 45a1b9a0e1665b1fc71165e1bc4bcfdca2a9adf7 | 
  [production] | 
            
  | 18:00 | 
  <urandom> | 
  T133395: Convert local_group_*_title__revisions.{data,idx_by_rev_ever} tables to time-window compaction | 
  [production] | 
            
  | 17:56 | 
  <bd808@tin> | 
  Synchronized php-1.29.0-wmf.1/includes/exception/MWExceptionHandler.php: MWExceptionHandler: Do not use 'exception' for custom log data (T150106) (duration: 00m 47s) | 
  [production] | 
            
  | 17:40 | 
  <jynus> | 
  performing schema change on s7 (imagelinks) T139090 | 
  [production] | 
            
  | 16:01 | 
  <mark> | 
  Reactivated cr2-eqiad IX6 BGP group (ipv6 sessions) | 
  [production] | 
            
  | 16:00 | 
  <mark> | 
  Chris moved cr2-eqiad:xe-5/3/3 to xe-3/3/3 | 
  [production] | 
            
  | 15:54 | 
  <mark> | 
  Disabling cr2-eqiad BGP groups IX4/IX6 (all Equinix Ashburn BGP sessions) | 
  [production] | 
            
  | 15:44 | 
  <elukey> | 
  started kafka-mirror-main-eqiad_to_analytics.service on kafka1012 | 
  [production] | 
            
  | 15:39 | 
  <moritzm> | 
  rebooting radium for kernel update | 
  [production] | 
            
  | 15:38 | 
  <mark> | 
  Reenabling OSPF/OSPF3 on cr2-codfw:xe-5/0/1 after eqiad side port move to xe-3/2/3 | 
  [production] | 
            
  | 15:31 | 
  <mark> | 
  Disabling OSPF/OSPF3 on cr2-codfw:xe-5/0/1 for eqiad side port move | 
  [production] | 
            
  | 15:26 | 
  <elukey> | 
  rebooting kafka1013 for kernel upgrades | 
  [production] | 
            
  | 15:22 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2042 - T149553 (duration: 00m 49s) | 
  [production] | 
            
  | 15:19 | 
  <hashar> | 
  Restarting Jenkins (deadlock in beta cluster Jenkins jobs) | 
  [production] | 
            
  | 15:14 | 
  <mark> | 
  Reactivate cr2-eqiad BGP peering with pfw1-eqiad | 
  [production] | 
            
  | 15:13 | 
  <mark> | 
  Chris moved cr2-eqiad:xe-5/0/3 to xe-3/3/2 | 
  [production] | 
            
  | 15:10 | 
  <hashar> | 
  European SWAT completed | 
  [production] | 
            
  | 15:09 | 
  <hashar@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: shortUrl for bdwikimedia and tcywiki T146014 and T150166 (duration: 01m 51s) | 
  [production] | 
            
  | 15:08 | 
  <mark> | 
  Deactivate cr2-eqiad BGP peering with pfw1-eqiad | 
  [production] | 
            
  | 15:07 | 
  <marostegui> | 
  Enabling gtid_domain_id db1020 (m2 master) - T149418 | 
  [production] | 
            
  | 15:07 | 
  <mark> | 
  Reactivate cr1-eqiad BGP peering with pfw1-eqiad | 
  [production] | 
            
  | 15:05 | 
  <mark> | 
  Chris moved cr1-eqiad:xe-5/0/3 to xe-3/3/2 | 
  [production] | 
            
  | 15:03 | 
  <hashar> | 
  T146014 mwscript extensions/ShortUrl/populateShortUrlTable.php --wiki=bdwikimedia (714 titles done) | 
  [production] | 
            
  | 15:02 | 
  <hashar> | 
  T150166 mwscript extensions/ShortUrl/populateShortUrlTable.php --wiki=tcywiki  (1569 titles done) | 
  [production] | 
            
  | 15:02 | 
  <moritzm> | 
  rebooting mw1261-mw1265 (canary app servers) for kernel update | 
  [production] | 
            
  | 15:01 | 
  <hashar> | 
  T146014 mwscript sql.php --wiki=bdwikimedia /srv/mediawiki/php-1.29.0-wmf.1/extensions/ShortUrl/schemas/shorturls.sql | 
  [production] | 
            
  | 15:01 | 
  <hashar> | 
  T150166 mwscript sql.php --wiki=tcywiki /srv/mediawiki/php-1.29.0-wmf.1/extensions/ShortUrl/schemas/shorturls.sql | 
  [production] | 
            
  | 15:00 | 
  <mark> | 
  Deactivate cr1-eqiad BGP peering with pfw1-eqiad | 
  [production] | 
            
  | 14:49 | 
  <hashar> | 
  terbium: scap pull to add shortUrl tables to bdwikimedia and tcywiki | 
  [production] | 
            
  | 14:42 | 
  <hashar> | 
  fawiki: renaming user group 'autopatrol' to 'autopatrolled' for T139246 and T144699 with:  mwscript migrateUserGroup.php --wiki=fawiki 'autopatrol' 'autopatrolled' | 
  [production] | 
            
  | 14:42 | 
  <hashar> | 
  fawiki Done! 417 users in group 'autopatrol' are now in 'autopatrolled' instead. | 
  [production] | 
            
  | 14:40 | 
  <hashar@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Rename 'autopatrol' to 'autopatrolled' on fawiki - T144699 T139246 (duration: 00m 47s) | 
  [production] | 
            
  | 14:33 | 
  <gehel> | 
  reboot maps-test* for kernel upgrade | 
  [production] | 
            
  | 14:30 | 
  <hashar@tin> | 
  Synchronized wmf-config: (no message) (duration: 00m 53s) | 
  [production] | 
            
  | 14:10 | 
  <hashar@tin> | 
  Synchronized php-1.29.0-wmf.1/extensions/Kartographer/extension.json: Fix monobook <maplink> (missing debounce dep) T145521 (duration: 00m 47s) | 
  [production] | 
            
  | 13:56 | 
  <gehel> | 
  reboot wdqs1* for kernel upgrade | 
  [production] | 
            
  | 13:52 | 
  <bblack> | 
  depooling cp4018 nginx+varnish-fe services for debugging | 
  [production] | 
            
  | 13:36 | 
  <gehel> | 
  reboot wdqs2* for kernel upgrade | 
  [production] | 
            
  | 13:34 | 
  <hashar> | 
  Flushed nodepool instances. It is bringing up fresh one now. | 
  [production] | 
            
  | 13:26 | 
  <moritzm> | 
  rebooting labnodepool1001 for kernel update | 
  [production] | 
            
  | 13:19 | 
  <hashar> | 
  shutting down Nodepool (labnodepool1001.eqiad.wmnet reboot) | 
  [production] | 
            
  | 13:06 | 
  <moritzm> | 
  rebooting scandium for kernel update | 
  [production] | 
            
  | 12:09 | 
  <jynus> | 
  performing schema change on s6 (imagelinks) T139090 | 
  [production] |