| 2016-11-07
      
      ยง | 
    
  | 20:14 | <godog> | cmjohnson1 is performing work on LVS in row D, there might be flaps | [production] | 
            
  | 19:58 | <ejegg> | updated payments-wiki from ed98772ead6365d58356294ddd46bc4312204b1d to c1fa73c649986be89a6a038b9e19f6b3ea19e537 | [production] | 
            
  | 19:42 | <ejegg> | updated civicrm from bdc2786ddaf09e9f412f97406e3cffb13fcc96ab to 45a1b9a0e1665b1fc71165e1bc4bcfdca2a9adf7 | [production] | 
            
  | 18:00 | <urandom> | T133395: Convert local_group_*_title__revisions.{data,idx_by_rev_ever} tables to time-window compaction | [production] | 
            
  | 17:56 | <bd808@tin> | Synchronized php-1.29.0-wmf.1/includes/exception/MWExceptionHandler.php: MWExceptionHandler: Do not use 'exception' for custom log data (T150106) (duration: 00m 47s) | [production] | 
            
  | 17:40 | <jynus> | performing schema change on s7 (imagelinks) T139090 | [production] | 
            
  | 16:01 | <mark> | Reactivated cr2-eqiad IX6 BGP group (ipv6 sessions) | [production] | 
            
  | 16:00 | <mark> | Chris moved cr2-eqiad:xe-5/3/3 to xe-3/3/3 | [production] | 
            
  | 15:54 | <mark> | Disabling cr2-eqiad BGP groups IX4/IX6 (all Equinix Ashburn BGP sessions) | [production] | 
            
  | 15:44 | <elukey> | started kafka-mirror-main-eqiad_to_analytics.service on kafka1012 | [production] | 
            
  | 15:39 | <moritzm> | rebooting radium for kernel update | [production] | 
            
  | 15:38 | <mark> | Reenabling OSPF/OSPF3 on cr2-codfw:xe-5/0/1 after eqiad side port move to xe-3/2/3 | [production] | 
            
  | 15:31 | <mark> | Disabling OSPF/OSPF3 on cr2-codfw:xe-5/0/1 for eqiad side port move | [production] | 
            
  | 15:26 | <elukey> | rebooting kafka1013 for kernel upgrades | [production] | 
            
  | 15:22 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Repool db2042 - T149553 (duration: 00m 49s) | [production] | 
            
  | 15:19 | <hashar> | Restarting Jenkins (deadlock in beta cluster Jenkins jobs) | [production] | 
            
  | 15:14 | <mark> | Reactivate cr2-eqiad BGP peering with pfw1-eqiad | [production] | 
            
  | 15:13 | <mark> | Chris moved cr2-eqiad:xe-5/0/3 to xe-3/3/2 | [production] | 
            
  | 15:10 | <hashar> | European SWAT completed | [production] | 
            
  | 15:09 | <hashar@tin> | Synchronized wmf-config/InitialiseSettings.php: shortUrl for bdwikimedia and tcywiki T146014 and T150166 (duration: 01m 51s) | [production] | 
            
  | 15:08 | <mark> | Deactivate cr2-eqiad BGP peering with pfw1-eqiad | [production] | 
            
  | 15:07 | <marostegui> | Enabling gtid_domain_id db1020 (m2 master) - T149418 | [production] | 
            
  | 15:07 | <mark> | Reactivate cr1-eqiad BGP peering with pfw1-eqiad | [production] | 
            
  | 15:05 | <mark> | Chris moved cr1-eqiad:xe-5/0/3 to xe-3/3/2 | [production] | 
            
  | 15:03 | <hashar> | T146014 mwscript extensions/ShortUrl/populateShortUrlTable.php --wiki=bdwikimedia (714 titles done) | [production] | 
            
  | 15:02 | <hashar> | T150166 mwscript extensions/ShortUrl/populateShortUrlTable.php --wiki=tcywiki  (1569 titles done) | [production] | 
            
  | 15:02 | <moritzm> | rebooting mw1261-mw1265 (canary app servers) for kernel update | [production] | 
            
  | 15:01 | <hashar> | T146014 mwscript sql.php --wiki=bdwikimedia /srv/mediawiki/php-1.29.0-wmf.1/extensions/ShortUrl/schemas/shorturls.sql | [production] | 
            
  | 15:01 | <hashar> | T150166 mwscript sql.php --wiki=tcywiki /srv/mediawiki/php-1.29.0-wmf.1/extensions/ShortUrl/schemas/shorturls.sql | [production] | 
            
  | 15:00 | <mark> | Deactivate cr1-eqiad BGP peering with pfw1-eqiad | [production] | 
            
  | 14:49 | <hashar> | terbium: scap pull to add shortUrl tables to bdwikimedia and tcywiki | [production] | 
            
  | 14:42 | <hashar> | fawiki: renaming user group 'autopatrol' to 'autopatrolled' for T139246 and T144699 with:  mwscript migrateUserGroup.php --wiki=fawiki 'autopatrol' 'autopatrolled' | [production] | 
            
  | 14:42 | <hashar> | fawiki Done! 417 users in group 'autopatrol' are now in 'autopatrolled' instead. | [production] | 
            
  | 14:40 | <hashar@tin> | Synchronized wmf-config/InitialiseSettings.php: Rename 'autopatrol' to 'autopatrolled' on fawiki - T144699 T139246 (duration: 00m 47s) | [production] | 
            
  | 14:33 | <gehel> | reboot maps-test* for kernel upgrade | [production] | 
            
  | 14:30 | <hashar@tin> | Synchronized wmf-config: (no message) (duration: 00m 53s) | [production] | 
            
  | 14:10 | <hashar@tin> | Synchronized php-1.29.0-wmf.1/extensions/Kartographer/extension.json: Fix monobook <maplink> (missing debounce dep) T145521 (duration: 00m 47s) | [production] | 
            
  | 13:56 | <gehel> | reboot wdqs1* for kernel upgrade | [production] | 
            
  | 13:52 | <bblack> | depooling cp4018 nginx+varnish-fe services for debugging | [production] | 
            
  | 13:36 | <gehel> | reboot wdqs2* for kernel upgrade | [production] | 
            
  | 13:34 | <hashar> | Flushed nodepool instances. It is bringing up fresh one now. | [production] | 
            
  | 13:26 | <moritzm> | rebooting labnodepool1001 for kernel update | [production] | 
            
  | 13:19 | <hashar> | shutting down Nodepool (labnodepool1001.eqiad.wmnet reboot) | [production] | 
            
  | 13:06 | <moritzm> | rebooting scandium for kernel update | [production] | 
            
  | 12:09 | <jynus> | performing schema change on s6 (imagelinks) T139090 | [production] | 
            
  | 12:00 | <moritzm> | rebooting wtp1001 for kernel update | [production] | 
            
  | 11:40 | <ema> | cp3043: repool varnish-be and varnish-be-rand (T149881) | [production] | 
            
  | 11:33 | <moritzm> | rebooting cassandra test hosts (cerium, praseodymium, xenon) for kernel update | [production] | 
            
  | 10:49 | <moritzm> | rebooting mw1017/mw1099 for kernel update | [production] | 
            
  | 10:26 | <moritzm> | rebooting cp1008 for kernel update | [production] |