| 
      
        2015-08-21
      
      §
     | 
  
    
  | 23:34 | 
  <urandom> | 
  restarting Cassandra on restbase1001 to restore baseline settings | 
  [production] | 
            
  | 23:11 | 
  <yurik> | 
  synced kartotherian | 
  [production] | 
            
  | 22:35 | 
  <mutante> | 
  deleting held messages on mailman that are older than 1 year | 
  [production] | 
            
  | 21:44 | 
  <mutante> | 
  had to reset list creator password for mailman - ask me if you think you should have it and don't (this is not the master pass) | 
  [production] | 
            
  | 20:37 | 
  <ori@tin> | 
  Synchronized php-1.26wmf19/includes: I1eb8dfc: Revert Count API and hook calls, with 1:1000 sampling (duration: 01m 09s) | 
  [production] | 
            
  | 16:06 | 
  <jynus> | 
  checksumming dewiki database, higher write rate/dbstore lag expected temporarily | 
  [production] | 
            
  | 15:10 | 
  <ottomata> | 
  rebooting kafka broker analytics1021 to hopefully reload /dev/sdg with new disk, also will turn on hyperthreading | 
  [production] | 
            
  | 14:13 | 
  <ottomata> | 
  rebooting analytics1056 after upgrading kernel to linux-image-3.13.0-61-generic | 
  [production] | 
            
  | 13:57 | 
  <urandom> | 
  restarting restbase1001 to apply temporary GC setting | 
  [production] | 
            
  | 13:33 | 
  <ottomata> | 
  stopping kafka broker on analytics1021 due to bad disk.   | 
  [production] | 
            
  | 13:30 | 
  <bblack> | 
  wiped ganglia apache access log on uranium, to free up half of the (full) rootfs | 
  [production] | 
            
  | 10:07 | 
  <godog> | 
  enable puppet on ms-fe1/ms-be1 | 
  [production] | 
            
  | 09:49 | 
  <godog> | 
  disable puppet on ms-fe1/ms-be1 before merging https://gerrit.wikimedia.org/r/#/c/231240/ | 
  [production] | 
            
  | 07:06 | 
  <_joe_> | 
  restarting gitblit, because it will be decommissioned "soon"... | 
  [production] | 
            
  | 02:34 | 
  <l10nupdate@tin> | 
  Synchronized php-1.26wmf19/cache/l10n: l10nupdate for 1.26wmf19 (duration: 11m 19s) | 
  [production] | 
            
  
    | 
      
        2015-08-20
      
      §
     | 
  
    
  | 23:40 | 
  <ebernhardson@tin> | 
  Synchronized php-1.26wmf19/extensions/CirrusSearch/: Fix some cirrussearch logspam (duration: 00m 13s) | 
  [production] | 
            
  | 23:30 | 
  <ebernhardson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 12s) | 
  [production] | 
            
  | 23:29 | 
  <ebernhardson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 13s) | 
  [production] | 
            
  | 23:23 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/232854/ (duration: 00m 13s) | 
  [production] | 
            
  | 23:22 | 
  <krenair@tin> | 
  Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/232671/ (duration: 00m 12s) | 
  [production] | 
            
  | 23:15 | 
  <krenair@tin> | 
  Synchronized php-1.26wmf19/extensions/LiquidThreads/classes/Hooks.php: https://gerrit.wikimedia.org/r/#/c/232783/ (duration: 00m 12s) | 
  [production] | 
            
  | 23:13 | 
  <krinkle@tin> | 
  Synchronized php-1.26wmf19/includes/resourceloader/ResourceLoaderFileModule.php: T102578 (duration: 00m 13s) | 
  [production] | 
            
  | 23:08 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/232782/ (duration: 00m 12s) | 
  [production] | 
            
  | 22:47 | 
  <ori@tin> | 
  Synchronized php-1.26wmf19/includes/libs/CSSMin.php: Icc1c23a2: CSSMin: remove dot segments in relative local URLs (duration: 00m 12s) | 
  [production] | 
            
  | 21:36 | 
  <cscott> | 
  updated Parsoid to version db6e6404f67a9f971b4fbefe9de239735426c738 | 
  [production] | 
            
  | 21:25 | 
  <matt_flaschen> | 
  Ran FlowUpdateRevContentModelFromOccupyPages.php on all wikis | 
  [production] | 
            
  | 20:41 | 
  <twentyafterfour> | 
  scap failed to sync to mw2180.codwf.wmnet | 
  [production] | 
            
  | 20:41 | 
  <twentyafterfour@tin> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedia wikis to 1.26wmf19 | 
  [production] | 
            
  | 20:38 | 
  <twentyafterfour@tin> | 
  Synchronized php-1.26wmf19: Silence the undefined index error in CirrusSearch (duration: 06m 24s) | 
  [production] | 
            
  | 19:40 | 
  <chasemp> | 
  moving enwiki_content_1432182861 elastic shard from 1022 to 1004 due to space (1022 is at 91%) | 
  [production] | 
            
  | 18:57 | 
  <mutante> | 
  no log bot  | 
  [production] | 
            
  | 18:56 | 
  <mutante> | 
  labvirt1007 "only" 29G space left - but since we have 2.2T there that means 99% full | 
  [production] | 
            
  | 17:39 | 
  <ottomata> | 
  stopping kafka on analytics1018 and bringing it down for reinstall as kafka1018 with Jessie | 
  [production] | 
            
  | 16:38 | 
  <YuviPanda> | 
  puppet swat done | 
  [production] | 
            
  | 15:44 | 
  <akosiaris> | 
  uploaded to apt.wikimedia.org jessie-wikimedia: etherpad-lite_1.5.7-1 | 
  [production] | 
            
  | 15:43 | 
  <krenair@tin> | 
  Synchronized php-1.26wmf18/extensions/ContentTranslation/modules/tools/ext.cx.tools.reference.js: https://gerrit.wikimedia.org/r/#/c/232729/ (duration: 00m 12s) | 
  [production] | 
            
  | 15:42 | 
  <krenair@tin> | 
  Synchronized php-1.26wmf19/extensions/ContentTranslation/modules/tools/ext.cx.tools.reference.js: https://gerrit.wikimedia.org/r/#/c/232730/ (duration: 00m 13s) | 
  [production] | 
            
  | 15:39 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/206480/ (duration: 00m 13s) | 
  [production] | 
            
  | 15:38 | 
  <krenair@tin> | 
  Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/206480/ (duration: 00m 13s) | 
  [production] | 
            
  | 15:32 | 
  <krenair@tin> | 
  Synchronized php-1.26wmf18/extensions/ContentTranslation/api/ApiContentTranslationPublish.php: https://gerrit.wikimedia.org/r/#/c/232687/ (duration: 00m 13s) | 
  [production] | 
            
  | 15:31 | 
  <krenair@tin> | 
  Synchronized php-1.26wmf19/extensions/ContentTranslation/api/ApiContentTranslationPublish.php: https://gerrit.wikimedia.org/r/#/c/232688/ (duration: 00m 11s) | 
  [production] | 
            
  | 15:27 | 
  <greg-g> | 
  on mw2187: rsync: failed to set times on "/srv/mediawiki/wmf-config": Read-only file system (30) | 
  [production] | 
            
  | 15:25 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/231464/ (duration: 00m 13s) | 
  [production] | 
            
  | 15:08 | 
  <urandom> | 
  restarting restbase1001 to apply temporary heap size of 12G | 
  [production] | 
            
  | 15:02 | 
  <jynus> | 
  performing online schema change on wikidata | 
  [production] | 
            
  | 15:00 | 
  <andrewbogott> | 
  rebooting labvirt1008 | 
  [production] | 
            
  | 12:48 | 
  <jynus> | 
  restarted nutcracker on mw1142 | 
  [production] | 
            
  | 12:08 | 
  <godog> | 
  reenable puppet on ms-fe1/ms-be1 | 
  [production] |