| 
      
        2015-09-15
      
      §
     | 
  
    
  | 12:53 | 
  <moritzm> | 
  depooled mw1114 (for enabling ferm) | 
  [production] | 
            
  | 11:42 | 
  <moritzm> | 
  repool mw1018 (with ferm enabled) | 
  [production] | 
            
  | 11:23 | 
  <moritzm> | 
  depooled mw1018 (for enabling ferm) | 
  [production] | 
            
  | 08:53 | 
  <_joe_> | 
  created a 100 G partition on a LV on copper, for /tmp | 
  [production] | 
            
  | 08:24 | 
  <godog> | 
  bounce ms-be2006, xfs | 
  [production] | 
            
  | 08:22 | 
  <moritzm> | 
  bumped default size of iptables connection tracking table to 256k | 
  [production] | 
            
  | 06:10 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Tue Sep 15 06:10:52 UTC 2015 (duration 10m 51s) | 
  [production] | 
            
  | 02:46 | 
  <l10nupdate@tin> | 
  LocalisationUpdate completed (1.26wmf22) at 2015-09-15 02:46:50+00:00 | 
  [production] | 
            
  | 02:40 | 
  <l10nupdate@tin> | 
  Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 10m 53s) | 
  [production] | 
            
  | 02:18 | 
  <legoktm@tin> | 
  Synchronized php-1.26wmf22/extensions/MobileFrontend: Revert Echo to 1.26wmf21 state (duration: 00m 11s) | 
  [production] | 
            
  | 02:18 | 
  <legoktm@tin> | 
  Synchronized php-1.26wmf22/extensions/Echo: Revert Echo to 1.26wmf21 state (duration: 00m 12s) | 
  [production] | 
            
  | 01:30 | 
  <krinkle@tin> | 
  Synchronized php-1.26wmf22/resources/src: T112287 (duration: 00m 11s) | 
  [production] | 
            
  | 00:49 | 
  <bblack> | 
  reinstalling lvs300[34] to jessie | 
  [production] | 
            
  | 00:43 | 
  <ebernhardson@tin> | 
  Synchronized wmf-config/CirrusSearch-labs.php: noop sync of labs config change (duration: 00m 11s) | 
  [production] | 
            
  | 00:03 | 
  <tstarling@tin> | 
  Synchronized php-1.26wmf22/extensions/ParsoidBatchAPI: for I56d28e9a for RT testing, not live yet (duration: 00m 13s) | 
  [production] | 
            
  
    | 
      
        2015-09-14
      
      §
     | 
  
    
  | 23:27 | 
  <ebernhardson@tin> | 
  Synchronized php-1.26wmf22/extensions/WikimediaEvents/: Change bucket selection methods in CompletionSuggestions AB test (duration: 00m 12s) | 
  [production] | 
            
  | 23:23 | 
  <ebernhardson@tin> | 
  Synchronized php-1.26wmf22/extensions/UploadWizard/: Swat out badtoken fix to UploadWizard in 1.26wmf22 (duration: 00m 12s) | 
  [production] | 
            
  | 22:37 | 
  <yurik> | 
  deployed tilerator | 
  [production] | 
            
  | 21:15 | 
  <ori@tin> | 
  Synchronized php-1.26wmf22/extensions/TitleBlacklist: Ie44fcb500: Avoid checking blacklists in isBlacklisted() for existing titles (duration: 00m 12s) | 
  [production] | 
            
  | 21:15 | 
  <mutante> | 
  labnodepool1001 - re-enable puppet and nodepool | 
  [production] | 
            
  | 20:59 | 
  <legoktm@tin> | 
  Synchronized php-1.26wmf22/extensions/Echo/: Hack around OOUI's icon pack being too large by creating our own (duration: 00m 12s) | 
  [production] | 
            
  | 20:53 | 
  <cscott> | 
  updated OCG to version 5811056e28f2bc6408b6da96095352ab381bb11f | 
  [production] | 
            
  | 20:21 | 
  <andrewbogott> | 
  graceful’d apache2 on labcontrol1001 | 
  [production] | 
            
  | 20:15 | 
  <subbu> | 
  deployed parsoid sha 3d5f4359 | 
  [production] | 
            
  | 19:25 | 
  <legoktm@tin> | 
  Synchronized php-1.26wmf22/extensions/Echo/: Only load nojs Special:Notifications styles on the special page (duration: 00m 12s) | 
  [production] | 
            
  | 18:05 | 
  <urandom> | 
  rebuilding restbase-test2001.codfw (nodetool rebuild -- eqiad) | 
  [production] | 
            
  | 16:12 | 
  <catrope@tin> | 
  Synchronized php-1.26wmf22/extensions/Echo/: For real this time (duration: 00m 11s) | 
  [production] | 
            
  | 16:06 | 
  <ottomata> | 
  stopping hdfs journalnode on analytics1011 to copy journal edits to new journalnodes on analytics1035 and analytics1052 | 
  [production] | 
            
  | 15:46 | 
  <godog> | 
  switch to openjdk-8 and bounce cassandra on restbase-test200* | 
  [production] | 
            
  | 15:39 | 
  <bblack> | 
  reinstalling lvs4003, lvs4004 (jessie upgrade: T96375) (typo earlier) | 
  [production] | 
            
  | 15:39 | 
  <bblack> | 
  reinstalling lvs4003, lvs4003 (jessie upgrade: T96375) | 
  [production] | 
            
  | 15:34 | 
  <catrope@tin> | 
  Synchronized php-1.26wmf22/extensions/Echo/: SWAT (duration: 00m 13s) | 
  [production] | 
            
  | 15:05 | 
  <krenair@tin> | 
  Synchronized .gitignore: https://gerrit.wikimedia.org/r/#/c/237529/ (duration: 00m 13s) | 
  [production] | 
            
  | 15:05 | 
  <krenair@tin> | 
  Synchronized docroot/noc: https://gerrit.wikimedia.org/r/#/c/237529/ (duration: 00m 12s) | 
  [production] | 
            
  | 15:04 | 
  <krenair@tin> | 
  Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/237529/ (duration: 00m 11s) | 
  [production] | 
            
  | 15:01 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/234980/ (duration: 00m 12s) | 
  [production] | 
            
  | 13:38 | 
  <godog> | 
  stop puppet on restbase-test2001 and turn up cassandra | 
  [production] | 
            
  | 12:56 | 
  <bblack> | 
  rebooting lvs2006 to test eth hw params stuff... | 
  [production] | 
            
  | 12:55 | 
  <krenair@tin> | 
  Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/238125/ (duration: 00m 13s) | 
  [production] | 
            
  | 12:50 | 
  <urandom> | 
  starting Cassandra repair on restbase1003 (nodetool repair -pr) | 
  [production] | 
            
  | 12:32 | 
  <godog> | 
  enable dc encryption on cassandra test cluster and rolling restart | 
  [production] | 
            
  | 11:33 | 
  <mobrovac> | 
  citoid deploying d569951 | 
  [production] | 
            
  | 10:35 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool es1002, es1005, es1008 (duration: 00m 12s) | 
  [production] | 
            
  | 10:04 | 
  <jynus> | 
  db1029 (x1-master) temporarily saturated by connections- flow was unresponsive for 10 minutes; migration partially aborted | 
  [production] | 
            
  | 09:08 | 
  <jynus> | 
  applying schema change to flowdb | 
  [production] | 
            
  | 08:52 | 
  <godog> | 
  rename cassandra test cluster and restart | 
  [production] | 
            
  | 08:44 | 
  <godog> | 
  silence mendelevium for today, status unclear T111532 | 
  [production] | 
            
  | 08:30 | 
  <jynus> | 
  endinf profiling and executing pt-query-digest on db1043 [ETA:4h] | 
  [production] | 
            
  | 07:52 | 
  <godog> | 
  reboot ms-be1010 to pick up disk ordering change | 
  [production] | 
            
  | 04:47 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Mon Sep 14 04:47:58 UTC 2015 (duration 47m 57s) | 
  [production] |