| 
      
        2015-09-01
      
      §
     | 
  
    
  | 22:18 | 
  <MaxSem> | 
  Maps: creating and populating admin table | 
  [production] | 
            
  | 21:20 | 
  <krenair@tin> | 
  Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/235177/ (duration: 00m 12s) | 
  [production] | 
            
  | 20:54 | 
  <ori> | 
  restarted nutcracker on mw1142 | 
  [production] | 
            
  | 20:33 | 
  <twentyafterfour@tin> | 
  Finished scap: sync 1.26wmf21 (duration: 30m 37s) | 
  [production] | 
            
  | 20:03 | 
  <dcausse> | 
  unfreezing elasticsearch indices | 
  [production] | 
            
  | 20:03 | 
  <twentyafterfour@tin> | 
  Started scap: sync 1.26wmf21 | 
  [production] | 
            
  | 19:52 | 
  <YuviPanda> | 
  removed tools20150901132642 from labstore vg on labstore1002 | 
  [production] | 
            
  | 19:36 | 
  <ori@tin> | 
  Synchronized php-1.26wmf20/includes/skins/SkinTemplate.php: cc643a0934: Deprecate unconditional loading of mediawiki.ui.button on all pages (duration: 00m 13s) | 
  [production] | 
            
  | 17:31 | 
  <urandom> | 
  bouncing Cassandra on restbase1001 to apply temporary GC setting | 
  [production] | 
            
  | 17:28 | 
  <dcausse> | 
  freezing elasticsearch indices before applying ferm fules on master | 
  [production] | 
            
  | 17:23 | 
  <aude@tin> | 
  Synchronized php-1.26wmf20/extensions/Wikidata: Fix for change dispatcher (duration: 00m 20s) | 
  [production] | 
            
  | 16:45 | 
  <jynus> | 
  performing schema change on testwiki and metawiki | 
  [production] | 
            
  | 16:12 | 
  <robh> | 
  policy.wikimedia.org dns change happening now | 
  [production] | 
            
  | 15:59 | 
  <chasemp> | 
  ferm for elastic1003/2/1(master) | 
  [production] | 
            
  | 15:57 | 
  <krenair@tin> | 
  Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/235168/ (duration: 00m 13s) | 
  [production] | 
            
  | 15:51 | 
  <YuviPanda> | 
  stopped replicate-tools on labstore1002, and cleaned out lockdir | 
  [production] | 
            
  | 15:47 | 
  <reedy@tin> | 
  Synchronized php-1.26wmf20/extensions/SecurePoll/: Stop cronspam (duration: 00m 13s) | 
  [production] | 
            
  | 15:47 | 
  <mark> | 
  labstore1002: echo 10000 > /sys/block/md123/md/sync_speed_min | 
  [production] | 
            
  | 15:44 | 
  <mark> | 
  labstore1002: update-initramfs -k all -u | 
  [production] | 
            
  | 15:38 | 
  <mark> | 
  labstore1002: mdadm /dev/md/slice51 --add /dev/sd{bh,bg,bf,be,bd,bc} | 
  [production] | 
            
  | 15:36 | 
  <moritzm> | 
  disabled ferm in analytic1028, needs some more work on possibly dynamic mapreduce ports | 
  [production] | 
            
  | 15:16 | 
  <mark> | 
  labstore1002: mdadm /dev/md/slice15 --re-add /dev/sd{bb,ba,az} | 
  [production] | 
            
  | 15:14 | 
  <mark> | 
  labstore1002: mdadm /dev/md/slice15 --re-add /dev/sdaw | 
  [production] | 
            
  | 15:07 | 
  <mark> | 
  labstore1002: mdadm --zero-superblock /dev/sd{aw,bh,bg,bf,be,bd,bc,bb,ba,az}1 | 
  [production] | 
            
  | 15:04 | 
  <moritzm> | 
  enabled ferm in analytic1028 (initial hadoop worker) | 
  [production] | 
            
  | 15:04 | 
  <mark> | 
  labstore1002: mdadm --zero-superblock /dev/sdax1 && mdadm /dev/md/slice15 --re-add /dev/sdax | 
  [production] | 
            
  | 15:03 | 
  <krenair@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/231465/ - VE for all new enwiki accounts (duration: 00m 13s) | 
  [production] | 
            
  | 14:58 | 
  <mark> | 
  labstore1002: mdadm /dev/md/slice15 --re-add /dev/sday | 
  [production] | 
            
  | 14:58 | 
  <mark> | 
  labstore1002: mdadm --zero-superblock /dev/sday1 | 
  [production] | 
            
  | 14:53 | 
  <mark> | 
  labstore1002: mdadm --stop /dev/md3 | 
  [production] | 
            
  | 14:37 | 
  <ebernhardson> | 
  reset elasticsearch cluster.routing.allocation.disk.high back to 90% | 
  [production] | 
            
  | 13:38 | 
  <krinkle@tin> | 
  Synchronized w/: Remove rl-test.php (duration: 00m 13s) | 
  [production] | 
            
  | 13:17 | 
  <moritzm> | 
  enabled ferm on db1048 | 
  [production] | 
            
  | 13:09 | 
  <moritzm> | 
  enabled ferm on labsdb100[467] | 
  [production] | 
            
  | 12:01 | 
  <YuviPanda> | 
  disable puppet on labsdb1006 | 
  [production] | 
            
  | 08:58 | 
  <moritzm> | 
  enabled ferm on labsdb1001 | 
  [production] | 
            
  | 08:58 | 
  <godog> | 
  fixup current graphite retention for metrics under "servers" hierarchy T96662 | 
  [production] | 
            
  | 08:51 | 
  <moritzm> | 
  enabled ferm on labsdb1002 | 
  [production] | 
            
  | 08:31 | 
  <moritzm> | 
  enabled ferm on labsdb1003 | 
  [production] | 
            
  | 08:29 | 
  <godog> | 
  repool mw1125 mw1142 after nutcracker failures | 
  [production] | 
            
  | 07:45 | 
  <jynus> | 
  cloning mysql data from es1010 to es1017 [ETA: 6h] | 
  [production] | 
            
  | 07:23 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool es1010 (duration: 00m 12s) | 
  [production] | 
            
  | 07:13 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool es1007, pool es1013 (duration: 00m 13s) | 
  [production] | 
            
  | 06:36 | 
  <mutante> | 
  uploaded survey2012 to dumps/dataset1001; ownership as it is for survey2011; - T110746 in time for midnight PST | 
  [production] | 
            
  | 05:18 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Tue Sep  1 05:18:09 UTC 2015 (duration 18m 8s) | 
  [production] | 
            
  | 02:28 | 
  <l10nupdate@tin> | 
  LocalisationUpdate completed (1.26wmf20) at 2015-09-01 02:28:30+00:00 | 
  [production] | 
            
  | 02:25 | 
  <l10nupdate@tin> | 
  Synchronized php-1.26wmf20/cache/l10n: l10nupdate for 1.26wmf20 (duration: 06m 00s) | 
  [production] |