| 
      
        2017-02-14
      
      ยง
     | 
  
    
  | 20:38 | 
  <eileen1> | 
  update civicrm from 8d04e75939c5300f0a18e03fdb044c6673b3a74e to 0fb289fd385f9f3a71df0c1dedbba8d1b5555730 | 
  [production] | 
            
  | 20:36 | 
  <otto@tin> | 
  Finished deploy [analytics/refinery@67c3924]: Deploying refinery with update to drop hourly partitions script (duration: 02m 25s) | 
  [production] | 
            
  | 20:33 | 
  <otto@tin> | 
  Started deploy [analytics/refinery@67c3924]: Deploying refinery with update to drop hourly partitions script | 
  [production] | 
            
  | 20:22 | 
  <Dereckson> | 
  Update site statistics for pam.wikipedia (T158110, now 454 images) | 
  [production] | 
            
  | 20:03 | 
  <thcipriani@tin> | 
  Started scap: testwiki to php-1.29.0-wmf.12 and rebuild l10n cache | 
  [production] | 
            
  | 19:23 | 
  <krinkle@tin> | 
  Synchronized docroot/noc/conf: I67194fceffd3f61 (duration: 00m 48s) | 
  [production] | 
            
  | 19:22 | 
  <krinkle@tin> | 
  Synchronized dblists/: I67194fceffd3f61 (duration: 01m 37s) | 
  [production] | 
            
  | 18:39 | 
  <arlolra> | 
  Updated Parsoid to 79ccfb93 (T58381, T108216) | 
  [production] | 
            
  | 18:28 | 
  <thcipriani> | 
  starting branch cut for 1.29.0-wmf.12 | 
  [production] | 
            
  | 18:27 | 
  <arlolra@tin> | 
  Finished deploy [parsoid/deploy@1bfb86b]: Updating Parsoid to 79ccfb93 (duration: 09m 58s) | 
  [production] | 
            
  | 18:17 | 
  <arlolra@tin> | 
  Started deploy [parsoid/deploy@1bfb86b]: Updating Parsoid to 79ccfb93 | 
  [production] | 
            
  | 18:11 | 
  <MaxSem> | 
  Purged https://he.wikipedia.org/static/images/mobile/copyright/wikipedia-wordmark-he.svg with purgeList.php | 
  [production] | 
            
  | 17:34 | 
  <bblack@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=cp1067.eqiad.wmnet | 
  [production] | 
            
  | 16:18 | 
  <andrewbogott> | 
  rebooting californium | 
  [production] | 
            
  | 16:15 | 
  <andrewbogott> | 
  dist-upgrade californium (as part of the liberty->mitaka upgrade) | 
  [production] | 
            
  | 16:06 | 
  <Niharika> | 
  Updated wikimania app to 5c44d06 Removed stale translations | 
  [production] | 
            
  | 16:02 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Change db2062 IP after its move to another rack - T156478 (duration: 00m 39s) | 
  [production] | 
            
  | 16:01 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Change db2062 IP after its move to another rack - T156478 (duration: 00m 40s) | 
  [production] | 
            
  | 15:11 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Allow sysops to add/revoke account creator on it.wikiversity (T158062) (duration: 00m 41s) | 
  [production] | 
            
  | 15:05 | 
  <marostegui> | 
  Shutdown mysql (and later the whole host) on  db2062 for maintenance - T156478 | 
  [production] | 
            
  | 15:04 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Depool db2062 to change its rack - T156478 (duration: 00m 41s) | 
  [production] | 
            
  | 15:01 | 
  <ema> | 
  lvs10*: upgrade to pybal 1.13.5 T147425 | 
  [production] | 
            
  | 14:48 | 
  <moritzm> | 
  installing php security updates on einsteinium | 
  [production] | 
            
  | 14:41 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1073,89,90,91,92 (duration: 00m 41s) | 
  [production] | 
            
  | 14:38 | 
  <ema> | 
  lvs300[12]: upgrade to jessie 8.7, pybal 1.13.5, reboot into kernel 4.4.2-3+wmf8 T155401 T147425 | 
  [production] | 
            
  | 14:35 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1073,89,90,91,92 (duration: 00m 40s) | 
  [production] | 
            
  | 14:20 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1055,72,83,56,84,76,78,87,71 (duration: 00m 40s) | 
  [production] | 
            
  | 14:17 | 
  <hashar> | 
  European SWAT is complete | 
  [production] | 
            
  | 14:16 | 
  <hashar@tin> | 
  Synchronized php-1.29.0-wmf.11/extensions/CirrusSearch/profiles/SimilarityProfiles.php: Explicitly use BM25 as default for wmf_defaults similarity profile (duration: 00m 47s) | 
  [production] | 
            
  | 14:14 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1055,72,83,56,84,76,78,87,71 (duration: 00m 41s) | 
  [production] | 
            
  | 14:11 | 
  <ema> | 
  lvs300[34]: upgrade to jessie 8.7, pybal 1.13.5, reboot into kernel 4.4.2-3+wmf8 T155401 T147425 | 
  [production] | 
            
  | 14:09 | 
  <hashar@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable TwoColConflict on dewiki - T155721 (duration: 00m 40s) | 
  [production] | 
            
  | 14:06 | 
  <hashar@tin> | 
  Synchronized wmf-config/throttle.php: Throttle rule for cswiki - T158040 (duration: 00m 40s) | 
  [production] | 
            
  | 13:50 | 
  <ema> | 
  lvs400[12]: upgrade to jessie 8.7, pybal 1.13.5, reboot into kernel 4.4.2-3+wmf8 T155401 T147425 | 
  [production] | 
            
  | 13:32 | 
  <ema> | 
  lvs400[34]: upgrade to jessie 8.7, pybal 1.13.5, reboot into kernel 4.4.2-3+wmf8 T155401 T147425 | 
  [production] | 
            
  | 13:28 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1051,66,80,74,77,56,81,70,82 (duration: 00m 44s) | 
  [production] | 
            
  | 13:15 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1051,66,80,74,77,56,81,70,82 (duration: 00m 40s) | 
  [production] | 
            
  | 12:59 | 
  <jynus> | 
  reloading/restarting gerrint on cobalt, too slow | 
  [production] | 
            
  | 11:57 | 
  <elukey@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=mw2221.codfw.wmnet | 
  [production] | 
            
  | 11:30 | 
  <moritzm> | 
  manual fix up of exim spool permissions on krypton (used to run the heavy exim variant) | 
  [production] | 
            
  | 11:28 | 
  <jynus> | 
  performing schema change on all mariadb servers T150474 | 
  [production] | 
            
  | 10:51 | 
  <ema> | 
  lvs200[123]: upgrade to jessie 8.7, pybal 1.13.5, reboot into kernel 4.4.2-3+wmf8 T155401 T147425 | 
  [production] | 
            
  | 10:48 | 
  <moritzm> | 
  installing tomcat security updates | 
  [production] | 
            
  | 10:23 | 
  <ema> | 
  lvs200[456]: upgrade to jessie 8.7, pybal 1.13.5, reboot into kernel 4.4.2-3+wmf8 T155401 T147425 | 
  [production] | 
            
  | 10:17 | 
  <ema> | 
  uploading pybal 1.13.5 to apt.w.o T147425 | 
  [production] | 
            
  | 09:05 | 
  <moritzm> | 
  upgrading firejail on sca cluster | 
  [production] | 
            
  | 08:45 | 
  <moritzm> | 
  installing vim security updates | 
  [production] | 
            
  | 08:33 | 
  <moritzm> | 
  restarting zookeeper on conf1003 | 
  [production] | 
            
  | 08:23 | 
  <moritzm> | 
  restarting zookeeper on conf1002 to pick up OpenJDK update (restarts were stopped yesterday to further investigate gc behaviour) | 
  [production] | 
            
  | 06:56 | 
  <marostegui> | 
  Deploy alter table on x1 echo_notification tables - T136428 | 
  [production] |