Apache Spark 2.0 Technical Preview

Two years after the first release of Apache Spark, Databricks announced the technical preview of Apache Spark 2.0 , based on upstream branch 2.0.0-preview. The preview is not ready for production, both in terms of stability and also API, but is a release intended to gather feedback from the community ahead of the general availability release. This new release is focused on feature improvements based on community feedback. There are two main areas of improvement regarding Spark’s development. One of the most used interfaces for Apache Spark based applications is SQL. Spark 2.0 offers support for all the 99 TPC-DS queries which are largely based on SQL:2003 specification. This alone can help porting existing data loads into a Spark backend with minimal rewriting of the application stack. The second aspect…


Link to Full Article: Apache Spark 2.0 Technical Preview