Spark Streaming update to address growing torrent of big data

The Apache Spark distributed data processing framework is being prepped for another step forward.Details of the 2.0 version of the software, disclosed at Spark Summit 2016 East in New York, indicated the next open source Spark revision will include improvements to Spark Streaming. Data streaming has gained increasing interest of late, as ever-larger amounts of Web and mobile data arrive, and more applications focus on keeping that big data in motion. Also on tap with Spark 2.0 are some merged APIs and a general boost in the core Spark system’s performance. Updates to its SQL query capabilities and machine learning APIs are due, too. But the streaming updates hold particular importance for an emerging class of distributed processing that is based on what is described as a Lambda architecture. In…


Link to Full Article: Spark Streaming update to address growing torrent of big data