Hadoop, Spark, Deep Learning Mesh on Single GPU Cluster

February 24, 2016 Nicole Hemsoth When it comes to leveraging existing Hadoop infrastructure to extend what is possible with large volumes of data and various applications, Yahoo is in a unique position–it has the data and just as important, it has the long history with Hadoop, MapReduce and other key tools in the open source big data stack close at hand and manned with seasoned experts. With right around 36,000 Hadoop nodes and 600 petabytes of storage across 19 clusters running everything from MapReduce, Tez, Spark, and more, it stands to reason teams behind fresh AI and machine learning initiatives would have little interest in building out new clusters for training and execution of deep learning and more advanced machine learning frameworks. All the work on Hadoop at Yahoo has…


Link to Full Article: Hadoop, Spark, Deep Learning Mesh on Single GPU Cluster