Spark Singularity

README.md Use Spark on Heroku in a single dyno. Experiment inexpensively with Spark in the Common Runtime. Production-quality Spark clusters may be deployed into Private Spaces using spark-in-space. This buildpack provides the following three processes, children of the main web process: Nginx proxy for basic password authentication set via environment variable SPACE_PROXY_BASIC_AUTH format username:{PLAIN}password Spark master web UI https://your-spark-app.herokuapp.com/ REST API https://your-spark-app.herokuapp.com/rest one Spark worker https://your-spark-app.herokuapp.com/worker ? This app should not be scaled beyond a single dyno. (There is no coordination mechanism between multiple instances; implicitly use 127.0.0.1:7077 as Spark Master.) Submitting & controlling jobs Because Spark Singularity is contained in a single dyno with only port 80 exposed, there are two options for submitting jobs: Spark’s REST API, proxied at https://your-spark-app.herokuapp.com/rest Declare the Spark jobs to submit on start-up…


Link to Full Article: Spark Singularity

Pin It on Pinterest

Share This

Join Our Newsletter

Sign up to our mailing list to receive the latest news and updates about homeAI.info and the Informed.AI Network of AI related websites which includes Events.AI, Neurons.AI, Awards.AI, and Vocation.AI

You have Successfully Subscribed!