This repository was archived by the owner on Nov 23, 2017. It is now read-only.
This repository was archived by the owner on Nov 23, 2017. It is now read-only.
How can launch Spark 2.0.1 and Hadoop 2.7? #64
Description
Right now it seems the latest hadoop version in spark-ec2 is 2.4, but actually in spark download page, it can be up to 2.7, also it is available in aws s3 http://s3.amazonaws.com/spark-related-packages. So the question is - how to launch such Spark cluster and if it does not support is there any workaround?
--hadoop-major-version=HADOOP_MAJOR_VERSION
Major version of Hadoop. Valid options are 1 (Hadoop
1.0.4), 2 (CDH 4.2.0), yarn (Hadoop 2.4.0) (default:
yarn)