配置Spark standalone HA 主机:node1,node2,node3master: node1,node2slave:node2,node3 修改配置文件:node1,node3: spark-env.shexport SPARK_MASTER_IP=no ...
配置Spark standalone HA
主机:node1,node2,node3
master: node1,node2
slave:node2,node3
修改配置文件:
node1,node3: spark-env.sh
export SPARK_MASTER_IP=node1export SPARK_MASTER_PORT=7077export SPARK_WORKER_CORES=1export SPARK_WORKER_INSTANCES=1export SPARK_WORKER_MEMORY=1024mexport SPARK_LOCAL_DIRS=/data/spark/dataDirexport SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=node1:2181,node2:2181,node3:2181 -Dspark.deploy.zookeeper.dir=/sparkHA"
原标题:Spark standalone HA
关键词:
*特别声明:以上内容来自于网络收集,著作权属原作者所有,如有侵权,请联系我们:
admin#shaoqun.com
(#换成@)。