Click here to Skip to main content
15,867,977 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi I am trying to read hive table through spark but spark is giving below error. Please find below spark-submit command i used.

spark-submit --master yarn --deploy-mode cluster 
--jars PATH_TO_APPJAR,path/hive-jdbc-1.2.1.spark2.hdp.1.jar 
--files /path/to/hive-site.xml 
--conf spark.yarn.maxAppAttempts=1 
--conf spark.sql.hive.hiveserver2.jdbc.url="correct_jdbc_URL" 
--conf spark.datasource.hive.warehouse.metastoreUri="thrift://HOSTNAME:9083" 
--conf spark.hadoop.hive.llap.daemon.service.hosts="@llap0" 
--conf spark.hadoop.hive.zookeeper.quorum="HOSTNAME:PORT" 
--class CLASSNAME


in my code is used spark session as below
SparkSession spark = SparkSession.builder()
        .appName("spark_practice")
        .config("spark.sql.warehouse.dir","PATHNAME")
        .enableHiveSupport()
        .getOrCreate();


ERROR looks like below
<pre>org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
.
.
.
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: Peer indicated failure: DIGEST-MD5: IO error acquiring password



What I have tried:

i have tried using HiveContext instead of SparkSession but getting same error
also i ran below command to start metastore service but its giving different error as below. Not sure if both are related
Posted

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS


CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900