Fix Databricks HDFS Authentication Error

₹600-1500 INR

Completed

Posted about 1 month ago

₹600-1500 INR

Paid on delivery

I have an Azure Databricks SQL warehouse that needs to read from an external Hive metastore hosted on HDFS using thrift service. The moment the warehouse tries to reach that store, the job fails with an “authentication failure” coming back from HDFS. I am using a token-based authentication flow rather than Kerberos or a simple username/password combination, and I suspect the problem lies either in how the token is passed or in mismatched Hadoop core-site settings inside Databricks.

Cloud Computing

Data Warehousing

Project ID: 40290711

About the project

3 proposals

Remote project

Active 1 mo ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

Awarded to:

Hello, I understand you are facing a "token-based authentication failure" between Azure Databricks and your external HDFS Hive metastore. This is likely due to a mismatch in the [login to view URL] configurations or the way the delegation token is being passed in the Spark session. I can help you debug and fix this by: Verifying the [login to view URL] and token provider settings. Checking the [login to view URL] and Thrift service connectivity. Ensuring the Hadoop core-site settings within your Databricks cluster are correctly mapped to your HDFS token flow. I have a strong background in SQL, Database Programming, and Data Warehousing. I am available to start right now and can jump on a quick chat to resolve this for you. Best regards, Amit S.

₹1,000 INR in 1 day

5.0

(1 review)

0.0

0.0

3 freelancers are bidding on average ₹1,183 INR for this job

I have expertise in skills of SQL, Cloud Computing, Azure, Hadoop, Data Warehousing, big data, and Apache Spark

₹1,050 INR in 7 days

0.0

(0 reviews)

0.0

0.0

Post a project like this

About the client

New Delhi, India

5.0

5

Payment method verified

Member since May 1, 2016

Client Verification

Other jobs from this client

Tech Niche Blog Website Development

₹1500-12500 INR

UI Design Basics Online

₹600-1500 INR

Looking for content development for class 9-10 maths science

₹12500-37500 INR

Build a website

₹1500-12500 INR

Looking for azure architect training

₹1500-12500 INR

Similar jobs

SQL Server to PostgreSQL Migration Specialist

₹400-750 INR / hour

Palantir Foundry Developer (Python/SQL, PySpark, AIP, Ontology,Workshop) – ₹45K+ Negotiable – India

₹45000-60000 INR

SQL Expert Needed – Debug & Fix the Stored Procedures

$30-250 USD

Fix website glitches , Search, Email Issues for bulk email campaigns

$250-750 USD

Oracle Forms 4.5 Database Connection

₹1500-12500 INR

Discord-Controlled Xbox Cloud Gaming

$1500-3000 USD

AWS Infrastructure Setup & Automation

₹500000-521000 INR

Zero-Downtime WordPress Cloud Migration

$200 USD

Build Azure Razor Pages App

$8-15 USD / hour

ERPNext & POS Integration for Liquor Retail

₹12500-37500 INR

Cloud Zerodha Quantitative Trading Bot

₹12500-37500 INR

A Blockchain-based Decentralized SDN Control Plane to Mitigate Single-Point-of-Failure DDoS Attacks in Vehicular Networks

₹1500-12500 INR

Python Script for Tachograph Analysis

$750-1500 USD

Diagnosed a subtle RAG retrieval regression — fast, thorough, and precise

$30-250 USD

Tech Space Optimization & Travel Help

$15-25 USD / hour

Entry-Level Data Analyst Resume

₹750-1250 INR / hour

Android Social Media App Build

₹750-1250 INR / hour

Real-Time Storm Tracker Development

$3000-5000 USD

HAProxy Configuration Implementation

€6-12 EUR / hour

Audit App Transformation: Local and Remote Capabilities

$30-250 USD

Post a project like this