Data Bricks Infra Engineer

Job Description: Role: Data Bricks Infra Engineer

Location: Gurgaon (5 days mandatory work from office)

Type: Full-time

Experience: 8+ years

Key Responsibilities

• Design, build, and maintain secure connectors and pipelines between Databricks workspaces (Delta Lake, Unity Catalog) and Model Context Protocol (MCP) servers.

• Implement and configure MCP servers/clients to expose Databricks data, schemas, and analytical tools securely to AI models and LLM applications.

• Optimize data retrieval, caching mechanisms, and query performance between Databricks and LLM orchestration frameworks to minimize latency.

• Ensure all data exposed through the MCP server adheres to strict enterprise data governance, access controls, and Unity Catalog permissions.

• Partner with AI/ML engineers, data scientists, and software architects to define the context, tools, and prompts required for LLM applications to effectively query Databricks.

• Establish robust logging, error-handling, and monitoring for the Databricks-MCP middleware to ensure high availability and reliability.
  

Must-Have Skills

• Proven, hands-on experience building, configuring, or extending MCP servers (using Python or TypeScript/Node.js SDKs) to connect LLMs to external data sources.

• Deep production experience with Databricks (Delta Lake, Unity Catalog, Databricks SQL, and Managed MLflow

• Strong understanding of SSE (Server-Sent Events), WebSockets, and JSON-RPC 2.0 protocols, which underpin MCP communication

Responsibilities: Key Responsibilities

• Design, build, and maintain secure connectors and pipelines between Databricks workspaces (Delta Lake, Unity Catalog) and Model Context Protocol (MCP) servers.

• Implement and configure MCP servers/clients to expose Databricks data, schemas, and analytical tools securely to AI models and LLM applications.

• Optimize data retrieval, caching mechanisms, and query performance between Databricks and LLM orchestration frameworks to minimize latency.

• Ensure all data exposed through the MCP server adheres to strict enterprise data governance, access controls, and Unity Catalog permissions.

• Partner with AI/ML engineers, data scientists, and software architects to define the context, tools, and prompts required for LLM applications to effectively query Databricks.

• Establish robust logging, error-handling, and monitoring for the Databricks-MCP middleware to ensure high availability and reliability.

• Qualifications: • Proven, hands-on experience building, configuring, or extending MCP servers (using Python or TypeScript/Node.js SDKs) to connect LLMs to external data sources.

• Deep production experience with Databricks (Delta Lake, Unity Catalog, Databricks SQL, and Managed MLflow

• Strong understanding of SSE (Server-Sent Events), WebSockets, and JSON-RPC 2.0 protocols, which underpin MCP communication