Skip to main content

Build with (drivers, SDKs, programmatic mAPIs)

  • JDBC Driver — official Databricks JDBC (v3+) and legacy Simba docs; Java 11+ supported.
  • ODBC Driver — Windows/macOS/Linux; connects to clusters or SQL warehouses.
  • REST APIs — Workspace/Account APIs, Jobs, MLflow; unified reference hub.
  • CLI — modern Databricks CLI (wraps REST); install via Homebrew/WinGet; legacy CLI deprecated.
  • SQL CLI — run SQL against SQL Warehouses from terminal (as-is).

Connect from (ingest / stream / ELT)

  • Kafka (Structured Streaming) — read/write Kafka as source/sink; MSK IAM guidance.
  • Partner Connect — curated partner setup flows from the Databricks UI (creates SQL Wh, PAT, etc.).
  • Unity Catalog Integrations — third-party engines via Unity REST / Iceberg REST catalog.

Orchestration & DevOps

  • Airflow ProviderDatabricksSubmitRunOperator, RunNow, SQL Statements operator (varies by provider version).
  • Terraform Provider — manage workspaces, clusters, jobs, UC objects with IaC.
  • dbt-databricks Adapter — dbt Core on Databricks; Unity Catalog aware; Photon-accelerated SQL.

BI & Analytics

  • Power BI — native Databricks connector; OAuth/Service Principal; ADBC driver option in Preview; Delta Sharing connector.
  • Tableau — native connector (Desktop/Cloud/Server), OAuth via Partner Connect; “Explore in Tableau” + Delta Sharing connector.

“Which should I use?” (quick decision flow)

  1. I’m building an app and need SQL access
    → Use JDBC/ODBC based on your runtime; for automation, prefer REST APIs + CLI. JDBC v3+ is the current track; Simba docs apply to v2.x.
  1. Streaming events (pub/sub) to/from the lakehouse
    → Use Structured Streaming with Kafka (source/sink). If on AWS MSK, follow IAM notes.
  1. Spin up partner tools fast (ETL/ELT, catalogs, security)
    → Start in Partner Connect to provision SQL Warehouses, tokens, and connector configs automatically.
  1. Orchestrate jobs & infra
    Airflow provider operators for jobs/SQL; Terraform for reproducible workspace/UC config; dbt-databricks for governed transformations.
  1. BI consumption
    Power BI with OAuth (or Service Principal) and optionally ADBC; or Tableau with OAuth/Partner Connect and Explore in Tableau; consider Delta Sharing to serve external consumers.