A Comprehensive Overview of the Databricks AI Capabilities

In 2025 and early 2026, Databricks offers a comprehensive suite of generative AI tools built on its data lakehouse and Data Intelligence Platform foundation. This article focuses specifically on these GenAI capabilities.

Over the past two years, the Databricks platform has evolved significantly, moving beyond the traditional lakehouse paradigm toward a fully integrated environment for data engineering, machine learning, and AI-native analytics.

The GenAI Databricks toolkit includes:

• Mosaic AI Gateway: Unified access to both Databricks-native models (such as DBRX) and external LLMs
• Vector Search: A fully integrated vector database optimized for retrieval-augmented generation (RAG)
• AI Agent Framework: Tools for creating, deploying, and evaluating AI agents
• SQL and BI Integration: Natural language interfaces for data interaction
• Agent Bricks: A framework for building production-ready enterprise AI agents
• Mosaic AI Model Serving and Training: Infrastructure for hosting and fine-tuning large language models
• Genie: Conversational analytics for enterprise data

These tools integrate seamlessly with Databricks’ governance framework (Unity Catalog), enabling enterprises to implement production-grade AI while maintaining security, lineage tracking, and regulatory compliance.

If you want to verify whether your current Databricks setup is secure, cost-efficient, and ready for large-scale GenAI workloads, consider running a dedicated Databricks Audit. At Addepto, we help organizations uncover misconfigurations, optimize performance, and strengthen governance across workspaces, clusters, and data layers.

Key Insights

• Databricks shifted from a “lakehouse + Spark” tool into a fully integrated GenAI platform, where data, models, and governance coexist; GenAI is not an add-on but embedded across the stack.
• The Mosaic AI layer acts as a control plane for LLM usage: one API abstracts multiple providers, enabling routing, monitoring, guardrails, and cost optimization without changing application code.
• Vector Search operationalizes RAG natively: embeddings are synchronized with Delta tables, so AI systems always query fresh, governed enterprise data without separate pipelines.
• The agent ecosystem turns LLMs into composable systems: agents combine retrieval, tools, and reasoning, with MLflow providing full traceability and Unity Catalog enforcing strict access boundaries.
• The platform’s strength is tight integration (data + AI + governance), but this comes with engineering trade-offs: higher cost (DBU), notebook-first workflow, and some constraints in flexibility vs. fully custom cloud-native stacks.

Databricks Platform Fundamentals

Before diving into the details of these GenAI capabilities, we briefly cover Databricks platform fundamentals to ensure all readers share a common understanding of the underlying architecture. If you are familiar with those concepts, skip to “GenAI Tools Available in Databricks.”

What is Databricks?

Databricks is a cloud-native data and AI platform available from three major cloud providers: Azure, AWS, and Google Cloud.

Originally built on Apache Spark, the platform now combines distributed compute, storage abstraction, governance, machine learning infrastructure, and generative AI services into a single environment.

In practice, Databricks operates as a unified analytics and AI platform layered on top of cloud object storage, such as Amazon S3, Azure Data Lake Storage, or Google Cloud Storage.

Organizations choose Databricks primarily for its ability to process and analyze large volumes of structured, semi-structured, and unstructured data.

The platform supports a wide range of workloads:

• traditional BI and analytics
• data engineering pipelines
• machine learning model development
• generative AI and agent-based systems

Recent developments have further expanded the platform toward the concept of a Data Intelligence Platform, where data, metadata, and AI models operate within a unified ecosystem.

If your team is at the beginning of its Databricks journey, a structured Databricks deployment can ensure that the platform is set up with the right architecture, security boundaries, and compute configuration from day one. Our team supports organizations in launching Databricks environments tailored to both analytical and GenAI use cases.

Read more: Best practices for Databricks PoC (Proof of Concept)

Databricks Architectural Components

Delta Lake

One of the most important elements of Databricks’ architecture is Delta Lake, a storage layer that provides reliable data processing in the ETL (extract, transform, load) model.

Delta Lake introduces features such as:

• ACID transactions
• schema enforcement
• time travel
• scalable metadata handling

These capabilities allow organizations to build reliable data pipelines on top of cloud object storage.

Recent developments such as Delta Lake UniForm allow Delta tables to be accessed using Iceberg-compatible APIs, improving interoperability with other analytics engines.

Unity Catalog

Above the data layer operates Unity Catalog, a centralized governance system that manages metadata, permissions, lineage tracking, and auditing.

Unity Catalog now governs not only datasets but also:

• machine learning models
• vector indexes
• dashboards and BI assets
• AI functions and agents

By centralizing governance across these assets, Unity Catalog enables enterprises to implement AI systems while maintaining strict security and compliance requirements.

Databricks Medallion Architecture

Data processing in Databricks often relies on the medallion architecture, which divides pipelines into three logical layers:

1. Bronze: Raw data ingestion from operational systems and external sources.
2. Silver: Cleaned and transformed datasets used for analytical processing.
3. Gold: Business-ready datasets optimized for analytics, reporting, and AI models.

This layered architecture helps maintain data quality and enables modular pipeline design.

It also simplifies the integration of AI pipelines into existing data workflows.

GenAI Tools Available in Databricks

Understanding the platform’s fundamental mechanisms is crucial because GenAI tools build on these foundations.

Let’s look at the most important AI solutions available in the Databricks ecosystem.

Mosaic AI Gateway

Mosaic AI Gateway serves as a central access point to various large language models (LLMs), enabling organizations to standardize how generative AI models are accessed, governed, and monitored within the Databricks platform.

Instead of integrating separately with multiple AI providers, organizations can route all model requests through a single gateway layer that provides unified management, security, and observability.

This includes access to:

• Databricks-native models, such as the DBRX large language model developed by Databricks

• open-source models, including families such as Mistral, Llama, and other community models

• commercial models from external providers, such as OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, and Google Vertex AI

The Gateway exposes a unified API compatible with the OpenAI interface, allowing developers to build applications once and switch between different models without changing application logic.

This significantly simplifies switching between models, implementing routing logic, and managing request traffic across multiple providers.

Organizations can use the Gateway to implement model routing strategies, such as:

• directing high-complexity requests to advanced models

• routing simple requests to lower-cost models

• distributing traffic across multiple providers for reliability

This abstraction layer also allows organizations to experiment with new models without rewriting application infrastructure.

Key functionalities of Mosaic AI

Advanced security mechanisms

The system includes guardrails and monitoring capabilities designed to prevent leakage of sensitive information and enforce governance policies.

These mechanisms can include:

• prompt filtering

• response filtering

• policy enforcement

• detection of sensitive data such as PII

All requests and responses can be logged and stored in Delta tables, allowing organizations to audit model usage, monitor outputs, and evaluate AI behavior.

This logging capability is particularly important for regulated industries that require full traceability of AI interactions.

Precise permission management

Access to model endpoints is managed through Unity Catalog, enabling organizations to define which users, services, or applications can access specific models.

This governance layer ensures that model usage aligns with organizational security policies and compliance requirements.

Permissions can be applied at multiple levels, including:

• individual models

• API endpoints

• datasets used for retrieval

Flexible deployment options

Organizations can run models using serverless endpoints, which automatically scale based on demand, or deploy models on dedicated GPU infrastructure for high-performance workloads.

This flexibility allows organizations to balance cost, performance, and control depending on the use case.

Custom models can also be deployed and fine-tuned using Mosaic AI Training and Model Serving, enabling enterprises to train models on proprietary datasets.

Inference tables and observability

Model interactions can be captured in inference tables, which store prompts, responses, metadata, and evaluation metrics.

These records enable organizations to:

• analyze model behavior

• detect hallucinations or errors

• evaluate model performance

• improve prompts and model configurations

When combined with MLflow tracing, these logs provide deep visibility into how AI systems behave in production environments.

Databricks Vector Search

Vector Search is a serverless vector database integrated directly with the Databricks ecosystem, designed to support large-scale semantic search and retrieval systems.

It is used primarily in retrieval-augmented generation (RAG) architectures, where large language models retrieve contextual information from enterprise datasets before generating responses.

Unlike standalone vector databases, Databricks Vector Search is deeply integrated with the Lakehouse architecture and works directly with Delta tables.

Key capabilities include

Automatic synchronization

Vector indexes can be automatically generated and updated based on Delta tables.

When new data is added to the table, embeddings can be automatically generated and inserted into the vector index.

This removes the need for complex synchronization pipelines and ensures that AI systems always operate on the latest data.

High scalability

Vector Search supports extremely large embedding datasets, potentially scaling to billions of vectors.

The architecture separates storage and compute resources, enabling efficient query processing even at large scale.

The system is optimized for high query throughput and low latency, making it suitable for enterprise search and AI assistant applications.

RAG integration

Vector search enables LLMs to retrieve contextual information from enterprise knowledge bases before generating responses.

This approach significantly improves the accuracy and reliability of AI outputs by grounding responses in real organizational data.

Typical RAG use cases include:

• internal knowledge assistants

• customer support systems

• document search platforms

• enterprise copilots

Governance integration

Vector indexes and embeddings are governed through Unity Catalog, ensuring that access policies applied to underlying datasets also apply to AI retrieval systems.

This governance model ensures that AI applications respect the same security policies as traditional analytics workloads.

AI Agents and Workflows

Databricks enables rapid development and deployment of AI agents through integration with MLflow and Mosaic AI tooling.

These capabilities support the development of agent-based systems, where AI models can perform complex tasks by interacting with data, tools, and external systems.

Key components of this ecosystem include

Mosaic AI Agent Framework

The Mosaic AI Agent Framework provides infrastructure for building agents capable of:

• retrieving data from enterprise datasets

• calling APIs or tools

• performing reasoning steps

• generating structured outputs

Agents can combine multiple capabilities such as RAG, tool use, and reasoning workflows.

MLflow tracing

MLflow provides detailed tracing for agent workflows, allowing developers to track every step of an agent’s reasoning process.

This includes:

• prompts sent to models

• tool calls

• retrieved documents

• intermediate reasoning steps

Tracing helps identify errors such as hallucinations or incorrect tool usage.

AI Playground

The AI Playground is an interactive environment where developers can experiment with prompts, models, and agent workflows.

It allows rapid prototyping and testing of AI systems before deploying them to production.

Agent evaluation tools

Databricks provides evaluation frameworks for measuring agent performance.

These tools support both automated evaluation and human-in-the-loop feedback, enabling organizations to continuously improve AI systems.

Unity Catalog functions

Unity Catalog functions allow organizations to define controlled tools that agents can access.

This ensures that agents only interact with approved data sources and APIs, improving system security.

Together, these components allow organizations to deploy complex AI workflows with full observability, governance, and monitoring capabilities.

Agent Bricks

Agent Bricks is a framework designed for building enterprise-grade AI agents on top of Databricks data infrastructure.

While many agent frameworks exist in the open-source ecosystem, Agent Bricks focuses specifically on production-ready enterprise deployments.

It provides templates for common agent patterns such as:

• knowledge assistants

• document processing agents

• analytical agents

• multi-agent orchestration systems

Agent Bricks automates many of the tasks typically required when building AI agents.

These include:

• prompt tuning and optimization

• generation of evaluation datasets

• model benchmarking and selection

• automated evaluation pipelines

By automating these tasks, Agent Bricks reduces the complexity of deploying AI agents in production environments.

It also integrates tightly with MLflow, Unity Catalog, and Mosaic AI, enabling full governance and lifecycle management.

GenAI in SQL and BI Tools

Databricks has introduced the ability to call AI models directly from SQL queries, bringing generative AI capabilities into traditional analytics workflows.

These SQL functions allow analysts to use AI models without writing Python code or building separate machine learning pipelines.

This enables tasks such as:

• text classification

• document summarization

• automated translation

• sentiment analysis

• semantic enrichment of datasets

These capabilities allow analysts and BI teams to incorporate AI workflows into existing reporting pipelines.

For example, AI functions can automatically categorize customer feedback, summarize support tickets, or extract insights from textual datasets.

This significantly expands the role of AI in traditional business intelligence environments.

Genie — Conversational BI

Genie is a natural language interface for querying enterprise data.

It allows users to interact with datasets using conversational language rather than writing SQL queries.

Users can ask questions such as:

• “What were our top-selling products last quarter?”

• “How did revenue change across regions in the past year?”

The system translates these requests into SQL queries and generates visualizations or dashboards.

Genie understands dataset schemas and relationships between tables, enabling it to automatically join datasets when necessary.

Because Genie is integrated with the Data Intelligence Engine, it can also learn organizational terminology and business logic from historical queries.

Genie can also be integrated into agent-based workflows, allowing AI agents to retrieve analytical insights from enterprise data.

Real Advantages and Limitations of Databricks in the Context of GenAI

After discussing key functionalities, it’s worth looking at an objective analysis of the platform’s strengths and limitations, with particular emphasis on practical aspects of implementations.

Databricks advantages

1. Integrated tool ecosystem
   Databricks offers an extensive set of GenAI tools that are mutually integrated and work together within a single platform. Particularly important is the aforementioned integration with Unity Catalog, enabling precise access management to data and functions at the level of the entire ecosystem.
2. Granular permission management
   Unity Catalog Functions allow for detailed definition of tools and assignment of permissions to them. This makes it possible to design agents that have access only to selected functions and data, which significantly raises the level of security and flexibility of implementations.
3. Effective use of data from Delta Tables
   Advanced operation mechanisms on Delta Tables and the ability to process both structured and unstructured data enable rapid building of AI solutions that previously presented considerable implementation difficulties.
4. Support for microservice architecture
   The platform favors designing solutions based on AI microservices. Instead of one monolithic system, it’s possible to create an ecosystem of specialized agents, each responsible for a specific range of functionality and operating on dedicated data resources.
5. Automated deployments and efficient scaling
   Databricks enables quick transfer of solutions to production environments and automatic infrastructure scaling. This significantly simplifies the process of prototyping and implementation, eliminating the need to build custom DevOps systems.

Databricks limitations

1. Developer environment based on Notebooks
   The platform promotes development in notebooks, which can be a significant constraint for teams accustomed to working in local IDE environments. Although dedicated plugins are available, they don’t provide full independence and offline work comfort that some developers are accustomed to.
2. Preference for specific frameworks
   A significant part of the examples and recommended solutions is based on the Langchain framework. Despite declared support for alternative tools, Langchain is strongly promoted, which may create a barrier for teams preferring other technological solutions.
3. Limited availability of selected out-of-the-box functionalities
   Not all advanced tools, such as OCR, are available as standard. Building such pipelines often requires independent implementation and integration within Databricks notebooks.
4. Billing model and cost aspects
   Databricks is a more expensive solution compared to native cloud services—the additional overhead can range from 25% to even 100% relative to base cloud services. Billing is done in the Databricks Units (DBU) model, and dedicated cost calculators are available for individual tools, facilitating expense estimation.
5. Stream processing with noticeable delays
   Streaming in the Databricks environment doesn’t function in strictly real-time mode—delays can range from several to a dozen minutes, depending on the complexity of the transformations being performed.

Industry-specific implementation scenarios

Databricks’ AI capabilities are being deployed across industries in distinctive ways that address sector-specific challenges:

Finance

Financial institutions are leveraging Databricks’ unified platform to improve risk assessment, detect fraud, and enhance customer experiences. JP Morgan Chase uses Databricks to process over 1 billion transactions daily, applying AI models to detect potentially fraudulent activities in near real-time. The medallion architecture proves particularly valuable for maintaining regulatory compliance while enabling innovation.

Retail

H&M utilizes Databricks to analyze customer data from both online and in-store interactions. Real-time processing enables personalized recommendations, optimized inventory management, and trend prediction, leading to increased customer loyalty and reduced inventory costs

Supply Chain and Logistics

Chevron Phillips Chemical Company partnered with Databricks and Seeq to scale industrial IoT analytics and machine learning for time-series data, improving operational insights and efficiency.

Healthcare and Life Sciences

In healthcare, organizations are implementing Databricks to accelerate research, improve patient outcomes, and optimize operations. Mayo Clinic’s implementation integrates clinical, genomic, and imaging data to power AI models that predict disease progression and treatment effectiveness. The platform’s ability to handle both structured and unstructured data (including medical images and clinical notes) provides a comprehensive view of patient health.

Summary: Gen AI on Databricks

Databricks constitutes a comprehensive platform for designing and implementing GenAI solutions, offering a rich set of mutually integrated tools, precise access management, and support for modern microservice architecture. Among the significant limitations are the notebook-based development environment, preference for specific frameworks, higher costs, and insufficient availability of some advanced functions in the basic package.

The platform will work particularly well in scenarios requiring rapid integration of AI solutions with existing data resources and flexible permission management.

However, for more complex, non-standard implementations, it may require additional work and adaptations to the specific requirements of the organization.

This article was originally published on May 7, 2025, and was recently updated on Mar 17, 2026, to add information about new models and sections.