Exploring Vector Databases and API Models in Generative AI: Applications of Generative AI in Various Industries

We’ve talked with Kiryl Halozhyn, Solutions Architect at Databricks, about how Generative AI will change businesses and the technical aspects of implementing Generative AI within different infrastructures.

This interview was originally conducted in May 2024. The core architecture it describes — choosing between API-based and open-source models, and leaning on vector databases and RAG rather than fine-tuning — still holds up well. What has changed since is covered in a new section further down: the rise of agentic AI, and a more structured way to think through the RAG-versus-fine-tuning decision.

The Interview with Kiryl Halozhyn, Solutions Architect at Databricks

What are the main applications of Generative AI in companies?

Initially, Gen AI was mainly used in the creation of various types of chatbots, support bots, and simply Q&A engines. Today, these solutions seem quite basic, but in the beginning, they required courage because the technology was quite new, so the risk was significant. However, now we see more and more advanced MVP solutions that are being applied for specific scenarios to improve companies’ processes and are usually based on the use of vector databases utilizing both unstructured and structured data.

Thanks to them, the so-called foundational models are enriched with very specific knowledge, which allows them to address more specific business problems, decreasing the risk of, for example, hallucinations. It can be said that we combine impressive conversational abilities known from ChatGPT with the appropriate company-specific knowledge.

What models are used as the basis for such solutions?

API-based models from OpenAI are usually used initially in PoCs because using a ready-made API in a small test solution is usually not a high cost, and the model response quality is very high. However, many companies decide to reach for an open-source model at a later stage during the productionalization phase, looking for lower latencies and better cost performance metrics. In the case of a product that is to be implemented throughout the company, the costs of using a smaller single open-source model or multiple models in a chain are lower.

In the initial phase, there was a lot of talk about fine-tuning, but it doesn’t seem to be the solution that companies are eager to reach for at the moment.

Fine-tuning is the next stage for Gen AI use cases, even more complicated and costly. The use of vector databases does not require preparing data for fine tuning and a big amount of GPU computing power. RAG can be done using OpenAI and simply paying for the number of tokens retrieved from the API. This is quite simple. However, to do fine-tuning, you need to have your own model training architecture, well-prepared huge datasets of custom data and provisioning of a large number of GPU instances. There are multiple companies on the market which specialize in model fine tuning for the companies and thus will take part of the complexity away — MosaicML was one such example before its 2023 acquisition by Databricks, where its technology now operates as part of Databricks’ own Mosaic AI training tools rather than as an independent vendor.

Will the future of domain-specific Gen AI solutions be based on open-source models?

Rather yes. It is difficult to use one large model for every scenario, every company, so in my opinion, the future will favor smaller, faster, cheaper models that will be focused on specific use cases. Combining multiple open source models in a single use case will provide the best cost performance outcome.

However, most solutions based on LLMs are interfaces that essentially offer the same thing as ChatGPT.

Yes, many such SaaS have been created, but as it seems to me they have a fairly defined lifespan. In the beginning, it might have made sense because there was a lack of knowledge on how to implement these scenarios using LLM and Gen AI in general, how to define their parameters, etc. Now major tech companies on the market have their own foundational models and overall companies have more knowledge of how to implement their own solutions, whether using SaaS APIs or own hosted models.

Is there, or do you think, that the development of Generative AI will focus on finding and addressing these areas specific to given domains for a given company?

Using publicly available models involves companies having to give the green light to use their data and send it to 3rd party SaaS API. It seems to me that this will play a major role in the development of smaller open-source models hosted on their own infrastructure.

By using the Open AI API, our data theoretically is not used to train the model.

Theoretically, yes, but this is a Microsoft product, so, for example, someone who does not use Azure or is not in the cloud at all has to verify Microsoft in terms of processing personal data.

What other barriers are blocking the introduction of AI?

Company culture and quality of unstructured data. Many companies do not think about AI because they are still in the process of implementing foundational data platforms, such as data lakehouses and it will take time for them to start prioritizing Gen AI use cases. Moreover, as before LLMs companies usually focused on the quality of structured data and creating a single layer of truth for it, now unstructured data starts to play a crucial role in the success of Gen AI solutions. Therefore, many companies have to first do their homework with getting sharepoint, confluence, jira and other internal data prepared for future usage.

But I believe that with time, LLM use cases will become a crucial part of companies data strategy and now we’re just seeing the beginnings of this new trend.

What’s Changed Since 2024: Agentic AI and the RAG-vs-Fine-Tuning Decision

The interview above still describes the right starting architecture for most companies: start with an API-based model for a proof of concept, move to open-source models and RAG with a vector database for production, and treat fine-tuning as a last resort given its cost and complexity. Two things have changed since 2024 that are worth adding to that picture.

Agentic AI is the next layer, not a different architecture

Agentic AI — systems capable of carrying out multi-step workflows with reduced human intervention at each stage — is projected to reach roughly 40% of enterprise applications by the end of 2026, up from under 5% in 2025. Some estimates put the current pace of adoption in similar terms:

Named production examples are already emerging at scale: JPMorgan Chase has reportedly scaled to around 1,000 internal AI use cases, and Klarna’s AI assistant reportedly handles over 65% of customer service chats, with a cited $40 million annual profit improvement. Importantly, agentic AI doesn’t replace the RAG-and-vector-database architecture Kiryl describes above — it typically sits on top of it. An agent that handles a multi-step customer request still needs to retrieve accurate, up-to-date context from somewhere, and a vector database remains one of the most common ways to give it that context reliably.

A more structured way to choose between RAG and fine-tuning

At the time of the original interview, the guidance was directional: RAG is usually the more practical starting point, fine-tuning is a costly next step. Since then, this has become enough of a recurring decision that it’s worth a clearer framework rather than a general preference:

• Choose RAG when your knowledge base changes frequently, you need to cite or trace sources, or you want to avoid the upfront cost of building training infrastructure.
• Choose fine-tuning when you need the model to reliably follow a very specific style, format, or domain-specific reasoning pattern that retrieval alone doesn’t fix — and you can justify the GPU and data-preparation cost.
• Choose a hybrid approach when you need both: a lightly fine-tuned base model for tone and domain reasoning, combined with RAG for facts that change over time.

One practical risk worth flagging for any RAG-based architecture, including the kind of MVP solutions described in this interview: if the underlying embedding model is deprecated or changed, every vector previously generated with it can become incompatible with new queries, sometimes requiring a costly, multi-week re-embedding of the entire dataset. This is a governance and vendor-selection consideration worth raising early, not just a technical detail to handle later.

Naming the Vector Databases Behind the Architecture

The interview above discusses vector databases conceptually — as the mechanism that lets a foundational model retrieve specific, structured and unstructured company knowledge rather than relying purely on its training data. In practice, choosing which vector database to build on is itself a decision worth naming explicitly:

• pgvector — a PostgreSQL extension, a practical choice when a team already runs Postgres and doesn’t need to manage a separate database for smaller-scale deployments (commonly cited as suitable up to tens of millions of vectors).
• Pinecone — a fully managed service, often chosen for high-scale production deployments where a team prefers not to operate the infrastructure itself.
• Weaviate — open-source, with native hybrid search (combining vector similarity with traditional keyword search) built in.
• Qdrant — open-source, written in Rust, generally chosen for performance and strong metadata filtering.
• Milvus (and its managed variant, Zilliz Cloud) — a widely adopted open-source option.
• Chroma — a lightweight, developer-friendly option often used for local development and smaller projects before a production-scale database is needed.

For context on the pace of change here: the global vector database market was estimated at roughly $3.2 billion in 2025, growing at around 24% annually, and combining vector search with traditional keyword search in a hybrid setup commonly improves retrieval recall by 15–30% over pure vector search alone.

Sum Up

The core guidance from this interview still holds: start with an API-based model like OpenAI’s for a proof of concept, move to open-source models and a vector-database-backed RAG architecture once you’re heading toward production, and treat fine-tuning as a costly last resort rather than a default choice.

What’s changed since the original 2024 conversation is what sits around that core architecture, not the architecture itself. Agentic AI has emerged as a layer on top of RAG rather than a replacement for it — an agent handling a multi-step task still needs a reliable way to retrieve accurate, current context, and a vector database remains one of the most common ways to provide it. Choosing between RAG, fine-tuning, and a hybrid of the two has also become enough of a recurring decision to warrant a clearer framework, rather than the general directional preference that was reasonable in 2024. And where the original interview discussed vector databases only conceptually, naming the actual products in play today — pgvector, Pinecone, Weaviate, Qdrant, Milvus/Zilliz Cloud, Chroma — makes the decision concrete: it comes down to scale, whether a managed service or self-hosted infrastructure fits the team better, and whether hybrid search is a day-one requirement.

In short: the fundamentals Kiryl described are a solid starting point in 2026 as much as they were in 2024 — they now just need agentic AI, a structured RAG-vs-fine-tuning decision process, and a concrete vector database choice layered on top.

References

1. Databricks. “Databricks Completes Acquisition of MosaicML.” URL: https://www.databricks.com/company/newsroom/press-releases/databricks-completes-acquisition-mosaicml. Accessed Aug 14, 2026.
2. Addepto. “Fine-Tuning LLMs. Benefits, Costs, Challenges.” URL: https://addepto.com/blog/fine-tuning-llms-benefits-costs-challenges/. Accessed Aug 14, 2026.
3. Addepto. “Gen AI in Business: Global Trends Report 2024.” URL: https://addepto.com/news/gen-ai-in-business-global-trends-report-2024/. Accessed Aug 14, 2026.
4. G&CO. “Generative AI for Enterprise: Trends, Use Cases, Industry Analysis.” URL: https://www.g-co.agency/insights/generative-ai-for-enterprise-trends-use-cases-industry-analysis. Accessed Aug 14, 2026.
5. TechTarget. “The Future of Generative AI: Trends to Follow.” URL: https://www.techtarget.com/ai/feature/The-future-of-generative-AI-Trends-to-follow. Accessed Aug 14, 2026.
6. INI8 Labs. “Vector Databases & RAG: The 2026 Enterprise Guide.” URL: https://ini8labs.tech/blog/vector-databases-rag-enterprise-guide-2026. Accessed Aug 14, 2026.
7. Iternal.ai. “Best Vector Databases 2026.” URL: https://iternal.ai/insights/best-vector-databases-2026. Accessed Aug 14, 2026.

This article was updated on Aug 14, 2026. It corrects an outdated reference to MosaicML as an independent fine-tuning vendor — Databricks completed its acquisition of MosaicML in 2023, and its technology now operates as part of Databricks’ Mosaic AI tools rather than as a separate company. It also adds a new section on agentic AI, a more structured RAG-versus-fine-tuning decision framework, a new section naming the specific vector database products (pgvector, Pinecone, Weaviate, Qdrant, Milvus/Zilliz Cloud, Chroma) that the original title referenced conceptually but did not name, and a new Sum Up and FAQ section synthesizing the above. Several newly added statistics are cited from industry sources.