Building LLM Solutions with Amazon Bedrock

Since OpenAI launched the first versions of the GPT series, LLMs have advanced significantly, resulting in widespread adaptation across numerous sectors. The analytics and NLP understanding capabilities of LLMs make them invaluable assets in various business applications, including content generation, customer service, and business intelligence.

However, building a custom LLM solution isn’t cut and dry. Organizations require vast computational resources, access to huge datasets, and skilled human resources to put the system together. This has put a strain on smaller organizations without access to these resources, prompting them to rely on open-source models that often lack the level of specialization required to drive impactful ROI.

Amazon Web Services recently announced that Bedrock is now generally available. The platform comes with impressive features that empower developers to build LLMs with ease. And, as a fully managed service, Bedrock eliminates the need for vast computational resources required to train and fine-tune LLMs, making the technology more available.

This guide aims to provide insights on how organizations can leverage Bedrock to build LLMs, some case studies of effective development and deployment, and general information on LLMs.

Overview of large language models (LLMs)

Also known as Large Language Models, LLMs are deep learning models that can understand and generate human language. [1] These models are typically trained on huge sets of data, hence the name ‘large’. Their underlying infrastructure comprises a set of neural networks called a transformer model. The transformer model consists of an encoder and decoder with self-attention capacities, which enable the model to accurately understand the meaning and context of words in a series of texts.

Transformer-based LLMs are capable of unsupervised learning. They achieve this through self-learning mechanisms that enable them to understand languages, basic grammar, and general knowledge.

The way they process information also sets them apart from earlier recurrent neural networks (RNNs). Traditional RNNs process inputs sequentially. Transformer-based LLMs, on the other hand, process entire sequences in parallel. This unique capability allows developers to utilize GPUs to train LLMs, significantly reducing training time.

The transformer-based architecture also allows the use of very large language models, often containing billions of parameters. These models can ingest and process massive amounts of data, making them suitable for a wide variety of applications.

How do large language models work?

One of the most unique attributes of LLMs lies in their ability to respond to unpredictable queries. Unlike traditional computer programs, which receive commands in their accepted syntax, or from a certain set of inputs by the user, LLMs can respond to natural human language and use data analysis to respond to unstructured questions in a way that makes sense.

Likewise, earlier forms of machine learning used numerical tables to represent each word in a sentence. However, this approach came with several limitations, the most notable being their inability to recognize the relationship between words in a sentence, thus limiting their contextual understanding.

LLMs overcame this limitation by leveraging multi-dimensional vectors, better known as word embeddings. Word embeddings are generally used to represent words with a similar contextual meaning or other relationships that are close to each other in a given vector space.

Read more: Not only GPT. What LLMs can you choose from?

By leveraging word embeddings, transformer-based LLMs can preprocess text as numerical representations through the encoder. This way, the model is better able to understand the contextual meaning of words, phrases, and other relationships between words as part of speech.

Conversely, deep-learning LLMs utilize machine learning algorithms and probability analysis to learn and understand the context of words. This way, they are better able to predict how to logically finish an incomplete sentence and even generate new sentences.

There’s another subset of LLMs built on neural networks. These types of LLMs utilize an artificial neural network comprised of several layers: Typically, an input and output layer with several other layers in between. Information is typically passed from one layer to another once it passes a certain quality threshold, resulting in greater accuracy.

Introduction to Amazon Bedrock

Amazon’s bedrock is a generative AI tool designed for application development. It comes as a fully managed service that provides access to leading foundation models (FMs) through a single endpoint. With this release, Amazon Web Services aims to democratize access to generative AI technology and simplify the development of generative AI applications.

Essentially, Amazon’s Bedrock allows organizations to utilize various foundation models to build powerful generative AI applications for their business use case without having to deal with the hustle of managing associated infrastructures. This also means organizations don’t have to invest in skilled machine learning specialists to build their models – at least not as much as they would with total in-house development.

The available foundation models on the platform are trained on huge datasets using advanced techniques and also be customized to perform specific tasks. The foundation model also comes with unique strengths and application domains, facilitating seamless customization and integration with existing applications.

Read more: Introduction to Amazon Bedrock and Generative AI

Key features of Amazon Bedrock

Some of the most notable features of Bedrock include:

Access to leading foundation models

Bedrock offers a wide range of foundation models from different prominent AI companies. You can find models from Cohere, Anthropic, AI21 Labs, Stability.ai, Mistral AI, Meta, and some Amazon models as well. FM models are typically suited to different tasks. Having a variety of models to choose from gives organizations the flexibility they need to build Generative AI models for different use cases.

Model customization and retrieval augmented generation

For an organization to be able to extract maximum value from its generative AI applications, it must first infuse and customize the model with private organizational data. This makes the model more accurate and effective in specific use cases.

With Bedrock, organizations can customize models using various fine-tuning features. They can even create separate, private copies of models for convenient experimentation. Similarly, RAG allows organizations to enhance a model’s context with proprietary data sources for more accurate and relevant responses.

Security and compliance

Amazon deals directly with all FM-model AI companies. It manages the FMs on behalf of these companies. This means that any data you upload will remain within the AWS framework and will be protected in accordance with Amazon’s data protection and safety standards.

Amazon currently supports over 100 data security compliance certifications, helping organizations meet worldwide regulatory requirements.

Seamless integration with Amazon Web Services (AWS)

Bedrock integrates seamlessly and securely with various AWS services you’re already familiar with to help create secure, reliable, and scalable GenAI applications. For instance, Bedrock leverages Amazon S3 for data training and validation, CloudWatch for tracking metrics, and AWS Lambda for invoking actions.

Agents for executing multi-step tasks
As companies see the value in automation, AI companies are trying to figure out new ways to improve automation to deliver greater business value. 24% of companies have already started implementing low-code process automation systems. [2]

Such companies can benefit greatly from Bedrock, particularly in use cases where they want to automate processes and execute multi-step tasks based on the model’s performance. By leveraging agents in the Bedrock framework, organizations can significantly fast-track their prompt creation with customized instructions, call necessary APIs to fulfill the required task, orchestrate a series of actions, and trace the FM’s reasoning and orchestration of complex tasks.

Benefits of using Amazon Bedrock

Bedrock does more than just help organizations build GenAI applications faster and easier – it also offers other business and customer-oriented benefits that could not only transform how GenAI applications are built, but also increase the trust, reliability, and scalability of AI applications.

Some of the most notable benefits of using Bedrock include:

Secure customization

One of the most crucial aspects of organization-wide generative AI adaptation is the ability for organizations to securely customize applications to align with their specific needs and use cases.

Read more: RAG vs Fine-Tuning: A Comparative Analysis of LLM Learning Techniques

Besides RAG and fine-tuning capabilities, Bedrock also offers a unique approach to model customization, ensuring that sensitive data remains protected throughout the process, such as:

Model customization data security

Bedrock uses encrypted training data imported from an Amazon Simple Storage Service (Amazon S3) bucket through a VPC connection when fine-tuning a model. Additionally, Amazon doesn’t use your model customization data for any other purpose, i.e., your training data isn’t used to train Titan models or distributed to a third party.

Amazon also refrains from collecting personalized data such as logged account IDs, usage timestamps, or any other information logged by the service. What’s even more impressive is that none of the training or validation data you provide for training or fine-tuning is stored on Bedrock. Once you complete the customization process, all the data is isolated and encrypted with your KMS keys.

Centralized multi-account model access

In over 54% of cyber-attacks in the US over the past year, initial access was gained via compromised credentials. [3] This is a major concern, especially in organizations where multiple teams collaborate on projects requiring multi-account access.

With Bedrock, organizations can centrally manage their IT environments across multiple accounts. This way, team leaders can create and organize accounts, consolidate costs, and implement policies for custom environments.

Organizations with multiple AWS accounts or application architectures can greatly benefit from centralized governance and access to FMs. This way, they can better secure their AI environment, centrally manage permissions, and create and share resources.

Additionally, by simply using standard AWS cross-account IAM roles, team leaders and admins can give secure access to models in different accounts, facilitating secure and auditable usage while maintaining a centralized point of control.

Secure deployment of fine-tuned models

Fine-tuned and pre-trained models are typically deployed in isolated environments for individual accounts. Organizations can further encrypt these models with their KMS keys, thus preventing unauthorized access without appropriate IAM permissions.

Seamless audibility and visibility

Bedrock doesn’t just isolate and encrypt your data – it also provides various other capabilities that enable seamless audibility and accelerate incidence reports when needed. This way, organizations can use key metrics indicators to gain insights that allow for predictive analytics and proactive measures in order to prevent similar events from happening again. [4]

Some of these capabilities include:

Monitoring and logging

Bedrock’s integration with AWS CloudTrail and Amazon Cloudwatch provides comprehensive logging, monitoring, and visibility to token consumption, API activity, and other performance data. With these capabilities, organizations can continuously monitor their models for improvement, customization, and auditing.

Bedrock allows you to create detailed logs of all model inputs and outputs, including the IAM invocation role and all metadata associated with calls in your account. The data is stored on bedrock and is only available within your account. These logs can help monitor model response, allowing organizations to implement more effective AI policies and reputation guidelines.

Compliance certifications

Compliance with regulatory requirements is one of the biggest concerns for organizations that handle personalized customer data. [5]. However, with Bedrock, organizations can improve compliance with major regulations including the Health Insurance Portability and Accountability Act (HIPAA), General Data Protection Regulation (GDPR), and more.

AWS has also successfully extended Bedrock’s Infrastructure Service Providers in Europe Data Protection Code of Conduct (CISPE CODE) Public Register. The declaration provides independent verification as well as an added assurance that Bedrock can be used in compliance with the GDPR.

By staying compliant, organizations can successfully eliminate some of the legal bottlenecks of deploying their GenAI applications and build trust among their partners and customers, particularly around data security and governance.

How to build LLM solutions with Amazon Bedrock

You need an AWS account to access Bedrock and start building. Once your account is up and running, follow the instructions to set up Amazon’s Bedrock and request model access for the models you want to enable.

One of the best ways to experiment with the platform is using the Playground feature, which enables you to experiment with different models before settling for one. You can use the feature for image, text, and chat use cases. You can also leverage the configurable parameters for each model to customize randomness, response length, and diversity of answers.

Here’s how to build a generative AI solution with Bedrock:

Prompt engineering

Prompt engineering is the process of carefully crafting prompts to guide LLMs effectively. Well-designed prompts can significantly boost the performance of a GenAI application by providing context, clear instructions, and examples of the tasks at hand.

Bedrock supports multiple prompt engineering techniques. For instance, you can use few-shot prompting to provide the model with examples of desired outputs to help it better understand tasks. Similarly, zero-shot prompting is used to provide task descriptions without examples, and chain-of-thought prompting is used to enhance multi-step reasoning.

Retrieval Augmented Generation (RAG)

LLMs generally lack specialized knowledge, context, jargon, or up-to-date information related to specific tasks. For instance, legal professionals cannot rely on generalist LLM solutions for reliable, accurate, and up-to-date information in their domain.

That’s where RAG comes in. RAG is the process of allowing an LLM model to consult an authoritative knowledge base outside its training data before delivering a response, thereby increasing the model’s reliability in domain-specific tasks.

To increase your model’s reliability, you can leverage the RAG feature to connect the model to organizational data, enabling it to deliver accurate, relevant, and customized responses. Bedrock also allows you to securely access data from new sources like the web, index public web pages, or connect to enterprise data repositories like SharePoint, Confluence, and Salesforce.

Bedrock also has impressive advanced chunking options that enable you to create custom chunking algorithms tailored to your specific needs.

Model customization

Model customization is the process of customizing pre-trained models for specific tasks or domains. It often involves taking a pre-trained model and further training it on smaller, specialized datasets related to your specific use case.

This unique approach leverages the knowledge acquired during the initial training phase while adopting the model to your specific requirements without losing the original model’s capabilities.

Bedrock’s fine-tuning process is designed to be scalable, efficient, and cost-effective. This enables you to tailor LLMs to your specific needs without the need for extensive data or computational resources. The platform also supports model customization for both labeled and unlabeled data.

Amazon Bedrock: Case studies

Some of the most notable case studies for users who are successfully leveraging Bedrock’s capabilities to improve model development and service delivery include:

• Airtable: Airtable, a product of Airtable AI, uses bedrock to transform workflows with generative AI. The company has successfully leveraged AWS’s initiative to democratize AI adoption for non-technical users across the organization. This way, Airtable can successfully build and deploy models without extensive skills and computation resource requirements.
• Netsmart: Netsmart uses Bedrock to reduce the burden of clinical documentation. The company recently built a revolutionary application using Bedrock and Amazon Connect, improving customer service and employee satisfaction across the organization.
• Nasdaq: Nasdaq leverages generative AI capabilities to deliver sustainability insights. This has enabled the AI-powered serverless platform to successfully reduce the manual effort required for sustainability and environmental, social, and governance (ESG) disclosures.

The future of LLMs and Amazon Bedrock

The introduction of LLMs that can answer questions and generate text points to exciting possibilities in the future. These possibilities are further accentuated by improvements in model development provided by Bedrock.

As organizations continue to leverage Bedrock for LLM model development, they will significantly speed up the reliability of LLMs to near human-level performance.

Here are a few possible insights into the future of LLMs:

Increased capabilities

While they’ve come a long way since the first generation GPT models were released by OpenAI, the current level of technology has not yet attained perfection. However, as organizations continue to leverage new techniques and approaches in model development, newer releases will have increased accuracy and enhanced capabilities.

Audiovisual training

Most LLMs available today have been trained on text. However, a new trend is emerging: AI companies are leveraging Bedrock Playground to train models using video and audio inputs. This new approach could lead to faster model development and open up new possibilities for LLM solutions in autonomous vehicle applications.

Workplace transformation

LLMs will undoubtedly transform the workplace. As their capabilities improve, LLMs will significantly reduce manual, repetitive tasks, much like what robots did in the manufacturing sector.

Some of the most notable possibilities include customer service chatbots, repetitive clerical clerks, and simple automated copywriting.

Amazon Bedrock – final thoughts

Amazon Bedrock is transforming the way IT companies build and deploy models. By providing a fully managed platform with FMs suited for different tasks, any organization, regardless of size and access to vast computational resources, can successfully build a functional GenAI application that meets the organization’s unique requirements.

Eventually, this will lead to the proliferation of LLMs in nearly every sector, propelling the growth and utilization of generative AI. While there are fears that generative AI may replace certain human professions, its potential benefits in transforming the workplace far outweigh its limitations.

References

[1] AWS.amazon.com. What is a Large Language Model? URL: https://aws.amazon.com/what-is/large-language-model.  Accessed on August 23, 2024
[2] Imaginovation.net. Business Automation Statistics. URL: https://tiny.pl/93wgdzwr. Accessed on August 23, 2024
[3] Cisa.gov. CISA Analysis: Fiscal Year 2022 Risk and Vulnerability Assessments. URL:  https://www.cisa.gov/sites/default/files/2023-07/FY22-RVA-Analysis%20-%20Final_508c.pdf,Accessed on August 23, 2024
[4]Riskonnect. What is Incident Reporting and Why is it Important. URL: https://tiny.pl/16b45-51. Accessed on August 23, 2024
[5] Cio.com. 7 Biggest IT Compliance Headaches and How CIOS Can Cure Them. URL: https://www.cio.com/article/288731/compliance-7-biggest-it-compliance-headaches-and-how-cios-can-cure-them.html, Accessed on August 23, 2024