Meet ContextCheck: Our Open-Source Framework for LLM & RAG Testing! Check it out on Github!


Client: NDA

AI-driven Document Generation on AWS infrastructure

Case study details


Our client, an American company operating globally, specializes in organizing and coordinating private flights worldwide. The company sought a way to automate its internal processes related to knowledge management. Addepto was selected to design and implement an intelligent system for creating aviation documentation integrated with client’s infrastructure.



Challenge


The client sought to harness the potential of Large Language Models (LLMs) for Intelligent Document Processing (IDP) while mitigating their limitations in the sensitive area of flight operations. They chose Addepto for their expertise in creating domain-specific, seamlessly integrated solutions without costly infrastructure overhauls.



Approach


Addepto developed the Aviation Sourcing Chatbot to streamline ASR document generation, integrate with the TripGrade system, and operate on AWS. Despite initial plans to use Amazon Bedrock, Addepto opted for OpenAI models for better performance, and transformed the general-purpose model into a domain-specific one using business-specific knowledge and a microservices architecture.



Goal


The client’s objective was to develop a secure and reliable solution to streamline the creation of Aviation Service Request (ASR) documents, which detail all aspects of flight coordination. These documents include flight routes, aircraft availability, passenger numbers, VIP requirements, stops, and all legal, technical, and logistical details, with complexity arising from the interdependencies among these elements.



Outcome


The tool developed by Addepto enhanced the operators’ work by speeding up information retrieval and reducing errors. Acting as an assistant, the chatbot automatically created and submitted ASR documents, ensuring efficient trip planning and accurate detail collection for successful outcomes.


Challenge

Project Overview: Leveraging Large Language Models in Aviation Document Processing


The client was fully aware of the potential of Large Language Models (LLMs) in Intelligent Document Processing (IDP) systems and was determined to leverage this potential. At the same time, they recognized the limitations of LLMs, which were critical in the context of such sensitive issues as flight operations.

Addepto was chosen as the implementation partner for this project due to its experience in creating domain-specific solutions based on LLMs. Addepto’s solution eliminates hallucination effects and seamlessly integrates with the client’s systems without costly infrastructure overhauls.



Our team expert opinion







Approach

Developing the Aviation Sourcing Chatbot for Streamlining the Generation of ASR Documents


After consulting to thoroughly understand all nuances related to private flight planning, Addepto developed a plan to build and implement a solution named the Aviation Sourcing Chatbot. This chatbot’s primary goal was to streamline the generation of ASR documents, integrate with the TripGrade system, and operate on AWS infrastructure.

Initially, to reduce costs, Addepto’s Data Scientists planned to use Amazon Bedrock, which allows developers to easily create and deploy generative AI applications in AWS without managing any infrastructure. However, preliminary tests showed that while Bedrock’s default models, LLaMA 3 and Mistral, excelled in accuracy, they underperformed compared to OpenAI’s models (GPT 3.5 Turbo and GPT 4) in terms of performance. Therefore, Addepto decided to utilize OpenAI models.

The selection of the LLM was not the project’s most critical part. The core task was transforming a general-purpose model into a domain-specific one by “feeding” it business-specific knowledge. This involved connecting it to internal SQL databases and then creating operational logic that precisely defined the rules for using the available information.

Thus, a system plan with a microservices architecture was developed. It would connect to the client’s systems via API (FastAPI) and, thanks to prompt engineering, including dual response validation, determine how to use the available information. Developing the agent logic, which involves assigning specific prompts to specific actions, was a sensitive issue requiring extensive time and testing.

Here, Addepto had a significant advantage. Using a proprietary tool called ContentCheck, the team automated the testing and evaluation of prompts embedded in tools delivered to clients, significantly shortening their development process.


During project development, our team worked on:


  • Implementing the Aviation Sourcing Chatbot for ASR document generation.
  • Integrating with TripGrade and deploying on AWS.
  • Selecting and integrating OpenAI models for enhanced performance.
  • Connecting to internal SQL databases and designing a microservices architecture.
  • Developing efficient agent logic and utilizing ContentCheck for automated testing.

Goal

Enhancing ASR Document Creation in Flight Coordination




The client's objective was to develop a secure and reliable solution to streamline the creation of Aviation Service Request (ASR) documents. These documents encompass all details related to the coordination of a flight, which are numerous and interdependent.


An ASR document provides a comprehensive description of the flight route, including all its elements, such as the availability of an aircraft suitable for the required order, the number of passengers, and all VIP-specific requirements. It also includes the number of necessary stops and all legal, technical, and logistical details.


The complexity of handling such documentation lies not in the number of elements it must include but in the dependencies among them. For instance, the choice of aircraft for a selected route, determined by the number and requirements of passengers, impacts the number of required stops, airport availability, crew size, altitude limits, and many other factors. Although the document format is standardized, the logic behind its creation varies each time.


Technologies that we use



Commercial LLM

Platforms


GPT-4 (OpenAI)


GPT-4 (OpenAI) – The latest and most advanced language model from OpenAI, succeeding GPT-3.5. It has improved capabilities across various tasks like question-answering, writing, and coding.

GPT-3.5 (OpenAI)


GPT-3.5 (OpenAI) – The predecessor to GPT-4, known for its strong language generation abilities but with some limitations in areas like math and commonsense reasoning
AWS


AWS – Amazon SageMaker is a fully-managed service offered by AWS that makes it easy to build, train, and deploy machine learning models at scale. With Amazon SageMaker, Data Scientists and Machine Learning engineers can use pre-built algorithms, frameworks, and infrastructure to build and train custom models, or bring their own custom algorithms and models. It provides tools and features to support data labeling and preparation, automatic model tuning, and deployment to production. Amazon SageMaker is not the only ML service offered by AWS. There is also Amazon Rekognition for image and video analysis, Amazon Comprehend for natural language processing (NLP), and Amazon Personalize for building recommendation systems. AWS also provides APIs for machine learning tasks, such as Amazon Translate for language translation and Amazon Polly for text-to-speech conversion.

Outcome

Enhancing ASR Documentation with Automated Planning Assistance


The tool developed by Addepto did not replace the operators who manually filled out ASR documentation but significantly improved their work by speeding up information retrieval and reducing the number of potential errors.

The chatbot acted as an assistant, accelerating the planning process by automatically creating and submitting ASR documents.

Users benefited from a more efficient planning process with the automated creation and submission of ASR documents. This ensured that all essential details for trip preparation were accurately collected, leading to well-organized and successful trip outcomes.



Before


Manual ASR documentation with potential errors and slow information retrieval.



After


Automated ASR document generation with reduced errors and accelerated planning processes using an AI-powered chatbot.


About Addepto


Addepto is a leading AI consulting company, recognized by Forbes and Deloitte, for delivering cutting-edge AI and Data-driven solutions. We specialize in accelerating process automation and optimization within global enterprises using modern technologies.


Here you can learn more about the technologies used in this project:



About us


We are recognized as one of the best AI, BI, and Big Data consultants


We helped multiple companies achieve their goals, but - instead of making hollow marketing claims here - we encourage you to check our Clutch scoring.

Let's discuss
a solution
for you



Edwin Lisowski

will help you estimate
your project.










Required fields

For more information about how we process your personal data see our Privacy Policy





Message sent successfully!