To build Aviation GenAI Solutions for the FAA, Mosaic leveraged its award-winning Neural Search Engine to enhance the FAA’s Enterprise Information Management (EIM) project, developing state-of-the-art chatbot apps to process multiple data sources and provide GenAI-generated answers.
Take Our Content to Go
Market Analysis
The aviation industry generates vast amounts of data, making it ripe for efficient data management solutions. According to the FAA Aerospace Forecast FY 2024-2044, the aviation sector anticipates significant growth, with a substantial increase in data volume driven by advancements in commercial space operations and uninhabited aircraft systems (UAS). Efficient data management is critical, as outlined in the FAA’s strategic plans and initiatives, emphasizing the importance of leveraging technology to enhance safety and operations.
Background
Since 2004, Mosaic Data Science’s parent company, Mosaic ATM, has designed and deployed sophisticated air traffic management systems leveraging custom data analytics. Our roots in aviation analytics have equipped us with a unique perspective on applying AI and ML to complex, real-world challenges.
As a part of our work in aviation, Mosaic has maintained a long-standing relationship with the FAA. The FAA’s Enterprise Information Management (EIM) project is an internal FAA cloud-based data warehouse used for various purposes, including regulatory compliance, operational efficiency, safety management, and data analytics. In this project, Mosaic was tasked with developing AI/ML products, providing support for FAA users, and defining processes and use cases for AI/ML throughout the FAA. The challenge was managing and utilizing the vast amounts of unstructured data scattered across different systems efficiently to meet the FAA’s operational and regulatory needs and facilitate using the EIM platform.
Solution
Mosaic developed two AI/ML projects using the EIM platform, including an LLM applied to EIM documentation to create a chat tool and an LLM applied to search FAA TechOps documents, specifically those related to Instrument Landing Systems (ILSs). Both projects leveraged Mosaic’s award-winning Neural Search Engine and developed state-of-the-art LLM-powered chatbot apps to process multiple data sources and provide GenAI-generated answers.
Process for Developing Aviation GenAI Solutions for the FAA
The LLM apps implemented in the EIM project followed an enhanced deployment architecture based on the more general Neural Search Engine architecture. This Advanced Retrieval-Augmented Generation (RAG) architecture provides a systematic way of accessing enterprise knowledge relevant to a user-provided question, which is later fed to the LLM to provide the specific answer for the user without the need to browse a long list of potentially relevant results.
Many underlying technologies and software packages can be used to implement the RAG architecture. In the case of the EIM project, Mosaic’s team used the following technology stack. Due to security concerns and the requirement that the data stay within the EIM platform, this effort hosted an open-source LLM within the EIM platform. The advanced RAG implementation was tuned to maximize performance by tailoring the data processing and architecture to the needs of the FAA and EIM.
- Langchain is a framework designed to facilitate the development of LLM-powered applications.
- OpenSearch: a search and analytics engine designed to enable high-performance searching, real-time monitoring, and log analysis.
- FastChat: an open-source platform designed for building, deploying, and managing AI-powered chat applications.
- Vicuna: an open-source chatbot developed by LMSYS, fine-tuned from the LLaMA model using user-shared conversations
- FastAPI: a fast (high-performance), web framework for building APIs
Mosaic developed a data pipeline to ingest Confluence pages from multiple spaces, developer guides, Teams data, data management configurations, and SharePoint data into the EIM platform. This involved designing a sophisticated data ingestion process to extract, parse, and store the relevant data in a vector database. The EIM data management systems and applications were set to run in AWS. Figure 1 shows the two major AWS components the Mosaic team used: EC2 provides secure, resizable computing in the cloud, and the OpenSearch server, where the data previously ingested is stored and made available to the LLM APIs.
Figure 1. Deployment Approach for Developing Aviation GenAI Solutions for the FAA
The chat application’s development involved creating an end-to-end NLP and LLM search engine prototype. The goal was to provide a ChatGPT-like tool to search EIM documentation efficiently. The prototype underwent rigorous testing and refinement to ensure accuracy and efficiency in retrieving relevant documents. Mosaic deployed a Vicuna chat model within the FAA’s AWS environment. This model was chosen for its high performance and compatibility with AWS’s ecosystem.
The prompt engine was configured to interface seamlessly with the LLM via API endpoints, enabling real-time responses to user queries. When the user entered a question in the API, the question would be sent to the API component, which would first run the retriever to obtain the relevant data from the database. Finally, the reader would interact with the LLM via the hosted LLM API. The LLM answer and context used would then be sent to the front end and displayed to the user.
A screenshot of the application is shown in Figure 2. Three query modes were made available:
- LLM General: Where the general LLM knowledge is used to answer the questions.
- LLM General – Conversational: For conversational general questions. The system keeps track of previous questions and answers.
- LLM with EIM Tools Context: Using a RAG approach where answers are generated with EIM documentation context. The LLM answer is followed by the supporting context, which includes raw text extracted from the EIM documentation used to generate the answer. A thumbs up/down capability allows users to provide feedback, helping to guide further improvements in the backend.
Figure 2. ChatEIM UI Sample Question
The developed app acted as a single point of access to all the relevant information users need to get started with EIM, potentially allowing them to identify available documentation quickly. Comprehensive documentation detailing the solution’s architecture and features, along with extensive training sessions for FAA personnel were prepared. These sessions were designed to equip the staff with the necessary skills to effectively use and manage the chat platform, ensuring they could fully leverage its capabilities.
Results from the Aviation GenAI Solutions
The implementation of the Neural Search Engine significantly enhanced the efficiency of data retrieval and management processes. Key highlights included:
- Speed and Efficiency: The system could search vast amounts of EIM documentation and retrieve relevant information within seconds, significantly reducing the time required for manual searches.
- Enhanced Accuracy: The AI-driven search minimized errors and ensured that critical details were not overlooked, leading to a more thorough and reliable information retrieval process.
- User Feedback Integration: The system incorporated a feedback mechanism that allowed users to provide real-time input, helping to improve the platform and identify gaps in documentation continuously.
Conclusion
The collaboration between Mosaic and the FAA demonstrated the power of Neural Search technology in automating and enhancing the efficiency of complex data management processes. The ChatEIM platform not only potentially saves time and resources but also improves accuracy and cost-efficiency, ensuring that the FAA can effectively meet its operational and regulatory requirements. Mosaic’s tailored, scalable, and multi-modal search solution showcased how advanced AI technologies can address the limitations of traditional data management methods, providing a robust and flexible tool to meet the unique needs of the aviation industry.
