On Tools for Finding the Right Context for GenAI RAG Solutions. Q&A with Imran Chaudhri
The growing adoption of generative AI (GenAI) is transforming industries, but its effective deployment, particularly in enterprise applications, requires overcoming significant challenges, particularly when it comes to accuracy. Solutions like Retrieval Augmented Generation (RAG), when paired with advanced tools such as the Progress Data Platform, can provide the much-needed context and accuracy to GenAI outputs. In this Q&A with with Imran Chaudhri, we will explore key questions surrounding the challenges and tools necessary for successful GenAI RAG implementations.
Q1: What are the main challenges when using GenAI?
The most common challenges enterprises face with GenAI are:
1. Data Quality: Poor data quality leads to inaccurate or incomplete results. Enterprises struggle to maintain clean, relevant and updated data, especially when dealing with large datasets.
2. Hallucinations: GenAI models can generate false or misleading information, which is a major concern in professional environments where trust and accuracy are crucial.
3. High Costs: Embedding and inferencing can be expensive, especially when handling large volumes of data.
4. Security and Compliance: Using sensitive enterprise data in GenAI models can lead to breaches of privacy, regulatory non-compliance or exposure to cybersecurity threats.
Q2: Is it possible to reduce “hallucinations” in GenAI’s outputs?
Yes, it is possible to reduce hallucinations by realizing that “Context is King”. The better the context that is provided to the GenAI model, the more grounded and accurate its answers.
Context can be provided through several strategies:
– RAG Solutions: Incorporating RAG enables GenAI models to pull real-time data from reliable sources, grounding the AI’s responses in factual, enterprise-specific information. This reduces the likelihood of hallucinations as the model doesn’t rely solely on its pre-trained knowledge.
– Knowledge Graphs and Vector Databases: By employing solutions like MarkLogic and Semaphore (components of the Progress Data Platform) to capture knowledge graphs from subject matter experts (SMEs), you can have confidence that the model references semantically rich and contextually accurate information.
– Human-in-the-Loop: Leveraging SMEs to create, validate or curate the data used in generative AI workflows also decreases the risk of hallucinations.
In fact, the Progress Data Platform has patent pending technology called “Progressive Graphs” that rapidly enable the creation of use-case specific graphs by monitoring systems usage and identifying gaps in your knowledge graphs.
Q3: What are tools that can be used to produce the best context for GenAI systems?
Use-case specific SME driven customized knowledge graphs are the best way to find and deliver highly relevant context. Studies have shown, however, that multiple search and query technologies working together provide even better results. Enterprises should use all the tools at their disposal simultaneously to find the right context including graphs, vectors, search, tabular queries, co-occurrences and geospatial queries. All these technologies can be used simultaneously in the Progress Data Platform with the Optic APIs. Given that the Progress Data Platform is a transactional database with all these disparate views of the same data, the database is updated the moment new data is stored.
Q4: What are your tips to improve the reliability and trustworthiness of GenAI outputs?
I’ll start this answer by noting that “Great AI requires great IA” (information architecture).
“Great IA” improves the reliability of GenAI outputs by:
– Enhancing Data Quality: Solutions such as MarkLogic offer capabilities like data harmonization and deduplication, so that only accurate, clean data is used by the AI model.
– Transparent Data Lineage: Using enterprise data platforms like the Progress Data Platform promotes the traceability of every data source and its transformations. This allows users to audit and verify the origin of the GenAI output.
– Integration of Private Data: Merging proprietary enterprise data with the generative AI engine significantly enhances the trustworthiness of the output.
Q5: What are the implications of the complexity of GenAI and the accuracy of the final output?
The complexity and probabilistic nature of GenAI models often leads to unpredictable outputs, where a lack of clarity or incorrect data in the training set can result in biased or incomplete answers. In enterprise applications, this translates into increased risk of making faulty business decisions, higher costs due to unnecessary rework and potential legal liabilities if inaccurate information is disseminated.
Q6: How do you maintain data security and compliance with enterprise standards when using GenAI?
Enterprise-grade solutions like the Progress Data Platform provide strong mechanisms to help enterprises better adhere to security and compliance standards:
– Role-Based Access Control (RBAC) and Query-Based Access Control (QBAC): Access to data can be tightly controlled based on user roles so that GenAI only uses data the user is allowed to see. RBAC can be performed at the document/row level as well as the element/column level.
– Encryption and Governance: Data is encrypted in transit and at rest, with strict governance policies to monitor and track how data is accessed, modified or deleted.
– Multi-Domain Security: The Progress Data Platform can support thousands of enterprise security domains, supporting flexible and secure content access. In a large enterprise, you can easily have hundreds if not thousands of security domains. RBAC and QBAC can easily handle these environments whereas creating custom fine-tuned AI models for each data and security domain rapidly becomes cost prohibitive.
Q7: How do you promote data quality in GenAI systems?
Data quality can be improved through the following:
– Harmonization and Enrichment: Using a unified data solution, such as MarkLogic, allows organizations to harmonize data across various sources, improving consistency and reducing noise.
– Data Mastering: The platform can deduplicate and cleanse data while maintaining a single source of truth, essential for accurate generative AI results.
– Real-Time Access: Feeding only the most relevant, up-to-date data into the GenAI model enhances accuracy and relevancy. In fact, most of our customers don’t want to send older data to the GenAI model as that increases incorrect answers.
Q8: Why are GenAI RAG solutions important?
RAG solutions are critical because they bridge the gap between the broad, generalized training data used by generative models and the specific, proprietary data held by enterprises. By grounding generative outputs in real-time enterprise data, businesses can enhance the relevancy, accuracy and reliability of the AI’s responses.
Q9: How do you scale GenAI?
Scaling GenAI requires:
– Scaling Queries and Data: A distributed shared-nothing transactional architecture helps you scale both queries and data into petabyte scales. Further scaling can be achieved by creating data mesh, data fabrics and data hubs, and only storing the metadata needed for context search and discovery.
– Efficient Vectorization: Tools like Progressive Vectors only vectorize the data your users are actually interested in, reducing costs and computational overhead while scaling with the needs of the enterprise.
– Cloud Neutrality: The Progress Data Platform offers scalable performance in cloud environments, allowing businesses to deploy GenAI solutions across large datasets without sacrificing speed or security.
Qx. Anything else you wish to add?
As enterprises increasingly adopt GenAI, the need for more secure, reliable and cost-efficient solutions has never been greater. RAG solutions, backed by the Progress Data Platform, which includes robust solutions like MarkLogic and Semaphore, offer enterprises the ability to fine-tune generative AI outputs by grounding them in high-quality, proprietary data. By leveraging these tools, businesses can optimize accuracy and improve data security so that generative AI deployments are sustainable in the long term. For organizations looking to explore the future of GenAI with confidence, the Progress Data Platform offers a compelling solution.
……………………………………………………..
Imran Chaudhri, Chief Architect genAI, AI, Healthcare & Life Sciences, Progress
At Progress MarkLogic, Imran focuses on enterprise quality genAI and NoSQL solutions for managing large diverse data integrations and analytics to the healthcare and life sciences enterprise. Imran co-founded Apixio with the vision of solving the clinical data overload problem and has been developing a HIPAA compliant clinical AI big data analytics platform.
Sponsored by Progress