Industry: Industrial software
Key Technologies/Platforms: Databricks, Unity Catalog, Databricks Medallion Architecture, Vector Search, Model Serving, RelevanceAI, Large Language Models (LLMs), Azure API Management (APIM), Salesforce, Retrieval-Augmented Generation (RAG)
About the Client
The client is a global industrial software provider delivering engineering, asset performance, and operations solutions across asset-intensive industries. With an extensive product portfolio and growing technical documentation, the organization wanted to make trusted information easier to find and enable more efficient AI-powered self-service.
Business Requirements
- Build a governed, retrieval-ready documentation foundation on Databricks.
- Keep vector indexes synchronized with changing content.
- Deliver an enterprise RAG solution grounded in approved documentation.
- Improve retrieval precision and provide source references.
- Support controlled deployments across DEV, TEST, and PROD.
- Securely integrate AI capabilities with downstream enterprise applications.
Business Challenges
- Fragmented Enterprise Documentation: Product and support information was distributed across multiple sources, making accurate answers difficult to find quickly.
- Keeping Knowledge Up to Date: Product documentation changed frequently, making it impractical to reprocess and re-embed the entire content library with every update.
- Building Trust in AI-Generated Answers: Users needed answers they could verify. Responses had to stay within the available documentation and clearly point back to the supporting source
- Finding the Right Content Across Products: Similar products often used the same technical terms, making it harder to consistently retrieve information for the right product and context.
- Connecting AI Securely to Enterprise Systems: The solution needed to work with existing enterprise applications while providing the security, access controls, and governance required for production use.
Impact and Business Benefits
- Users can find relevant product and support information faster, with references back to the source documentation.
- Grounded responses make it easier to verify answers and build confidence in the information provided.
- Only new or changed content needs to be re-embedded, avoiding the effort and compute required to process the full documentation set.
- More focused retrieval helps distinguish between products that share similar terminology.
- The AI-powered self-service knowledge platform provides a foundation that can be extended to other enterprise knowledge use cases.
Our Approach and Solutions
- Governed Databricks Architecture: Jade Global set up Bronze, Silver, and Gold layers on Databricks to manage documentation from ingestion through production use. Unity Catalog helped maintain clear ownership and governance across shared and project-specific data.
- RAG-Based Knowledge Retrieval: The team cleaned and chunked the documentation, generated embeddings, and created indexes in Databricks Vector Search. This Databricks RAG architecture gave the retrieval layer a consistent source of approved product content.
- Intelligent Grounded Retrieval: Using RelevanceAI, the retrieval agent checks the question before running a search. It can split complex questions into focused queries and ask for clarification when the request is unclear. Answers are limited to retrieved content and include links to the supporting documentation.
- Incremental Index Synchronization: The index is updated only where the documentation has changed. Each chunk has a content-derived ID, which makes it possible to retain unchanged chunks, re-embed revised ones, and remove content that has been deleted. This avoids running the entire documentation set through the embedding process again.