Snowflake to Databricks Migration Case Study - Jade Global

Search

Client name: A leading global E-learning and skills marketplace

Industry: E-Learning / EdTech

Key Technologies/Platforms: Databricks Data Intelligence Platform, Snowflake, DBT, Tableau, GitHub

About the Client

The client stands out as a leading global online learning and skills marketplace, catering to millions of learners and organizations around the globe. Their business encompasses both consumer learning and enterprise SaaS, which creates a strong demand for scalable, reliable, and accessible financial reporting across various business segments.

Business Requirements

  • Modernise the data architecture: Migrate financial reporting from Snowflake to Databricks using a scalable Bronze, Silver, and Gold Medallion Architecture.
  • Enable dual reporting: Support detailed transaction-level reporting in Databricks while still providing aggregated executive reports through Tableau.
  • Boost cost and performance efficiency: Optimize compute and data processing to cut down on platform costs and enhance reporting performance.
  • Governance and reproducibility: Implement GitHub-based version control for our data models, transformations, and pipeline code.

Business Challenges

  • In financial and accounting reporting, was stuck in a loop with repetitive CTE-based SQL queries. This not only led to duplicated logic but also made maintenance a real headache and brought about performance issues.
  • On top of that, data architecture wasn’t as modular as it should be. It didn’t have a clear distinction between raw, transformed, and reporting-ready data, which raised the stakes for data quality and led to unnecessary rework.
  • As data volumes grew and reporting became more complex, resulting in some serious scalability and cost challenges. The increased compute costs were hard to ignore, and performance limitations started to creep in.
  • Lack of clear separation between transaction-level and executive-level reports, which made it tough for users to find the right financial information they needed.

Our Approach and Solutions

Modular Medallion Architecture to enhance financial and accounting reporting setup. This involved migrating from Snowflake to Databricks, structured around a three-layer architecture:

  • Bronze Layer: Stage our tables, pulling in all the necessary data from raw sources.
  • Silver Layer: Core schema tables and views that apply standardized transformations.
  • Gold Layer: This layer features DBT models and views that deliver business-ready data for reporting and analytics.

Dual Reporting: The Gold Layer was crafted to support two distinct reporting paths. Detailed transaction-level models and views are easily queryable in Databricks, while aggregated models are designed to feed into Tableau for executive dashboards and high-level financial reporting.

DBT-Based Data Transformations: DBT has taken over from repetitive CTE-based query patterns, allowing to create modular, reusable, testable, and well-documented data models. This shift has significantly improved maintainability and made future enhancements easy.

Impact and Business Benefits

  • 40%+ in cost savings by migrating to Databricks and fine-tuning compute and storage strategies.
  • 80% boost in processing performance due to modular Medallion Architecture and transformations driven by DBT.
  • The data foundation improved scalability, allowing us to handle growing data volumes and more complex reporting needs. Easy maintenance and swapping out repetitive CTE-based logic for reusable, modular DBT models. Business users can now quickly access detailed transaction-level data, while executives enjoy aggregated insights through Tableau, leading to faster financial insights.
  • GitHub-based version control enhanced governance and reproducibility, making collaboration, traceability, and auditability across data engineering workflows much smoother.

Download Case Study

Check out our Databricks services