“The world of data is changing. I feel it in the water, I feel it in the earth, I smell it in the air…”
It might sound like something from The Lord of the Rings, but it captures the sense of transformation happening in the data world today. Data is everywhere, but for years, companies have struggled with data tools scattered across different platforms. Some teams use Snowflake, others use Databricks, and others might prefer Redshift or AWS Athena. At dbt Coalesce Conference 2024 in Las Vegas at Resorts World with 1,800+ data practitioners, team leaders, and executives attendees, dbt Labs revealed its solution: One dbt, a unified platform designed to bring together data teams, no matter which cloud or tools they use.
Unlike the “One Ring” from Lord of the Rings, which aimed to rule all, One dbt is designed to empower everyone—data engineers, analysts, and decision-makers alike. Instead of controlling data, this platform helps businesses to work smarter, faster, and more collaboratively.
So grab your coffee (or whatever gets you going), and let’s walk through the major announcements from Coalesce 2024, where dbt Labs introduced some exciting updates that are set to change the world of data analytics. Data-Sleek was one of the attendees, and we’re excited to share these highlights with you!
“One dbt” Vision: One Platform to Empower Them All
Just like in The Lord of the Rings, where each group—Elves, Dwarves, and Men—had their own strengths and roles, the world of data analytics also has many players. You have engineers building the data pipelines, analysts digging through data for insights, and decision-makers who depend on clear reports to make their next big move. All of these teams need to work together to solve the common challenge of disconnected data systems.
One dbt is dbt Labs’ solution to bring these teams together under one system. Unlike Sauron’s “One Ring,” which aimed to control, dbt’s platform is designed to empower. It offers clarity and flexibility, helping companies avoid the need to choose between different cloud platforms or workflows. With dbt’s One Platform, businesses can use the tools that suit them best, all under a unified analytics development lifecycle (ADLC).
According to Tristan Handy, Founder & CEO of dbt Labs, the vision behind One dbt is a direct response to three trends currently shaping the data world:
- Platform Flexibility: As businesses increasingly adopt multi-cloud strategies using tools like AWS Athena, Snowflake, Databricks, and Redshift, they need seamless collaboration across platforms. dbt Labs ensures that data teams can work smoothly in these varied environments.
- Collaborative Analytics: The rise of self-service BI tools and generative AI has allowed more business users to engage with data workflows. dbt Cloud, a powerful data control plane, makes these tools accessible even to non-technical users. This helps ensure that data insights are available to a broader range of collaborators.
- Trust Imperative: Poor data quality and fragmented ownership can cause mistrust between teams. To address this, dbt Labs provides transparency and control over the entire analytics development lifecycle (ADLC), ensuring that data teams can rely on their data.
Major Announcements from Coalesce 2024
dbt Labs team members introduced several new features and tools at Coalesce 2024, all aimed at breaking down the walls between data platforms and enabling data teams to work smarter, faster, and more efficiently. Here are the highlights:
1. dbt Mesh: Uniting Data Teams Across Platforms
Imagine a world where data teams—much like the races of Middle Earth—are no longer divided by their technology stacks. dbt Mesh makes this possible, enabling cross-platform collaboration between various data estates. Whether your company uses Snowflake, Databricks, Redshift, or AWS Athena, dbt Mesh provides a centralized, governed approach to working across different data platforms. Teams can reference and reuse data assets seamlessly, breaking down the silos that previously kept them apart.
2. Iceberg Table Support: Supercharging Query Performance
Large-scale data lakes often resemble the treacherous plains of Rohan—challenging to navigate but essential to cross. dbt Cloud now supports the Apache Iceberg table format to smooth out these rough edges. This integration enhances query performance and supports schema evolution, allowing data teams to manage large datasets efficiently. This support significantly improves performance for large enterprises handling the largest datasets, making data lakes much easier to work with.
3. New Visual Editing Experience: Data for Everyone
Like the diverse members of the Fellowship, where everyone had a role to play, dbt’s Visual Editing Experience opens up data workflows to non-technical users. This intuitive drag-and-drop interface (currently in beta) is designed for business users who may not have SQL skills but still need to interact with data. Now, they can safely build and manage data models while adhering to governance standards.
For data engineers and analysts, the Visual Editing Experience offers a powerful visual representation of complex models. This helps teams explore, optimize, and manage their data pipelines more intuitively. It’s all about making data accessible to everyone, from technical experts to domain specialists.
4. dbt Copilot: AI-Powered Productivity Booster
dbt Copilot is dbt Cloud’s answer to repetitive data tasks. This embedded AI engine automates everything from test generation to documentation, significantly improving productivity and data quality across the board. With Copilot, teams can take advantage of AI for semantic model creation and more. It even integrates with Snowflake and OpenAI, allowing businesses to use custom API keys for even greater control.
This is the future of data analytics—where AI and human intelligence work hand in hand to deliver better, faster results.
5. Advanced Continuous Integration (CI) with Compare Changes
Every data team needs a well-forged sword to protect their workflows, and dbt’s Advanced Continuous Integration (CI) with Compare Changes is just that. This feature allows teams to track exactly what has changed in their data models, providing transparency and ensuring no unexpected errors occur during deployment.
With this feature, teams can compare changes within Git providers, ensuring they’re always battle-ready and keeping data quality at the forefront of your analytics.
6. Auto-Exposures for Tableau: Keeping Data Fresh
To ensure that decision-makers always have access to the most up-to-date data, dbt Cloud now supports Auto-Exposures for Tableau, with support for Power BI coming soon. This feature automatically updates dashboards with the latest data, ensuring that stakeholders always have the freshest information to work with.
If you’re a technical user or a business leader, this feature guarantees that the data driving your decisions is accurate and up-to-date.
7. Data Health Tiles: Trust in Every Dashboard
Much like Gondor’s watchtowers guarded against unseen threats, Data Health Tiles monitor the quality and freshness of your data. Embedded directly within dashboards, these tiles provide an at-a-glance view of how reliable your data is, ensuring that business users can trust the information driving their decisions.
These health signals contribute to dbt’s overall commitment to improving data health and transparency across organizations.
New Integrations and Open Source Contributions
dbt Labs didn’t stop there. At Coalesce 2024, they also announced several new integrations, including AWS Athena (now GA) and Teradata (currently in preview). These integrations are part of dbt’s commitment to making dbt Cloud compatible with more platforms, ensuring that no matter what tools you use, you can integrate them into One dbt.
On the open-source side, dbt Core 1.9 introduced a new microbatch strategy for handling large datasets, streamlining how teams manage large-scale data without hitting bottlenecks. These improvements align with dbt’s mission to evolve continually and push forward the best practices for the entire data industry.
Why Is This Important for the Future?
This is important because it represents the future of how companies will handle and use data. Companies need to work faster and smarter, and dbt’s platform helps make that happen by bringing different tools and teams together.
Instead of dealing with disconnected systems, everyone can now work on the same page, making it easier to get insights, collaborate, and make better decisions. It’s not just about controlling data—it’s about empowering everyone to use it in the best way possible. This shift is what will drive success for businesses as they face more complex challenges in the future.
The Future of Data Collaboration Has Arrived
If you thought the Fellowship of the Ring had a tough job, imagine managing a data team without dbt’s latest tools. But with dbt Labs’ One dbt platform unveiled in Coalesce 2024, data collaboration has become smooth, powerful, and even enjoyable. The familiar dbt workflow has been enhanced with tools like Cross-Platform dbt Mesh to break down platform barriers and the Visual Editing Experience to make sure even non-technical users can contribute. It’s a seamless process that boosts your team’s ROI and allows you to work faster and smarter.
The takeaway from Coalesce 2024 is simple: it’s time for everyone to work together, no matter what platform or skill level. The tools are ready, the vision is clear, and the future of data is shining bright. Now’s the time to unlock the full potential of your data team and start your journey toward faster, better, and more reliable analytics.
