Data Observability for Data Lakes: The Next Frontier of Data Engineering

Ever had your CEO look at a report and say the numbers look way off? Has a customer ever called out incorrect data in one of your product dashboards? If this sounds familiar, data reliability should be the cornerstone of your data engineering strategy.This talk will introduce the concept of “data downtime”—periods of time when data is partial, erroneous, missing or otherwise inaccurate—and how to eliminate it in your data lake, as well as the rest of your data ecosystem. Data downtime is costly for organizations, yet is often addressed ad hoc. This session will discuss why data downtime matters to building a better data lake and tactics best-in-class organizations use to address it—including org structure, culture and technology.