Invited Talk - Building Bitmap Indexes for Hybrid Workloads: A tale of UPBIT, CUBIT, and RABIT

Speaker: Manos Athanassoulis (Boston University)

Abstract: In this talk, I will describe our journey to elevate bitmap indexes from read-only indexes to update-friendly, highly concurrent, range-query friendly, secondary indexes that can be used as general-purpose secondary indexes for hybrid workloads. Our designs rely on the following principles: First, we employ a horizontal bitwise representation of updated bits, which enables efficient atomic updates without locking entire bitvectors. Second, we propose a lightweight snapshotting mechanism that allows queries to run on separate snapshots and provides a wait-free progress guarantee. Third, we consolidate updates in a latch-free manner, providing a strong progress guarantee. Fourth, we propose a new encoding scheme that extends the traditional equality encoding with group bitvectors that facilitate both short- and long-range queries without sacrificing update or space efficiency. Our evaluation when integrating with DuckDB query engine shows that our bitmap index variants (CUBIT and RABIT) can outperform state-of-the-art analytical engines by several orders of magnitude. Overall, our work breaks the assumption that bitmap indexes target only read-only queries with low cardinality, making them general-purpose contenders for hybrid workloads. For HTAP workloads with real-time updates, our designs achieve 2–11x performance improvement over the state-of-the-art approaches. We will conclude by discussing future directions about bitmap indexes and query processing with bitvectors.

Invited Talk - Hybrid Workloads Redefined by a Single DDL

Speaker: Nikos R. Katsipoulakis (Snowflake)

Abstract: Snowflake's hybrid tables represent a paradigm shift in hybrid workload management by architecturally unifying siloed transactional and analytical data within a singular platform. Hybrid tables facilitate both workload types via a declarative CREATE TABLE statement, prioritizing simplicity, operational ease (zero-configuration approach), and self-optimizing performance. This presentation will detail the fundamental modifications implemented across the Snowflake stack to seamlessly integrate and support hybrid workloads.

Invited Talk - Enabling Fast and Correct Approximate Query Processing in HTAP Systems

Speaker: Zhuoyue Zhao (University at Buffalo)

Abstract: In HTAP workloads, replacing exact query evaluation with sampling-based Approximate Query Processing (AQP) can improve freshness of results, lower latencies, and reduce resource consumption when approximation is appropriate. However, most existing AQP systems are built as standalone systems with no transaction support, and thus cannot readily support HTAP workloads. In this talk, I will discuss the challenges and our solutions for enabling AQP in HTAP systems, including concurrency-safe sampling index design and query cost reduction techniques for data with unknown statistics. We built these ideas into PostgreSQL, a traditional transactional system, which can support fast and correct concurrent transactional updates and approximate queries under snapshot isolation. In addition, I will discuss future challenges to extend our work to in-memory systems.

Invited Talk - The Paths to HTAP

Speaker: Ronen Grosman (Huawei Canada)

Abstract: The desire to combine multiple workloads together on the same database has been a longstanding desire for database designers. Naive systems usually start as a single database, but due to operational reasons typically grow into a set of OLTP systems feeding into OLAP systems. However if we could without performance or operational issues query a single copy of the data it would obviously be superior. However techniques such as columnar storage, scale out analytics, and de-normalization provide such significant performance benefits they cannot be ignored. Separating systems also enable cost savings through compression, and runtime efficiency, and disaggregation. They also provide significant availability benefits by removing unpredictable analytical workloads from the transactional system, and reducing the failure blast radius. In this talk we will discuss the various approaches that have historically been taken to solve these issues and build HTAP system, from automate “Zero-ETL” to more complex solutions such as columnar indexes and their advantages and disadvantages. We will then discuss emerging approaches and opportunities presented by emerging hardware and algorithms.