Iraklis Psaroudakis is working remotely from Greece as a Principal Software Engineer at Elastic, focusing on the distributed scalability of Elasticsearch. Previously, he was a Principal Member of Technical Staff at Oracle Labs in Zürich, Switzerland, concentrating on parallel and distributed programming, and graph-based analytical & machine learning workloads, especially in financial crime & compliance applications. Prior to Oracle, he completed his Ph.D. at the DIAS lab of EPFL in Lausanne, focusing on scaling up highly concurrent analytical database workloads on multi-socket multi-core servers through (a) sharing data and work across concurrent queries, and (b) adaptive NUMA-aware data placement and task scheduling. During his Ph.D., he cooperated with the SAP HANA database team. Before his Ph.D., he completed his studies in Electrical & Computer Engineering at NTUA in Athens.
Modern-day observability & security demand fast-paced searches in ever-increasing data volumes. Elasticsearch (ES), at the core of the Elastic Stack, is a leading distributed AI search and analytics engine. ES has been stateful so far, using primarily the disk to store data. In this presentation, we show the technical design of the new ES Serverless (ES3) mode, that achieves unparalleled scalability without any administration burden. We focus on the basic premise of decoupling compute from storage, and offloading data to an affordable highly available object store (e.g., S3), while supporting the same APIs and read-after-write semantics. Specifically, we show why and how we simplify node tiers to just two: indexing and search. We describe how we use batched compound commits to wrap Lucene files onto the object store, how the translog is buffered on the object store for recovery, and how refresh and search semantics operate.
Searching for speaker images...