How Query Engines Work

February 27, 2020

Over the past decade I’ve spent a fair bit of time either building query engines or building integrations with query engines so I decided to write an introductory book on the subject.

The book walks through every step of building a SQL query engine in Kotlin with full source code available in a companion github repository. Most of the book is programming-language agnostic and Kotlin was chosen for the code examples due to its conciseness and readability. The concepts should be easily translatable to other programming languages.

Table of contents:

  • Introduction
  • Choosing a Type System
  • Apache Arrow Overview
  • Data Sources
  • Logical Plan
  • Physical Plan
  • Execution
  • Optimization
  • Adding SQL Support

Future revisions will cover more advanced topics including:

  • Parallel Query Execution
  • Distributed Query Execution
  • Compiling Query Plans
  • Supporting User-defined Types
  • Supporting User-defined Code

The book will be available for purchase soon from leanpub.com. You can sign up to receive a notification when it is published.