Architecting petabyte-scale analytics by scaling out Postgres on Azure and Citus

Architecting petabyte-scale analytics by scaling out Postgres on Azure with Citus.

A story about powering a 1.5 petabyte internal analytics application at Microsoft with 2816 cores and 18.7 TB of memory in the Citus cluster.

The internal RQV analytics dashboard at Microsoft helps the Windows team to assess the quality of upcoming Windows releases. The system tracks 20,000 diagnostic and quality metrics, digests data from 800 million Windows devices and currently supports over 6 million queries per day, with hundreds of concurrent users. The RQV analytics dashboard relies on Postgres—along with the Citus extension to Postgres to scale out horizontally—and is deployed on Microsoft Azure.

Abstract:

Alicja Kucharczyk, currently working at Microsoft as EMEA Global Black Belt OSS Data tech Specialist - is a PostgreSQL expert, both an experienced developer as well as administrator and PostgreSQL coach with strong practical knowledge of Linux and their mutual interactions. Particularly interested in performance optimization at different levels. She has consulted a lot of companies, mainly in Poland providing them with working solutions and supporting them in architecting, deploying and maintaining PostgreSQL. She is the founder and organizer of Warsaw PostgreSQL Users Group - the group gathering polish PostgreSQL users, speaker at local users groups and conferences and trying to advocate PostgreSQL and open source in many possible ways.

In the free time seen usually with two dogs 13 and 19 years old, benchmarking on 4U home server or writing games in python.