IN-PERSON: Apache Kafka Meetup Warsaw

Join us for an Apache Kafka® meetup on Thursday, June 5th from 6:00pm hosted by Snowflake!

Venue:

Snowflake

Marcina Kasprzaka 4, 01-211 Warszawa, Poland

Speaker One:

Artur Chyży, Senior Software Engineer, Snowflake

Title of Talk:

Unleashing AI on Real-Time Data Streams with OpenFlow and Snowflake Cortex

Abstract:

Discover the power of OpenFlow, a versatile processing platform designed to analyze real-time data streams. In this session, we'll explore how OpenFlow can ingest and process data originating from Apache Kafka, the leading distributed streaming platform. While network data will serve as a compelling example to showcase OpenFlow's capabilities, the principles and techniques discussed are applicable to a wide range of real-time data sources.

We'll delve into how OpenFlow can perform complex transformations, aggregations, and enrichments on incoming Kafka data streams. Furthermore, we'll unveil the exciting integration of OpenFlow with Snowflake Cortex, bringing the power of AI and ML directly to your real-time data. See firsthand how you can leverage Cortex within the OpenFlow processing pipeline to analyze network data (and other data types), identify anomalies, predict trends, and gain unprecedented insights. This session will demonstrate the potential of combining real-time data processing with cutting-edge AI for intelligent decision-making and automation.

Bio:

Artur Chyży is a highly experienced software engineer with over a decade dedicated to designing and implementing robust data-driven solutions. As a Senior Engineer at Snowflake, he focuses on helping organizations build and optimize their data architectures within the Data Cloud, solving complex challenges at scale. Prior to Snowflake, Artur honed his skills in the financial and logistics sectors, where he specialized in crafting high-performance systems for managing vast datasets. A keen problem-solver and technology enthusiast, he enjoys sharing his deep technical insights, often as a speaker at industry events.

Speaker Two:

Adam Warski, Chief R&D Officer, Softwaremill

Title of Talk:

Kafka queued up!

Abstract:

Kafka is widely known as a distributed, durable log, and is often used for its event streaming capabilities. Message queueing, while always possible, was a rather secondary feature. Until now!

Kafka 4.x introduces a long-requested improvement: a new mode of consuming messages via share groups. These enable implementation of queueing use-cases.

Let’s see Queues for Kafka, implemented as part of Kafka’s Improvement Process (KIP-932), in action; we'll talk about its architecture, features and limitations.

Bio:

I am one of the co-founders of SoftwareMill, where I primarily code using Java, Scala, and other interesting technologies. I am actively involved in open-source projects, such as Ox, Tapir, sttp, Quicklens, ElasticMQ, and others. I have also been a speaker at major conferences, including JavaOne, Devoxx, GeeCON and ScalaDays.

In addition to writing closed- and open-source software, I spend my free time exploring various (functional) programming-related subjects. Any ideas or insights I gain usually end up with a blog.

Speaker Three:

Gunnar Morling, Principal Technologist, Confluent

Title of Talk:

Ins and Outs of The Outbox Pattern

Abstract:

The outbox pattern is a common solution for implementing data flows between microservices. By channeling messages through an outbox table, it enables services to update their own local datastore and at the same time send out notifications to other services via data streaming platforms such as Apache Kafka, in a reliable and consistent way.

However, as with everything in IT, there’s no free lunch. How to handle backfills of outbox events, how to ensure idempotency for event consumers? Doesn’t the pattern cause the database to become a bottleneck? And what about alternatives such as “Listen-to-Yourself”, or the upcoming Kafka support for 2-phase commit transactions (KIP-939)?

It’s time to take another look at the outbox pattern! In this session I’ll start by bringing you up to speed on what the outbox pattern *is*, and then go on to discuss more details such as:

• Implementing the pattern safely and efficiently
• Its semantics, pros and cons
• Dealing with backfills
• Potential alternatives to the outbox pattern and the trade-offs they make