RJUG #19

Jacek Laskowski - (The Internals of) Stateful Stream Processing with Spark Structured Streaming

Let’s focus on state management in Spark Structured Streaming 2.4.4 for streaming aggregations and stream-stream joins. During this talk you will learn the streaming concepts that are particularly relevant for stateful stream processing in Structured Streaming, e.g. watermark, state stores and output modes that should help you create more sophisticated and efficient streaming queries in Apache Spark. In other words, you will learn how to use the stateful streaming API and understand the internals.

Jacek is a freelance IT consultant, software engineer and technical instructor specializing in Apache Spark, Apache Kafka and Kafka Streams (with Scala and sbt). Jacek offers software development and consultancy services with hands-on in-depth workshops and mentoring. Contact him at [masked] or DM on twitter @jaceklaskowski to discuss opportunities with Apache Spark, Apache Kafka and Kafka Streams. "A day with no git commit is a lost day."

Miejsce meetupu podamy już wkrótce ;)