32. spotkanie Toruń JUG - Big Data and Asynchronous Communication

Panuje przekonanie, że Big Data jest drogie - wymaga sporo wiedzy, czasu i dużej inwestycji w infrastrukturę, dając przy tym niewielkie korzyści.

Podczas prezentacji pokażę alternatywne podejście do pracy z danymi w środowisku rozproszonym. Postaram się przekonać Was, że żeby wygodnie wyciągać wnioski z setek gigabajtów nie potrzeba całego ekosystemu usług czy drogich klastrów.

Będzie: tanio (na infrę nie wydamy więcej niż kilka złotych), łatwo (popracujemy w Sparku, chyba najwygodniejszym interfejsie dostępu do danych) i - mam nadzieję - ciekawie (wyciśniemy ile się da z publicznych zrzutów stackoverflow).

O prelegencie:

-----------------------------------------------------------------------

In distributed systems, synchronous communication (RPC-style) is tempting but can quickly get out of hand. Suddenly you need to think about retrying, fallbacks, circuit breakers, failover, and latency. Tomasz Nurkiewicz explains how all of this can be avoided by preferring asynchronous communication between services, pub-sub patterns, and event sourcing. Too often migrations from monolith to microservices architecture are unsuccessful. Simply replacing in-process method calls with RESTful interfaces doesn’t mean we have a distributed system consisting of loosely coupled independent services. Quite the opposite: our architecture is now a tangled web of interconnected, slow, chatty, and unreliable components. Dozens of patterns were introduced to ease the pain, such as circuit breakers, scaling services horizontally, and load balancing — all of this to prevent cascading failures and increased latencies. We can achieve fast, loosely coupled, independent services only if we apply the dependency inversion principle at the architecture level. Moving to asynchronous communication via message passing and pub-sub patterns can prevent temporal coupling. Such coupling requires two systems to exist and work reliably at the same time in order to communicate. This is the biggest challenge in distributed systems that increases complexity, latency, and the possibility of failure.

Tomasz demonstrates how pushing changes asynchronously between systems can improve fault tolerance and make systems more reliable, faster, and more independent, focusing on publishing (pushing) changes and rebuilding state on the client side as opposed to pulling data when needed. From this point jumping to event sourcing is quite natural. Along the way, Tomasz covers the drawbacks and challenges of this architectural style.

O prelegencie: