TDWI Schweiz 2018

05 - 06 November 2018 | Swissôtel Zürich

Session details

Talk: Di 2.2
Date: Tue, 04.02.2020
Time: 14:00 - 14:45
cart

The Changing Face Of ETL: Event-Driven Architectures For Data Engineers

Time: 14:00 - 14:45
Talk: Di 2.2

 

In this talk, we’ll discuss the concepts of events, their relevance to data engineers and their ability to unify architectures in a powerful way. We’ll see how stream processing makes sense in both a microservices and ETL environment, and why analytics, data integration and ETL fit naturally into a streaming world. The talk will conclude with a hands-on demonstration of these concepts in practice using Apache Kafka and commentary on the design choices made.

Target Audience: Architects, Data Engineers, Software Engineers, Analysts
Prerequisites: None
Level: Introductory

Extended Abstract
Data integration in architectures built on static, update-in-place datastores inevitably end up with pathologically high degrees of coupling and poor scalability. This has been the standard practice for decades, as we attempt to build data pipelines on top of databases that do a poor job modelling the fundamental objects that drive our businesses and systems: events.
Events carry both notification and state, and form a powerful primitive on which to build systems for developers and data engineers alike. Developers benefit from the asynchronous communication that events enable between services, and data engineers benefit from the integration capabilities. Everyone gains from using the standards-based, scalable and resilient streaming platform.
In this talk, we’ll discuss the concepts of events, their relevance to both software engineers and data engineers and their ability to unify architectures in a powerful way. We’ll see how stream processing makes sense in both a microservices and ETL environment, and why analytics, data integration and ETL fit naturally into a streaming world. The talk will conclude with a hands-on demonstration of these concepts in practice using Apache Kafka and commentary on the design choices made.
Join this talk to learn:

  • The power of events and unbounded data
  • Streaming is not just for real-time applications — it’s for everyone
  • Where a streaming platform fits in an analytic architecture
  • How event-driven architectures can enable greater scalability and flexibility of systems both now and in the future

 

Sponsoren der TDWI Schweiz

  • IT Logix AG Platinum Sponsor
  • NTT DATA Platinum Sponsor
  • Trivadis AG Platinum Sponsor
  • b.telligent Schweiz GmbH Gold Sponsor
  • BARC GmbH Gold Sponsor
  • BiG EVAL Gold Sponsor
  • biGenius Gold Sponsor
  • CIMACON GmbH Gold Sponsor
  • Digital Improvements GmbH Gold Sponsor
  • itelligence Gold Sponsor
  • MicroStrategy Gold Sponsor
  • NextStride AG Gold Sponsor
  • pmOne AG Gold Sponsor
  • Q_PERIOR Gold Sponsor
  • saracus consulting AG Gold Sponsor
  • Syncwork AG Gold Sponsor
  • ThoughtSpot Gold Sponsor
  • TIMETOACT Software & Consulting GmbH Gold Sponsor
  • WhereScape Europe Gold Sponsor