On April 1st, 2021, Google Open Source hosted a live event in which they presented about distributed data processing with Apache Beam and highlighted building data pipelines with CDC and Debezium along with multiple sessions presented by the Beam community. One of these community sessions was hosted by Akvelon, in which our Delivery Program Manager Alex Kosolapov presented Dataflow Templates.
We would like to thank all of Akvelon’s Google Cloud Practice team who made Akvelon’s Apache Beam and Google Cloud Dataflow Templates open source contributions possible – with special thank you to Artur Khanin, Ilya Kozyrev, and Mark Boyes!
During Akvelon’s session, Alex Kosolapov provided an overview of Dataflow templates and an example use case scenario. He also discussed creating and executing Dataflow templates and how to get started contributing to open source Dataflow templates.
Dataflow Use Case
Alex shared an example use case where Dataflow was a great solution for data transformation and a custom template helped to integrate external data protection service into the pipeline.
In this use case, one of Akvelon’s customers wanted to ingest large volumes of data, including sensitive data, and a third party data protection service was used for tokenization, converting data elements like credit card numbers into non-sensitive token representations that could be stored safely. Watch Akvelon’s session to learn how Akvelon leveraged Dataflow to help our client achieve this!
The Beam Day on Google Open Source Live event was a great success, with over 3,500 attendees on Meetup.com!