RAFAEL KANSY

Consultant, Architect, Data Engineer, DevOps Engineer and Lead Developer (Python/Java/Kotlin)

80939 Munich, Bavaria - Germany

Profile

I am an experienced Data Engineer and Software Architect with over two decades of expertise in delivering scalable, high-performance solutions. I have extensive experience in data engineering and software architecture, with a deep understanding of working at a terabyte scale (TB) using Python and Java. My proficiency extends to major cloud platforms.
I have built a wide range of sophisticated data architectures, from simple ETL processes to highly complex, scalable, and reliable systems for AI and BI. I am versatile in working with various database paradigms, including relational databases, column stores, NoSQL databases, graph databases, time series databases, and vector databases.
I am seeking opportunities as a Data Engineer. I am also interested in roles focused on Java/Kotlin microservices DevOps development and architecture. My high interest in financial data, machine learning, and AI drives my passion for continuous learning and innovation.
Remote work is highly preferred, but I also enjoy working in the office, especially if it is near Munich. Let's connect and explore how my skills and experience can contribute to your next project!

Education

Education never ends

07/2007 - to date

IT Specialist - Application Development

Vocational school I in Landshut 09/2004 - 06/2007

Bachelor of Computing

University of applied sciences in Landshut 09/2001 - 03/2004

Skills

  • Python
  • Java
  • Kotlin
  • Spring
  • Data Mesh
  • Data Lakehouse
  • Data Warehouse
  • Databricks
  • Apache Kafka
  • Apache Flink
  • Software Architecture
  • Backend Development
  • Data Engineering
  • Big Data
  • AWS
  • Azure
  • Google Cloud
  • Kubernetes
  • OpenShift
  • Quality

Experience

Software Architect and Data Engineer

From 01/2024 to 06/2026 | Versicherungskammer Bayern in Munich

Transition from an ETL architecture to an Event Sourcing architecture with Spring Boot, Confluent Kafka, Apache Spark and MongoDB, deployed on OpenShift / Kubernetes.

Conception and implementation of architecture blueprints and PoCs on Spring Boot and Confluent Kafka.

Definition and enforcement of Data Contracts across Kafka event streams (JSON Schemas and Avro via Schema Registry) as the foundation for cross-domain Data Mesh interfaces between producing and consuming teams.

Training and support of internal employees, implementation of various use cases, code reviews, improvement of test coverage and data lineage across distributed systems.

Software Architect and Data Engineer

From 09/2025 to 03/2026 | Knauf KG in Iphofen

Design and implementation of a Medallion Data Architecture as a Data Lakehouse on Azure Databricks (managed Kubernetes clusters on AKS), including the end-to-end migration of several business-critical use cases from SAP into Databricks.

Transformation and streaming of data with Apache Kafka and Apache Flink. Hands-on work with Unity Catalog (governance, lineage; evaluated against Collibra as the enterprise catalog alternative), Delta Lake (bronze/silver/gold layers), PySpark and Structured Streaming (Kafka → Databricks).

Deployment and environment promotion via Databricks Asset Bundles (DABs), analytics transformations modelled with dbt, contract-first approach with Data Contracts between SAP source systems and the Databricks target, CI/CD with GitHub Actions driving DABs rollouts.

IoT Telemetry Architecture Audit

From 05/2025 to 07/2025 | Grob Werke GmbH & Co. KG in Mindelheim

Technical end-to-end audit of a factory-floor IoT telemetry pipeline: high-volume MQTT ingress from production robots and machines via HiveMQ, stream processing with Apache Kafka Streams, persistence in TimescaleDB hypertables.

The platform combined data-in-motion (streaming sensor events) with data-at-rest (historical and reference datasets in TimescaleDB), joined in-stream for enrichment and correlation.

Recommended improvements across MQTT topic design and QoS strategy, Kafka Streams topology and state stores, schema evolution and Data Contracts between producer devices and downstream consumers, TimescaleDB hypertable partitioning, continuous aggregates and compression policies.

Data Engineer and Management Consultant

From 10/2023 to 11/2024 | ING DiBa in Frankfurt

Transformation of a globally distributed, multi-terabyte SQL-based Oracle data warehouse into a real-time streaming Data Lakehouse with Apache Kafka and Apache Flink, deployed on OpenShift / Kubernetes.

Development and verification of a migration concept from SQL-based systems-of-record workflows to real-time systems-of-engagement, including Data Contracts for cross-team schema governance, Collibra as the enterprise data catalog and business glossary across producing and consuming domains, and full data lineage as the foundation for a federated Data Mesh.

Evaluation of Snowflake, Databricks and Apache Spark as target platforms across AWS, Azure and Google Cloud.

Data and AI Engineer

From 01/2024 to 06/2024 | Ippen Digital in Munich

Support for the evaluation of several Large Language Models (ChatGPT, LLaMA, Anthropic, Mixtral) and consulting on the integration of AI workflows into the existing data architecture.

Development with Python, HuggingFace, LangChain, Apache Kafka and various cloud technologies, deployed on Kubernetes clusters on AWS.

Integration of Agentic AI capabilities into the customer's existing data and editorial workflows.

Lead Data Engineer / Architect

From 07/2019 to 07/2023 | Ippen Digital in Munich

Conception of a microservice-oriented real-time fast data / big data platform based on streaming with Kafka and Druid as well as Elasticsearch various SQL, Graph, Vector NoSQL databases, deployed on Kubernetes clusters on AWS.

Lead architect, Data / DevOps engineer, service and data owner for various components of the business intelligence team.

Responsible for a large part of Ippen Digital's system landscape, providing about 16.4 TB of retrievable, real-time data with a retention of 3 to 6 months and over 100 TB of offline data, as well as more than 1.5 billion events per day processed with Kafka Streams for about 320 million visits per month.

Consulting, conception, implementation and coaching of internal employees on the topics of architecture, Java, Python, Spring Boot, scaling, API evolution, taking into account high quality standards.

Team-wide design and implementation of business use cases as well as their documentation, testing and monitoring.

Data Engineer and Lead Developer

From 02/2018 to 05/2019 | Allianz SE in Munich

Conception, development and extension of the risk assessment platform for Allianz SE in Munich.

Conceptual design of a microservice oriented Big Data platform based on streaming with Kafka and Flink.

Architectural and technological leadership of a microservice development team, as well as implementation of several microservices with Spring Boot, Spring, Hibernate, Kafka, REST and Postgres.

Setup of CI/CD with Docker, Kubernetes and Azure, including an early-adopter implementation of Azure Kubernetes Service (AKS) in close collaboration with Microsoft engineering.

Integration of relevant data via various EAI components into a data warehouse.

Data Engineer

From 05/2017 to 01/2018 | Device Insight GmbH in Munich

Development, maintenance and extension of an IoT platform for a customer from the automotive sector at Device-Insight (IoT Leader Germany).

Design, implementation, testing and documentation of the cloud-native microservice architecture and of various software components. Building a highly scalable, highly available, microservice-oriented IoT platform using Spring, Spring Boot, Spring Cloud. Using Kafka as messaging infrastructure and Apache Cassandra as data warehouse solution, as well as Docker and Kubernetes, deployed on Azure.

Diagnosing and fixing SQL performance bottlenecks on a Postgres cluster and designing and implementing test coverage for an EAI component with several dozen Apache Camel routes and then refactoring the EAI component.

Senior IT Consultant

From 04/2013 to 04/2017 | msg systems AG in Munich

Research, development and publications on Java development with Java EE and the Spring Framework, as well as prototyping microservices and architecture blueprints.

As early adopters of container technologies, evaluation and prototyping of PaaS and container platforms including Linux Containers, Docker, Docker Swarm, Kubernetes, OpenShift and Cloud Foundry on AWS, Azure and Google Cloud, including blueprint migrations from VMware to containerized deployments.

Consulting for many clients from various industries.

Java Software Engineer

From 04/2011 to 03/2013 | 1&1 AG in Munich

Jointly responsible for the portals web.de (14.02M unique users/month), gmx.net (10.93M unique users/month) and home.1und1.de (3.86M unique users/month).

Conception, design, development, testing, review and deployment of new features in an agile, high-load, service-oriented landscape.

Java Software Engineer

From 05/2010 to 03/2011 | ProSiebenSat.1 Media AG / lokalisten media GmbH in Munich

Conception, design, analysis, implementation and quality assurance of new features for the social network lokalisten.de (3.6M unique users/month).

Backend with Hibernate, Spring, EhCache and Apache Solr; frontend with Struts, Tiles, Spring MVC and JSP; QA with Selenium, JMeter and JUnit.

Java EE Software Engineer

From 01/2009 to 04/2010 | Cartago Software GmbH in Landshut

Design, analysis, implementation and quality assurance of the server stack. Backend with Hibernate / Spring; frontend with Google Web Toolkit; QA with JUnit and DBUnit.

Deployment, rollout and monitoring at customer sites; build processes and agile methods.

Java Software Engineer

From 07/2007 to 12/2008 | CSP Software GmbH in Landau

Conception, analysis, implementation and quality assurance of new features for the CSP product portfolio, and development of various backends including SAP.

Backend with Hibernate / Spring; frontend with JSF, JSP and Servlets; QA with JUnit.

Apprenticeship — IT Specialist (Application Development)

From 02/2005 to 06/2007 | Adito Software GmbH in Geisenhausen

Implementing features and backends for the Adito product portfolio, including MS Exchange and Lotus Domino; custom modules in JavaScript.

Graduation 06/2007 at Berufsschule I Landshut: Fachinformatiker — Anwendungsentwicklung (IHK).

Apprenticeship — IT Specialist (Application Development)

From 09/2004 to 01/2005 | Compus Computer GmbH in Munich

Design, analysis and implementation of various workflows for the Compus Computer product portfolio using Microsoft AXAPTA / Navision and .NET technologies.

Self-employed project

From 08/2003 to 10/2003 | Inteco GmbH in Landshut

Short freelance engagement alongside Computer Science studies at HAW Landshut.

Studies in Computer Science

From 09/2001 to 03/2004 | University of Applied Sciences Landshut

Computer Science at HAW Landshut, overlapping with the Inteco internship and the subsequent self-employed project.

Study-related internship

From 09/2002 to 01/2003 | Inteco GmbH in Landshut

Focus on Java programming, database design and project management.

Internship — IT systems electronics technician

From 06/2001 to 08/2001 | A&P Office Systems OHG in Landshut

Training as IT systems electronics technician.

Gymnasium

From 1997 to 2001 | Krappitz

Abitur grade 2.5.

Trainings I Design & Deliver

2025 | New Elements (IT-Schulungen.com)

Event-Driven Architecture and Stream Processing: Concepts, Systems & Patterns

2025 | New Elements (IT-Schulungen.com)

Apache Kafka in Practice – Architecture, APIs, High Availability & Monitoring

2025 | New Elements (IT-Schulungen.com)

Event-Driven Architectures with Kafka, Kinesis, Flink & Spark Streaming

2025 | New Elements (IT-Schulungen.com)

Realtime BI & Event-Driven Analytics: Kafka, Flink, Streaming Dashboards

2024 to date | Versicherungskammer Bayern

Internal training on Event Sourcing, Spring Boot and Kafka

2019 to 2023 | Ippen Digital

Internal training on Apache Kafka, Spring Boot and Software Architecture

2013 to 2017 | msg systems AG

Trainings on Spring Boot, Spring Cloud, Pivotal CloudFoundry, Docker and Apache Maven

Certifications & Awards

Cloudera Developer Training for Spark and Hadoop

The course focused on advanced Apache Spark skills, covering Spark SQL for structured data querying and Spark Streaming for real-time data processing. The course ensured mastery of Apache Hadoop's role in data processing and enables effective use of Spark shell, Spark SQL, DataFrames, and Datasets for comprehensive data analysis and real-time processing.

ISAQB Certificated Software Architect (CPSA-F)

The CPSA-Foundation Level provided me the ability to design and document appropriate software architectures for well-described requirements and systems of moderate size within a team setting. It emphasizes ensuring and evaluating quality, as well as familiarity with necessary tools. This certification served as a valuable endorsement of internationally recognized standards in software architecture, offering significant advantages for professional growth and development.

Linux Foundation - LFS258 - Kubernetes Fundamentals

The Kubernetes Fundamentals course taught me how to use the container management platform used by companies like Google to manage their application infrastructure. I learned how to install and configure a production-grade Kubernetes cluster, from network configuration to upgrades to making deployments available via services. The course also distills key principles, such as pods, deployments and services, and will give you enough information so that you can start using Kubernetes on your own.

Scrum Team Training

Participants in the Scrum Team Training gained practical insights into Scrum roles, ceremonies, and artifacts, equipping them with the tools to successfully navigate complex project environments and drive continuous improvement.

Microsoft Certificated Professional (MCP)

The Microsoft Certified Professional (MCP) certification recognizes proficiency in Microsoft technologies and products. It validates expertise in software development, cloud computing, data management, and other key areas. Obtaining MCP certification demonstrates a high level of competence and is globally recognized, significantly enhancing career opportunities in the IT sector.

Cisco Certificated Network Associate (CCNA-1)

Networking and security are highly complex disciplines that should be handled by professionals and I don't count myself among them.

KMK - English language certificate II

English is the lingua franca in computer science. This certificate ensures that my English is sufficient.

Publications

CIO Online

Learning from the industry giants - Getting high availability right

Entwickler.press shortcuts

Microservices-Architecture - Implementing Patterns in Spring

Java Magazine

Enterprise goes Cloud - 4 part series on Cloud Architectures in Enterprises

Languages

  • German
  • English
  • Polish

Personal Projects

Smart Portfolio Management

From 01/2022 to date

Evaluation of a dataset of about 10,000 algorithms for automatic construction and rebalancing of portfolios.

Part of a community where traders share their algorithms with each other. Goal: test all algorithms for quality and determine under which conditions each algorithm performs best, in order to operate a self-built Robo Advisor.

Using Python, Pandas, Jupyter, Numpy, SciPy, Scikit, Matplotlib, Redis, TimeScaleDB, dbt and Spark as well as other frameworks, with Kubernetes as the production runtime.

Interests

Travel
Gourmet
Read
Finance