RAFAEL KANSY
Consultant, Architect, Data Engineer, DevOps Engineer and Lead Developer (Python/Java/Kotlin)
Profile
I am an experienced Data Engineer and Software Architect with over two decades of
expertise in delivering scalable, high-performance solutions. I have extensive experience in data
engineering and software architecture, with a deep understanding of working at a terabyte scale (TB)
using Python and Java. My proficiency extends to major cloud platforms.
I have built a wide range of sophisticated data architectures, from simple ETL processes to highly
complex, scalable, and reliable systems for AI and BI. I am versatile in working with
various database paradigms, including relational databases, column stores, NoSQL databases, graph
databases, time series databases, and vector databases.
I am seeking opportunities as a Data Engineer. I am also interested in roles focused on
Java/Kotlin microservices DevOps development and architecture. My high interest in financial
data, machine learning, and AI drives my passion for continuous learning and innovation.
Remote work is highly preferred, but I also enjoy working in the office, especially if it is near
Munich.
Let's connect and explore how my skills and experience can contribute to your next project!
Education
Education never ends
07/2007 - to dateIT Specialist - Application Development
Vocational school I in Landshut 09/2004 - 06/2007Bachelor of Computing
University of applied sciences in Landshut 09/2001 - 03/2004Skills
- Software Architecture
- Python
- Java
- Kotlin
- Spring
- Backend Development
- Data Engineering
- Big Data
- Cloud
- Kubernetes
- Quality
- Data Warehouse
Experience
AI & Data Engineering Consultant
From 10/2023 to date | Confidential customerEvaluation and integration of various Large Language Models (LLM) into the business processes of different customers.
Using Kafka, Redis, MongoDB, ElasticSearch, SQL, Java, Python, Lang Chain, huggingface.io, Chat-GPT and various other Large Language Models, such as LLaMA, Mistral, Mid-Journey, etc...
Data Engineering Consultant
From 10/2023 to date | Confidential customerEvaluation of realtime Streaming ingestion into an Oracle data warehouse with Confluent Kafka.
Evaluation of next generation data warehouse architecture with Snowflake or Databricks and Confluent Kafka.
Management consulting, prototyping technology, evaluation, E2E implementation and coaching of various stakeholders.
Data Engineering Consultant and Lead Developer
From 07/2019 to 07/2023 | Ippen Digital in MunichConception of a microservice-oriented real-time fast data / big data platform based on streaming with Kafka and Druid as well as Elasticsearch various SQL, Graph, Vector NoSQL databases.
Lead architect, Data / DevOps engineer, service and data owner for various components of the business intelligence team.
Responsible for a large part of Ippen Digital's system landscape, providing about 16.4 TB of retrievable, real-time data with a retention of 3 to 6 months and over 100 TB of offline data, as well as more than 1.5 billion events per day processed with Kafka Streams for about 320 million visits per month.
Consulting, conception, implementation and coaching of internal employees on the topics of architecture, Java, Python,Spring Boot, scaling, API evolution, taking into account high quality standards.
Team-wide design and implementation of business use cases as well as their documentation, testing and monitoring.
Data Engineering Consultant
From 02/2018 to 05/2019 | Allianz SE in MunichConception, development and extension of the risk assessment platform for Allianz SE in Munich.
Conceptual design of a microservice orientedBig Data platform based on streaming with Kafka and Flink.
Architectural and technological leadership of a microservice development team, as well as implementation of several microservices with Spring Boot, Spring, Hibernate, Kafka, REST and Postgres.
Integration of relevant data via various EAI components into a data warehouse.
Data Engineering Consultant
From 05/2017 to 01/2018 | Device Insight GmbH in MunichDevelopment, maintenance and extension of an IoT platform for a customer from the automotive sector at Device-Insight (IoT Leader Germany).
Design, implementation, testing and documentation of the cloud-native microservice architecture and of various software components. Building a highly scalable, highly available, microservice-oriented IoT platform using Spring, Spring Boot, Spring Cloud. Using Kafka as messaging infrastructure and Apache Cassandra as data warehouse solution, as well as Docker and Kubernetes.
Diagnosing and fixing SQL performance bottlenecks on a Postgres cluster and designing and implementing test coverage for an EAI component with several dozen Apache Camel routes and then refactoring the EAI component.
Certificates and Awards
Cloudera Developer Training for Spark and Hadoop
The course focused on advanced Apache Spark skills, covering Spark SQL for structured data querying and Spark Streaming for real-time data processing. The course ensured mastery of Apache Hadoop's role in data processing and enables effective use of Spark shell, Spark SQL, DataFrames, and Datasets for comprehensive data analysis and real-time processing.
ISAQB Certificated Software Architect (CPSA-F)
The CPSA-Foundation Level provided me the ability to design and document appropriate software architectures for well-described requirements and systems of moderate size within a team setting. It emphasizes ensuring and evaluating quality, as well as familiarity with necessary tools. This certification served as a valuable endorsement of internationally recognized standards in software architecture, offering significant advantages for professional growth and development.
Linux Foundation - LFS258 - Kubernetes Fundamentals
The Kubernetes Fundamentals course taught me how to use the container management platform used by companies like Google to manage their application infrastructure. I learned how to install and configure a production-grade Kubernetes cluster, from network configuration to upgrades to making deployments available via services. The course also distills key principles, such as pods, deployments and services, and will give you enough information so that you can start using Kubernetes on your own.
Scrum Team Training
Participants in the Scrum Team Training gained practical insights into Scrum roles, ceremonies, and artifacts, equipping them with the tools to successfully navigate complex project environments and drive continuous improvement.
Microsoft Certificated Professional (MCP)
The Microsoft Certified Professional (MCP) certification recognizes proficiency in Microsoft technologies and products. It validates expertise in software development, cloud computing, data management, and other key areas. Obtaining MCP certification demonstrates a high level of competence and is globally recognized, significantly enhancing career opportunities in the IT sector.
Cisco Certificated Network Associate (CCNA-1)
Networking and security are highly complex disciplines that should be handled by professionals and I don't count myself among them.
KMK - English language certificate II
English is the lingua franca in computer science. This certificate ensures that my English is sufficient.
Publications
Learning from the industry giants - Getting high availability right
- Link: https://www.cio.de/
Microservices-Architecture - Implementing Patterns in Spring
Enterprise goes Cloud - 4 part series on Cloud Architectures in Enterprises
Languages
- German
- English
- Polish
Personal Projects
Smart Portfolio Management
From 01/2023 to dateEvaluation of a dataset of about 5500 algorithms for automatic construction and balancing of portfolios.
I am part of a community where traders share their algorithms with each other. My goal is to test these algorithms all for quality and to find out under which conditions which algorithms work how well.
To achieve this, so far I use Python, Pandas, Jupyter, Numpy, SciPy, Matplotlib, Redis as well as other frameworks.