The Project

In recent years, quirion has emerged as one of the leading digital investment advisors in Germany. quirion’s ever-increasing customer base and managed funds attest to its financial service reputation. Keeping up with customer demand while maintaining high standards of security and customer satisfaction posed a great challenge for software engineers and analysts. The solution was a complete re-architecture of the legacy system onto cloud (AWS) technology, which opened an opportunity for enhancing quirion’s business analysis processes.

As a certified AWS partner with an established knowledge of the financial services industry, kreuzwerker worked together with quirion to deliver an Elasticsearch business intelligence solution that was tailored to their unique needs.

The Problem

Business analytics at quirion was a sophisticated but labour-intensive activity that involved multiple teams and heterogeneous tools. Different analysts established custom data pipelines, all sharing a common source bottleneck. The need for a more cohesive and streamlined solution for business analysis became apparent as the customer base continued to grow and new financial services rolled out. That’s where kreuzwerker came in.

We joined forces with quirion in the most intense phase of the migration of their digital services to AWS. To speed up the process, quirion engineers relied on AWS Amplify, a development framework that aids provision of AWS Cloud services based on the desired application features, such as data persistence, authentication, and even analytics. On the downsides of Amplify, you often trade time-to-market with inefficient resource utilisation and design restrictions. Fine-tuning the resulting setup is therefore paramount, especially from a data security and costs standpoint.

Given Amplify’s native support and our long experience with Elastic, we agreed with quirion that AWS Elasticsearch would be the unifying solution for business analytics. More challenging was the transformation of the business data for financial and marketing analysis, and the creation of usable and actionable dashboards for all stakeholders.

The Solution

First, we reviewed the configuration of the AWS Elasticsearch domains provisioned by Amplify. Although Amplify ships with sensible defaults, we significantly optimised costs and resilience of the domains by selecting the most appropriate number and type of instances for each deployed environment. Most importantly, we enhanced security by enabling HTTPS for all traffic to the domains, defining stringent access policies to the data and setting up Amazon Cognito Authentication.

Then, we focused on the data. Amplify mirrors data from the primary data source to AWS Elasticsearch while keeping the original schema. In our case, the highly normalised schema - appropriate for the primary data source - was unsuitable for search and analysis. We had to redesign the data fed into Elasticsearch from the ground up, involving all data stakeholders (engineers, analysts, customer care) and leveraging our financial domain knowledge to build the optimal schema for business analytics at quirion. To transform and store the redesigned data into Elasticsearch, we developed a custom solution using Logstash, the open-source ETL tool by Elastic.

Finally, we invested significant energy on Kibana, Elasticsearch’s web user interface. The goal was to serve business intelligence capability to financial and marketing analysts providing user-friendly interactive dashboards, drill-down and through capabilities, live boards of business KPIs, scheduled email reports, Amazon CloudWatch alarms triggered upon detection of certain financial conditions or events. Thanks to the flexibility of AWS Elasticsearch and Kibana, we could rapidly adapt some of these solutions as proofs-of-concept for other teams at quirion, such as customer care and IT.

The Upshot

„With kreuzwerker we found a technical experienced, reliable and committed partner for one of our core cloud application developments. This platform has to be developed to be highly scalable and performant serving several 100.000s customers in the future“, Marcel Müller, CTO quirion.

The joint work of quirion and kreuzwerker resulted in a business intelligence solution based on AWS Elasticsearch, a solution which is both effective and efficient. Effective because, thanks to the coordinated joint effort made by multiple teams, as well as the setup of the appropriate tools, quirion analysts now have capabilities at their fingertips that were too time-consuming or just not possible before. Efficient because we could almost halve the AWS Elasticsearch costs by tuning the service configuration and internals, while optimising data for financial and marketing analysis at the same time.