Supporting the support: Scaling customer success at Scalable Capital with AWS AppStream

The Project

Scalable Capital is growing rapidly, as well as their need to scale customer support. The Client Success team, which takes care of customer requests, is supported by external employees. The goal is to provide a Virtual Desktop Interface (VDI) solution to external employees. Access and usage must be limited to a few specific components that are needed to fulfill the tasks in client success. It was decided to use AWS AppStream to create a scalable environment that can be rolled out to external employees.

The Challenges

Although it was already decided to use AWS AppStream, several requirements and challenges made the project interesting:

• The solution had to be delivered as IaC using Terraform
• The solution had to be highly available and scalable to hundreds of users
• All outgoing traffic must be restricted and filtered to a handful of allowed domains
• In addition to that, all outgoing traffic must originate from a fixed set of static IP addresses
• Access to the AppStream instances should be managed using GSuite SAML 2.0 federation

The Solution

Before going into the details, you’ll find an architectural diagram of the solution below.

Networking

One of the biggest hurdles was to build the correct networking set up that would fulfill the security, high availability, and scalability requirements. We first built three workload subnets in three different Availability Zones to host the AppStream instances. This setup ensures fault tolerance against the failure of up to two AZs. In addition to that, the subnets are designed to have a CIDR range of /21 to have enough capacity for future scaling. In addition to that, we placed 3 AWS-managed NAT Gateways in front of the workload subnets to have static IP addresses for outgoing traffic, which can be allowed in other places and tools.

A fascinating problem was filtering and restricting outgoing traffic to allowed domains. The issue of filtering outgoing traffic itself is not complicated, but to have a solution that is easy to maintain, scale, and manage, is more challenging. In addition, we not only had to whitelist single IP addresses, but fully qualified domain names (FQDN). Using AWS security groups and NACLs does not work with FQDN. Using HTTP Proxies, such as squid, requires the setup of EC2 instances, maintenance, etc. Other 3rd party alternatives, e.g., Aviatrix, look promising, but essentially have the same downsides as using HTTP proxies.

Luckily, AWS released their own Network Firewall in November 2020, which offers precisely the features we needed: Filtering traffic based on domain names. To achieve that, you have to pass the traffic through a so-called “firewall-subnet,” a subnet containing the Network firewall (essentially a VPC endpoint). Then you have to adapt the route tables of the workload and NAT-Gateway subnets to route incoming and outgoing traffic through these firewall subnets.

Network topology can also be seen in this AWS blog post.

Using AWS Network Firewall drastically simplifies the networking setup regarding filtering outgoing traffic.

SAML 2.0 Integration with GSuite

The external client success employees are given GSuite logins and assigned to specific groups. It is possible in AWS Appstream 2.0 to use identify federation through a SAML2.0 identity provider. The example authentication workflow can be seen here.

In addition to that, there is an extensive step-by-step guide on how to set up G Suite SAML 2.0 federation with Amazon AppStream 2.0.

Using GSuite as an identity provider for AppStream completely removes the complexity of providing external employees some access to your AWS accounts; we can highly recommend it. Just be sure to follow the step-by-step guide thoroughly and not make any minor typos.

Image for AWS AppStream Instances

When setting up the Appstream instances, you have to specify whether the users can access the whole desktop or only start some specific applications. In this case, the users should only be allowed to start the Chrome browser. The Appstream instances are launched from base images, called image builders.

To create your own custom image, you connect to an image builder instance, install and configure your streaming applications, and then make your image by creating a snapshot of the image builder instance.

Configuring the applications in the image builder instance can be tedious, as there is a constant back and forth between configuring and testing. There is also a way to create an image programmatically; however, exploring that option showed that it was not delivering all the features we needed. For example, you still had to manually download the Chrome application onto the image builder at that point in time.

Autoscaling

Running many Appstream instances comes with a price. To be as cost-effective as possible, you can autoscale your Appstream instances.

In the case of Scalable Capital, we are utilizing two different auto-scaling strategies:

• One is to scale based on a schedule. Let’s assume all external employees are working from the same time zone. Then it does not make sense to have instances running when there are no office hours. We spin up a base-capacity of cases where office hours are starting and scale them down when workers leave the building.
• Apart from having a base capacity during office hours, Capacity Utilization scaling is in place. Whenever the number of used Appstream instances goes above a certain threshold (e.g., 80%), the fleet scales and adds instances

In reality, it is a bit more complicated. We are using a combination of schedule-based scaling and scaling based on Capacity Utilization.

One could get rid of autoscaling completely when utilizing the elastic-fleet type and app blocks and applications. This feature was released too close to the deadline and could not be considered part of the solution.

Summary

Using AWS Appstream 2.0 with autoscaling and GSuite for identity federation gives Scalable Capital a cost-effective VDI solution. If you need a highly reliable and scalable VDI solution, we can recommend AWS AppStream thoroughly. Another critical piece was incorporating the AWS Network Firewall into the network architecture to provide an easy and scalable way to control and administer outgoing and incoming traffic.