Cloud Compliance made Simple: Meet AreBOT

As an AWS Advanced Consulting Partner we have helped many enterprises to optimise their IT processes through modern cloud solutions. No doubt, this is easier said than done.
29.09.2017
Tags

As an AWS Advanced Consulting Partner, here at kreuzwerker we have helped many enterprises to optimise their IT processes through modern cloud solutions. No doubt, this is easier said than done, especially when you are touching running systems, as goes our motto. Among the many challenges, safeguarding the compliance of a large cloud environment, possibly spanning multiple regions and AWS accounts, is one of the most critical.

In our previous blog post, we took a brief overview of AWS Config, the Amazon Web Service solution for cloud governance. In particular, we focused on its automatic resource compliance checker and on how it can be configured by defining Config Rules. Then, we discussed what we believe are the main limitations of AWS Config: first and foremost, a non-negligible cost, but also its limited administration and customisation facilities.

In the need for adequate support to face the challenges of cloud compliance in complex infrastructures, we at kreuzwerker decided to take the plunge and develop our own solution: AreBOT.

AreBOT: an Overview

AreBOT as an automatic and highly configurable tool to monitor AWS cloud environments for resource compliance violations. The AWS resources currently supported are Elastic Compute Cloud (EC2) Instances, Security Groups, Elastic Block Store Volumes and Snapshots. Support for Simple Storage Service Buckets is also on the way.

Our tool offers both event-driven and time-driven compliance checking. Event-driven checks are triggered in response to resource’s configuration changes, which are tracked by the AWS auditing service CloudTrail and delivered to AreBOT as Amazon CloudWatch events using an Amazon SQS message queue. By contrast, time-driven checks are programmed to occur at regular intervals, triggered by a time-based job scheduler (i.e., cron). The two checking strategies complement each other: on the one hand, event-driven checks are particularly appropriate for monitoring real-time operations and reacting in short time; on the other hand, time-driven checks can be used to later determine whether a non-compliant resource has been fixed.

Any detected violation can be reported to a designated group of recipients, such as the cloud administrators or the resource creator. Furthermore, in some cases, corrective actions can be automatically set, like stopping a new EC2 Instance that is not free tier eligible (all but the “t2.micro” instance types, typically).

AreBOT, like other policy-as-code frameworks, allows the user to define complex compliance policies using a simple rule configuration language. In particular, the compliance policy of a given resource includes the set of compliance rules imposed on the resource properties (e.g., restrictions on resource names or tags), the set of actions to take in case of compliance violation (e.g., email notification or modification of the resource state), and a set of trigger rules to set-up time-driven compliance checks. Sound too complicated? Let’s look at an example.

Enforcing a Simple Compliance Policy using AreBOT

Consider the following scenario.

The admins of an AWS cloud infrastructure want to impose a naming restriction on the Security Groups created within a specific Amazon VPC. Specifically, the name of every Security Group must start with a capital letter. In case of compliance violation, the admins want to be immediately alerted via email, with a customised report.

The old, manual, un-fun solution: the admins start developing all the compliance evaluation and notification logic as a series of Amazon Lambda functions, integrate these functions with AWS Config and the Amazon Simple Email Service, and spend a fair amount of time and money to maintain them as the requirements change. Or, our recommended solution, the admins could just install AreBOT, and enjoy all those monitoring and management capabilities editing one simple configuration file. To see how simple it is, check out the excerpt of the AreBOT configuration that specifies the sample scenario.

security_group_policy "mySGpolicy" { // compliance policy for security groups
  vpc = "^vpc-9bec0bf2$" // apply the policy only to security groups within this Amazon VPC

  api_call "CreateSecurityGroup" { // monitor all the API Calls of this type
    compliant "GroupName" { // compliance rule to check every time the above API is called
      schema = "^[A-Z].*" // rule evaluation logic, expressed as a regex
      actions = [ "notify_admins" ] // the list of actions to trigger if non-compliant
    }
  }
  action "notify_admins" { // the action associated with the compliance rule
    email { receiver = [ "admin1@email.com" , "admin2@email.com" ] }
  }
}

In case of compliance violation, then AreBOT automatically composes a report as an HTML email (see the picture below) and sends it to the designated recipients. AreBOT comes with a set of predefined email templates, and new templates can be easily created as normal HTML documents.

arebot mail

Conclusion and Perspectives

In this post, we presented AreBOT, our recent cloud governance solution for enforcing compliance rules on AWS resources. Also, we showed how fast and easy is to create custom compliance rules and nice-looking notifications using its simple configuration language. The AreBOT client is written in GoLang, and uses the AWS SDK for Go to interact with the AWS services deployed on the AWS account to monitor.

Our plans for this project include extending support to other AWS resources and events, as well as developing new automatic corrective actions. Last but not least, we decided to release the AreBOT code on GitHub under the GPL, to share our experience and encourage contributions from the DevOps community.

Speaking of whom, shall we meet at the DevOpsDays Berlin - 2017? We hope to see you there, talk about AreBOT and get inspired by your projects!

Image credits for the cover image go to: TechRepublic