GOTO Berlin 2014 Summary

The conference was very pleasant, filled with interesting talks and attendees.
11.11.2014
Andreas Profous
Tags

Introduction

I went to the GOTO Berlin 2014 conference, and in this post I’ll try to summarize the event and talks I went to, and will occasionally also mesh in my personal thoughts. In general, the conference was very pleasant, filled with interesting talks and attendees. The venue and food was also very likable - but I suppose that’s not what you’re here for. So let’s jump to the first talk.

Microservices

Martin Fowlers keynote was split in two parts. In the first part he explained a part of his article about microservices. Microservices were a theme throughout the conference - a whole track was basically dedicated to it - and seem to be very hip. The first thing to note is that there is no concise definition of what a microservice architecture actually is. There are a common set of properties:

  • The system is componentized via separate services
  • Each service generally has their own persistence layer, rather than sharing the same database
  • The deployment of the system is automatized. Basically, because otherwise it would be too much effort to deploy the complete system.
  • A design of the system such that it is resilient in case a service fails. This is often accompanied by sophisticated monitoring.
  • Evolutionary design

Moreover, it was claimed that microservices are merely a subset of the SOA architecture (which is a very broad term, anyway). Interesting for me was that apparently microservices isn’t something new - it’s just a new name for what has been done since a long time by part of the SOA community. What’s also important is that there are advantages and disadvantages for choosing a microservice architecture versus a monolithic approach. Just trying to blindly forcing a microservice architecture for every particular problem is not a good decision - there are always trade-offs.

In the second part of his talk, he appealed to the responsibility we have as software developers. Generally, we should make an informed decision as to whether we dedicate our work for good causes (e.g. health care services) or bad causes (e.g. developing a spy tool for the NSA). Furthermore, he said it was not a good sign to not see many women in the audience. It’s a sign that we exclude certain groups, with no good reason.

Later, I joined the talk of Chad Fowler, in which he outlined the architecture of Wunderlist 3. At least for me, this talk seemed to present a practical example of a microservice architecture, as outlined in the keynote before. The interesting facts for me:

  • There is a sophisticated monitoring system in place. It is based on StatsD and Librato.
  • There are two layers of microservices doing most of the work. The first layer just encapsulates the logic, the second act as wrappers for the persistence. One advantage is that the persistence services encapsulate all mutations to the system, and can be used for event sourcing.
  • There are microservices implemented in a multitude of languages.
  • Each microservice must conform to a certain set of requirements. I can’t remember all of them, but first, they should be fairly small. Secondly, each service needs to serve metrics to StatsD. Moreover, there are naming conventions in place to indicate to which “layer” a service belongs.

In my view, the microservice architecture is often a natural choice for an existing system that grows to fulfill more features and more load. Creating prototypes and possibly even first product launches with microservices is very often a bad idea, since changes to the system are often more easy to do with a monolithic approach.

Fighting Mass Surveillance

The talk of Eric Dörnenburg and Martin Fowler continued where the keynote left off in terms of responsibility and started with the first paragraph of George Orwell’s 1984 novel. The claim was that at least in some sense we’re already in a much worse state than is described in that dystopia. For example, it’s possible to constantly locate us through our mobile phones. Most of the talk was dedicated to weaken the argument “I have nothing to hide”, by saying that even if your own privacy is unimportant to you, it is important that at least some people have protection from state intrusion. An example would be an investigative journalist researching a corruption case. If nobody is protected, we lose basic human rights that society fought to achieve for a long time.

The second part of the talk presented the Pixelated project, which is an open source solution to encrypt email that aims at mass adoption. One big problem with mail encryption right now is that since it’s so rarely used, you raise suspicion just by using it. If it were more widespread, the cost to read all communication of everybody may become infeasible even for the NSA.

Working in the Cloud

In the talk of GitHub’s Benjamin Toews, he mostly talked about what kind of security breaches happened in the past and were detected by GitHub’s security team. He argued that even though everybody seems to think the cloud is insecure - and it’s certainly worthwhile to think hard about security - security breaches are much more frequent when using on-premises software. The main reason is that - depending on whether you use SaaS, PaaS, or IaaS - you basically transfer main responsibilities to another company. On the one hand, this of course means that you need to trust that company. On the other hand, these companies can use economies of scale to deliver a service quality that would be infeasible for you. This applies to security, too.

Peter van Hardenberg’s talk emphasized this point, too. He is a founding member of the Heroku Postgres team and claimed that “Heroku Postgres deals with once-per-decade bugs daily”, which is of course completely infeasible to do for smaller companies running their own database service. He also pointed out that cloud computing changed the cost of certain things like creating new servers by orders of magnitude, such that we’re just beginning to explore the implications. Things like immutable deployments (never update servers, just create new ones) are just the start. As an example, he described fourchette, which basically (temporarily) forks all your applications such that you have an isolated testing environment. In this way, you can “test against production”.

In the next talk, Adrian Mouat talked about Docker. He gave a basic introduction to Docker, which had few new insights for me. For me, the interesting part of the talk was about the current shortcomings of Docker. Usually, a Dockerfile currently does not determine an immutable state of a machine, because of (among other things):

  • Image Labels in the Docker Hub are not immutable. If you base your image on an existing image, it’s possible that the base image changes over time.
  • Similarly, it is idiomatic to install packages in the Dockerfile via e.g. RUN apt-get. The packages can change over time, since they’re just referenced with a package name.

After Agile

In the talk of Markus Andrezak, he basically summarized the limits and the scope of agile software methodologies. Within a company, most of the work is done to generate revenue using well-known products and processes. He called that “Horizon 1”. It is this area where agile methods thrive. However, for Horizon 2 and 3, which are about tomorrow’s cash flow and future growth options, they are often counterproductive. Finding a good opportunity for the future is a highly non-linear and creative activity. Forcing small increments often limits your own vision - Markus called this “premature conversion”.

The talk of Gojko Adzic was concerned about how to improve user stories. They shouldn’t be a means to represent requirements, but be the base for adaptive planning. His first claim is that if your requirements never change, your project will not benefit from agile methods. I think that’s true, but on the other hand, software requirements almost always change significantly over time, so this is not really relevant in practice. In a gist, his next claim was that you should describe behaviour change instead of behaviour in a user story. The idea is that the trade-offs become more clearer then. Moreover, he says that one should have the possibility to select the best among many options. I personally think it’s rather wasteful to explicitly create many different user stories representing different options first and then throw many of them away. Creating a user story should be a process, where the whole team slowly converges to the best user story, i.e. is based on a lot of communication in the team. Of course, implicitly, you do select one option out of many then. Unfortunately, I couldn’t discuss this with him, since he had to leave to catch a flight.

Cool and Fun Stuff

Fun stories from an Architect’s life was the title of the talk from Stefan Tilkov. The name says it: Stefan essentially told many small episodes from projects that were fun to listen to, because the architecture was often atrocious. The takeaway message is that bad architecture is very widespread, but often involved people fail to recognize it, maybe because they took certain things for granted. So be attentive, and try to reflect on your own projects.

In Britta Weber’s talk, she gave a basic introduction to Logstash, Elasticsearch and Kibana, which is a common and powerful open-source software stack to analyze log data. She made a hands-on presentation and analyzed data from Apache logfiles of the German Wikipedia. It was very fun to see how certain peaks of page access counts could be linked to e.g. a certain television show. It was also very useful, since I will definitely consider to introduce this to our smaller projects, too.

One of my personal highlights of the conference was Stuart Sierra’s talk about interactive Clojure development. He started off by describing how being able to interact with your running application using the REPL was already successfully used at NASA in the Deep Space 1 project. Then he described how the REPL is deeply integrated in his Clojure software development process: first things are being tried out in the REPL, then bits of code are transferred to functional and test files. This kind of immediate feedback proved to be very productive for him. He then went on describing two Clojure libraries he’s written, tools.namespace and tools.component. These solve the problem of a long restart time in case the state in the REPL is “dirty”, which happens e.g. if a function is renamed (other function definitions still point to the old name in the REPL). Basically, tools.namespace reads in Clojure code in the source files at run-time and detects which other files depend on a particular source file to compute a dependency graph of files to refresh. Similarly, tools.component solves the problem in case code depends on external state, e.g. a web server that was started. You describe a system by enumerating in code what components there are and how each component can be started and stopped. As a result, you can often get a “clean” REPL in less than a second, which is a great improvement.

I’ll finish the summary with Gunter Dueck’s inspiring keynote. With stories about mice, cheese and bacon he illustrated the point that it is often crucial to leave the comfort zone in order to improve (and survive). For us techies, the comfort zone is often sitting in front of the computer and solving hard problems alone, without communicating to others what we do and why. This behaviour is dangerous, he claims, because such work can be more easily outsourced to low-wage countries. Team work, communication intelligence, competency to teach, selling competency, and caring for others are often underrepresented skills, but a mixture of skills is often crucial to success. He claimed that a culture of care, appreciation of others and holistic thinking is the most successful. Such a company culture is not common among tech companies, as in Europe and the US a more traditional, hierarchical and meritocratic culture is prevalent. He hinted at this being one main disadvantage of having relatively few female employees.

As you can hopefully see now if you are still reading this, the GOTO Berlin was a fun ride with a myriad of different topics.

Photos: © 2014 Trifork GmbH