Tearing apart our biggest Javascript application: Morgenpost's Flugrouten Radar - Part 1

If you are working as a web developer at some point you will find yourself dealing with a project bigger than you are. It's just a matter of time: ambitious goals, short deadlines, new technologies, uncertain outcomes.
24.07.2013
Giuseppe Sorce
Tags

flugrouten 01

If you are working as a web developer at some point you will find yourself dealing with a project bigger than you are. It’s just a matter of time: ambitious goals, short deadlines, new technologies, uncertain outcomes. Despite being demanding, those kind of projects can truly raise the bar of your knowledge and experience to a whole new level.

This is what Morgenpost’s Flugrouten-Radar  has been to us: a huge challenge demanding us to push our boundaries. And we did it. This is the first of a series of posts when we’ll start analyzing the stack we used and the way we structured our code.

An overview

Morgenpost’s Flugrouten-Radar has two goals:

  • Giving a graphical representation of the air traffic over a specific location in Berlin for a given date
  • Showing statistical data regarding the air traffic over the whole city of Berlin for a given range of months

The first one (incarnated by the “Mein Standort/My location” tab) is the most important and it had to be available for mobile platforms as well. Once the user types an address (or use their current location if the browser supports it) and selects a date, a 3D view of the flights traces over a 2km radius is shown: the user can rotate the view and click on the traces to get additional informations for a specific flight.

The second goal is embodied by the “Alle Ortstelle/All districts” tab and it is a more journalist-oriented desktop tool. A bar chart on the top represents the number of flights per month in the available time range. The user here can select a range of one or more months to analyze. A choropleth map of the city can be switched between showing:

  • All flights: opacity represent the normalized number of flights, color the average altitude
  • Night flights: shades of black representing the normalized number of night
  • flights Change: represent the change rate between the beginning and the end of the selected time range.

Right next to it a table shows all relevant data regarding the districts.

The stack

Data come from a PostGIS database and are delivered by a RESTful Rails API (which is beyond the topic of this article).

Given the level of complexity of some behaviors and the need to use a lot of external libraries, our obvious choice has been Backbone: this allowed us fine-grained tweaking (DOM manipulation, response parsing, etc.) while keeping our code well organized. Underscore comes along with it, of course.

At kreuzwerker we are big fans of Brunch when it comes to organize and build our Javascript applications. This project was no exception, and for the first time we integrated the build process in the automatic deployment process (using Capistrano, another XW favorite) with ease.

We picked SCSS for the stylesheets. We base our class-naming conventions on SMACSS and, as the application needed to be embedded in 3rd party pages as well, we adopted a name-spacing convention in order to reduce the chance of classes clashing. We also used a minimalistic build of Bootstrap providing the grid and the scaffolding style for a couple of plugins.

For our templates we relied on Handlebars. If you’re accustomed to Mustache templates, the transition is easy. Furthermore, Brunch has a module for it allowing automatic pre- compilation and minification of all your templates. But the key feature, and life saver, is the support for helpers. We needed massive data formatting for dates, number, locales (more  on that in the next paragraph), as well as finding labels via ids in specific configuration files, and Handlebars’ helpers were just the right tool for it.

Internationalization was needed, but we didn’t want to clutter the API with it, so we opted for i18next. If you have been working with any i18n implementation, this is pretty much it. The cool thing is, they provide helpers for Handlebars on their website ready to be used in your templates. Piece of cake.

Regarding maps we gave Leaflet a try and we’ve been impressed by it. We used to work with OpenLayers, but sometimes it’s just too big for the task. Leaflet is very easy to use and customize. It’s well documented and a lot of tutorials are available online. The perfect companion to Leaflet has been Mapbox. They offer customizable maps, tiles and single image services. Furthermore, it’s very cheap. Your customer will love that.

Raphaël provided the vector graphics needed for the 3D map. dc.js came handy for plotting data out of our big dataset.

Last but not least, Moment.js saved our day when it came to deal with dates. Dealing with dates is often underestimated, but when you need to format, convert and process dates, moment is the winner:

var isoFormat = "YYYY-MM-DD";
var oneMonthLaterIso = moment(this.model.get('dateTo'))
   .add('months', 1)
   .format(isoFormat);

'nuff said.

Structuring the App

After dealing with many Backbone-based projects we ended up having some conventions. If you’re familiar with Backbone, you know that it favors extensibility over forcing you to a particular pattern, which leads to a lot of manual housekeeping and the need to use plugins from time to time, depending on the app’s specifications. It’s now time to show some code. This section assumes you already have at least some preliminary experience with Backbone. A good reference is Addy Osmani’s 'Backbone Fundamentals’.

Folder structure

Brunch handles that for us nicely and we’re quite happy about it. The only additions we had to do for this project is a locale  folder inside assets storing i18next translations and a static folder inside lib where we keep some small geoJSON static files.

The main app module

Over time we created mainly widgets and plugins. Just as a matter of convention, we usually rename our main app.js as .js file. What it does:

  • Creates the application object and exports it as a CommonJS module
  • Initializes globally-useful modules (the router and config file for example, or a layout view if needed)
  • Creates an event bus and inject it into Backbone’s prototype
  • Initializes external libraries, if needed

It looks like this: There are some big advantages of exporting the app as an object. One is that you can pass options on initialization: Another one is that you can require it elsewhere. (Just make sure not to create circular dependencies! If you need to require external modules in your main app module, do that inside a function, see the setup function above). For example, if you need to access the Config in your Handlebars helper file:

var app = require('./../my-app-name'); // view-helper is scaffolded into lib by brunch

//Fetches a specific label based on the id from a Hash in Config

Handlebars.registerHelper('nameFor', function(options) {
  var what = options.hash.what;
  var table = options.hash.table;
  var el = app.config.get(table)[options.hash.id];
  return new Handlebars.SafeString( el ? el[what] : '');
});

Later in your template:

<p>You selected {{nameFor what=id table="products"}}</p>

Rule of thumb, whenever you need to access data in the Config, it’s a good place to require the app module. Sometimes you may want to include app-wise functions there, like custom sync function to share along some models. Or for example you may want to keep the event dispatcher private to the app and not injecting it in generic Backbone Models and Views:

//Creates a global event dispatcher.
//The dispatcher is NOT injected this time.

 initEventBus: function() {
   this.events = \_.extend({},Backbone.Events, { cid : 'dispatcher'});
 },

Then in your views:

var app = require('my-app-name');

// …

app.bus.on('message', function() { console.log('Do something'); } );

The Config model

The Config itself is nothing more than a Backbone model, pre-filled with default values:

module.exports = Model.extend({
  defaults: {
    VERSION: '0.1',
    locales: ['en','de'],
    defaultLocale: 'de',
    // ...
    el: '.main-app-container',
    apiUrl: '/api/v1'
  }
});

Nested-views conventions

If you worked with Backbone already, you faced ghost views at least once. Especially if you worked with nested views. There are many framework out there providing solutions to these problems (Marionette.js being probably one of most notable out there). However we decided to craft a simple solution ourselves, extending the basic view.js and letting the other views inheriting from it. The main goals were:

  • Use conventions to define sub-areas for sub-views
  • Call sub-views clean-up functions on destroy
  • Render sub-views on view render
  • Replace the content of a placeholder tag instead of appending the content to it (default behavior for brunch generated view)

This is how our view.js looks like: This way when declaring a new view with nested sub-views one must specify:

  • A blocks hash of names and selector
  • A views hash of names (must match the ones in blocks) and the views
  • An onDispose function with manual events cleanup called after dispose

This is what we do in the main view for “My Location”:

 
module.exports = View.extend({
   
  template: template,

   
  initialize: function() {
     
    \_.bindAll(this);

     
    this.blocks = {
       
      summary: '.asf-summary',
       
      map: '.asf-3d-view',
       
      socials: '.asf-socials',
       
      comparison: '.asf-comparison-chart'
     
    };

     
    /* Models */
     
    this.models = {
       
      locationFlights: new LocationFlights({
         
        selection: this.model
       
      }),
       
      locationTraces: new LocationTraces({
         
        selection: this.model
       
      }),
       
      comparisonData: new ComparisonData({
         
        selection: this.model
       
      })
     
    };

     
    /* Views */
     
    this.views = {
       
      summary: new SummaryView({
         
        model: this.models.locationFlights
       
      }),
       
      map: new MapView({
         
        model: this.models.locationTraces
       
      }),
       
      comparison: new ComparisonChartView({
         
        model: this.models.comparisonData
       
      }),
       
      socials: new SocialsView({
         
        model: this.models.locationFlights
       
      })
     
    };

     
    /* Bindings */
     
    this.subscribe();
   
  },

   
  onDispose: function() {
     
    _(this.models).each(function(model) {
       
      model.off();
       
      model.stopListening();
     
    });
   
  },

   
  // ...
 
}

and its template:

<div class="asf-pane">
   <div class="asf-my-location row-fluid">
       <div class="span3 asf-summary hidden-phone"></div>
       <div class="span9">
         <div class="row-fluid">
           <div class="asf-3d-view"></div>
         </div>
         <div class="row-fluid">
           <div class="asf-comparison-chart"></div>
         </div>
       </div>
   </div>
   <div class="asf-socials"></div>
</div>

The pattern that emerges here is that big views, with subviews, actually act as Controllers, despite their name. Some frameworks like Chaplin extend Backbone with Controllers in an explicit manner, this is reasonable as this kind of functionality is needed anyway. However this pattern worked fine for us, as regular views can slowly turn themselves into more abstract controllers or widgets as development goes forward, with minimal code changes.

This concludes our walkthrough of the overall structure of our Backbone app. In the next post we will dig more into the 3D view and the data rendering.