Why should you not use Cypress for Component Testing?

Storybook recently announced that they are working on an interaction testing feature
20.12.2021
Kailaash Balachandran
Tags

Given the rise of component-based libraries (Vue, React) and frameworks built on top of them (Nuxt, Next, etc), it is critical to end-end test the components in isolation. Cypress announced the alpha release of its dedicated Component Test Runner in version 7.0. It allows you to run your tests in the browser just like someone visiting your app would use it. These tests could sit next to your component file, where the goal is to create tests focused on each component rather than the whole application. These tests are less flaky, can run much faster, and with less overhead because component tests do not require page routing or loading the rest of the application.

However, I’m of the opinion that while Cypress component tests enforce developing modular and testable components, it certainly misses out on UI documentation. If you’d like to create UI documentation and a style guide for the components, you still have to rely on tools such as Storybook. (If you aren’t familiar with the tool, you can check out my write-up for an intro. Short version: Storybook lets you build pieces of a web app in isolation with way less overhead).

Given the case of a simple component e.g. a Button, it becomes a maintenance problem as it might end up having three-/four files for different use-cases as seen below.

  1. Button.js (the component)
  2. Button.unit.js (for unit tests)
  3. Button.storybook.js (UI documentation)
  4. Button.cypress.js (Cypress component tests)

So, instead of testing every individual component using component test runner, why don’t we use e2e-test Storybook using Cypress? This way, we get the best of the two worlds, i.e., beautiful UI documentation and also a well-tested style guide of components.

Why test Storybook

Before we look into storybook testing strategies, let’s discuss why testing Storybook is important. I’m a big fan of Storybook. But, like any software, it is prone to rot when left untested. Although it shares code with your web app, it has a separate configuration, build process, and code paths. This makes it easy to overlook testing it. One of the reasons is that developers tend to focus more on the unit and e2e tests, leaving storybook components untested.

If your project uses Storybook, it’s super important that we ask these questions:

  1. If the Storybook build were to fail, how would it be discovered?
  2. How would you be notified if your Storybook components failed to render?

The short answer to #1 is simple. That is, the CI should fail. If your app is not performing a Storybook build in CI, it is crucial to add it to the pipeline. Concerning #2, the answer is to leverage e2e testing using Cypress. There is also an upcoming integration testing feature in Storybook that appears to be a viable alternative for component testing. In the following sections, let’s discuss the approaches briefly.

Testing Storybook with Cypress

Storybook is essentially a stand-alone application with its own build configuration. In practice, it may fail as we work on the app, update dependencies, and so on. Let’s write a simple test that detects when the Storybuild build fails, at least in the most fundamental, easily detectable ways (for example, when the default story cannot render).

I’ll assume you’re already testing your app with Cypress. To start, make a second Cypress configuration (cypress.storybook.json) that points to your Storybook server’s URL (:9000 in the example below) and references a separate integration folder, so we introduce a separation of concerns between pure e2e and storybook tests.

//cypress.storybook.json
{
  "baseUrl": "http://localhost:9000",
  "integrationFolder": "cypress/storybook",
  ...
}

Add scripts to package.json for convenience.

//package.json 
"scripts": {
    "start:storybook": "start-storybook -p 9000 -s public",
    "cy:test:storybook": "cypress run --headless -C cypress.storybook.json",
    ...
 }

Now create a storybook.spec.js file inside the integration folder as set in the button.storybook.json file and add the following.

// button.spec.js
const getIframeBody = () => {
   // get the iframe > document > body
   return cy
       .get('iframe[id="storybook-preview-iframe"]')
       // and retry until the body element is not empty
       .its('0.contentDocument.body').should('not.be.empty')
       // wraps "body" DOM element
       // https://on.cypress.io/wrap
       .then(cy.wrap);
}

describe("Button", () => {
   before(() => {
       cy.visit("/");
   });
   it("loads primary button with default text", () => {
       getIframeBody().get('#root').contains('button', 'Button');
   });
});

As you would’ve noticed, the test uses iframes. Working with iframes is a bit tricky in Cypress. Because when Cypress DOM commands reach the #document node inside the iframe, the traversal operation comes to a halt. However, as documented here, it is possible to create a custom code to make it work. The above solution is minimal in the sense of what it does. But it secures a foothold, should we want to add further Cypress Storybook tests in the future. The logic can also be extended to even manipulate knobs and stuff via the query params or use the cypress-storybook library to add Cypress commands for Storybook. The library calls the Storybook router directly and offers commands to test the component knobs, labels, etc.

Storybook Interaction Testing

Storybook recently announced that they are working on an interaction testing feature that allows you to script interactions and check expectations in the story itself. That will enable you to run functional tests across UIs in the browser’s same environment you develop them. Powered by Testing Library, it comes with time-travel capabilities and also permalinks for easy debugging. Having the test setup inbuilt, we can write interaction tests inside the story itself. This also sets a clear boundary of concerns among Cypress and Storybook, wherein the former can focus on pure e2e tests while the latter on component documentation and testing.

Both Cypress and Storybook teams are working on expanding the surface of their tools, which seems to overlap now; Storybook with their Storybook Interaction Testing, Cypress with their Component Test Runners. As stated before, Storybook Interaction Testing is currently under active development. Once released, I believe this would be the way to go for testing isolated components. If your app doesn’t use Storybook yet, it’s high time that you introduce the tool as it streamlines UI development and documentation. If your app uses Storybook, writing Storybook Cypress tests appears to be a viable option for now. As for Cypress Component Testing, I would certainly not employ them for components that already have UI documentation in Storybook. I’m not saying that you should not use Cypress Component Tests at all, but if you do have UI documentation or building a design system, it is better to run Storybook’s interaction tests on an already isolated environment.

Disclaimer: At the time of writing this blog, Cypress Component Test Runner is an alpha release, and Storybook’s Interaction Testing is under active development. It is possible that, with subsequent releases, the cases discussed in this blog may hold untrue.