Fastboot rehydration: Server side HTML vs Client side HTML

I’m working on some code that tests fastboot rendered applications inside of Ember’s test runner.

I’m able to render the fastboot response and use assert.dom to test the server rendered content. It’s pretty cool!

Here’s an example test…

import { module, test, skip } from 'qunit';
import { setup, visit, renderedHtml } from 'ember-cli-fastboot-testing/test-support';

module('Fastboot | basic', function(hooks) {
  setup(hooks);

  skip('it renders html that matches the browser', async function(assert) {
    let { body } = await visit('/');

    assert.equal(body, renderedHtml());
  });

  test('it renders the correct h1 title', async function(assert) {
    await visit('/');

    assert.dom('h1').includesText('FastbootTesting');
  });

  test('it renders the correct og:title', async function(assert) {
    let { htmlDocument } = await visit('/');

    assert
      .dom('meta[property="og:title"]', htmlDocument)
      .hasAttribute('content', 'Fastboot testing');
  });

  test('it gets a success response code', async function(assert) {
    let { statusCode } = await visit('/');

    assert.equal(statusCode, 200);
  });

});

and here’s a preview of it running

Rehydration

@rwjblue suggested having these tests verify that the server rendered HTML matches the browser’s client side rendering, so that we know the client side HTML can be rehydrated correctly. It’s kinda awesome that we can do this!

Here’s why this matters… in fastboot, it’s possible generate html that’s invalid. For example, you could render this on the server:

<p><div>hi</div></p>

and the browser will auto correct it to:

<p></p>
<div>hi</div>
<p></p>

It turns out div tags are not allowed inside of p! This re-arranging of HTML will screw up Ember’s rehydration of the server rendered HTML.

In order to test this, I’m rendering the fastboot’s body html and then comparing it to the brower’s rendering.

let fastbootHtml = '<body> ...';

document.querySelector('#ember-testing').innerHTML = fastbootHtml;

let browserHtml = document.querySelector('#ember-testing').innerHTML;

fastbootHtml === browserHtml; // do these match!?

Rehydration is a black box to me and I’m noticing a couple of differences between the fastboot rendered content and the browser corrected content. I’d love to understand how these affect rehydration.

  1. Does fastbootHtml and browserHtml need to be an exact string match? If not, what does glimmer need to successfully rehydrate?

  2. What do I want to compare? Fastboot ships an entire HTML document, but the testing container is made for rendering an application template. I don’t believe I can render <!DOCTYPE>, <html>, or <head> tags inside of #ember-testing? Right now I’m only rendering what’s between the fastboot body boundaries.

  3. Is innerHTML the best way to test this? Or is comparing the dom’s tree structure better?

  4. In testing an app I noticed the server will render <div data-hello></div> and the browser correct that to <div data-hello=""></div>. How do attributes affect rehydration?

  5. In testing an addon that ships an SVG I noticed that the server renders <path ... /> in fastboot, but the browser auto corrects it to <path ...></path>. How do self closing tags affect rehydration?

  6. Do different browsers auto correct HTML in different ways? Are there differences that should be ignored?

  7. What other questions should I be asking!?

Thanks for reading!

10 Likes

Sorry didn’t have a chance to answer the questions just yet, but thank you for working on this! I really like the way the API is coming together.

3 Likes

You will want to diff the DOM trees, not the strings. That answers several of your sub-questions, because things like self-closing tags or data-hello vs data-hello="" aren’t going to affect the DOM structure.

The HTML spec covers error recovery, so they’re supposed to do it the same. I think you should just trust that you’re dealing with a spec-compliant HTML parser and diff the resulting DOM trees.

3 Likes

Ok that’s great to hear and like you said answers many of those questions. Also, DOM trees should be easier to compare!

I’ll whip something up this week and report back :smiley:

Btw, I had no idea that error recovery was part of the HTML spec

2 Likes

@ryanto thanks so much for working on this! My team is super excited to see rehydration progress and very much looking forward to getting it in a stable release!

EDIT: is there a github repo I can subscribe to or possibly help with?

Awesome @shull! I’m only working on the testing side of things. The repo for testing is here: GitHub - embermap/ember-cli-fastboot-testing: Test your FastBoot-rendered HTML alongside your application's tests.

Rehydration itself is above my pay grade. I know folks have been able to use it successfully. Are you currently running rehydration in production?

1 Like

@ryanto I tried out rehydration first thing when we upgraded to 3.2 but it didn’t quite work properly in our app. Our app is fairly complex and has probably deviated significantly from the standard “Ember Way™,” so I wouldn’t expect rehydration to work properly until it’s a bit more mature.

We’re in the process of upgrading to 3.4 & I’m excited to give rehydration another shot once that’s complete :smiley:

If it doesn’t work properly I have no idea how I’d go about debugging it, so your testing repo is cool because maybe I’d be able to come up with a failing test case :grimacing:

Trying it again just now, seems like the most glaring issue is our app renders <style> tags inline but for some reason these aren’t rendered by fastboot when I set EXPERIMENTAL_RENDER_MODE_SERIALIZE=true.

I also got some ember-wormhole error, complaining that the destination wasn’t in the dom. Came from rendering an atom with mobiledoc.

Update: after experimenting some more (we’re still not on 3.4 yet :weary:) I realized the problem is that the experimental renderer emits html comments around everything that is dynamic. HTML comments are not valid in <style> nodes (they need comments formatted for CSS, if they’re gonna have comments. So for example, we see:

<style><!--%+b:11%-->.bnd-4a61494f-e97f-424c-b946-2cb5467f522c .nav-container { background-color: #FFF !important; }

<!--%-b:11%--></style>

Until this is resolved, we won’t be able to use rehydration, as it causes a major Flash Of Unstyled Content.

Another, albeit smaller(?), issue is that if you’re using ember-cli-head to set the document title, it outputs with the comments (presumably not great for SEO?), like:

<title>&lt;!--%+b:10%--&gt;Introducing You, As a Hand-Drawn Emoji&lt;!--%-b:10%--&gt;</title>

These comments do display in the tab as the title until the app rehydrates. Not as noticeable but still not optimal.

Not sure whether or not it would be useful to add failing test cases to the ember-cli-fastboot-testing repo…

2 Likes

I start working on rehydration testing again, but I’m running into a problem turning the FastBoot HTML string into a tree.

Here’s some code that outlines what I’m running into.

let fastbootHtml = "<p data-paragraph><div>broken html!</div></p>";
let parser = new DOMParser();

// this will auto correct any html
let correctedTree = parser.parseFromString(fastbootHtml, "text/html");

// this will not correct any html
let uncorrectedTree = parser.parseFromString(fastbootHtml, "text/xml");

This works most of the time, but in the above case uncorrectedTree will have a pasererror because data-paragraph is not valid xml. Specifically, the xml spec doesn’t allow for valueless attributes while the html spec does.

I’m wonder if I should avoid using DOMParser, since it will put me in a strange place having to compare an XMLDocument and an HTMLDocument. If that’s the case, can anyone recommend a string to tree library?

Update:

I found this code by John Resig that parses HTML strings (John Resig - Pure JavaScript HTML Parser). I’ve been adapting it to create a tree structure to compare to a DOMParser tree and it’s been working well so far.

2 Likes