Why JSON as an article and book format?

When we create an article in JSON we're not simply stripping the closing tags from everything. We can also, for example, remove tags such as the <p> tags required in blockquotes, because the insertion of these can be implied.

If the parser we use to process the JSON finds a single array within a blockquote it knows there are not multiple paragraphs:

{"blockquote":["Single paragraph."]}

If the parser introspects and finds an array of arrays, it knows that each is a paragraph:

{"blockquote":[["First paragraph."], ["Second paragraph."]]}

Either can have emphasised text, citations, footnotes, etc.

{
 "blockquote": [
  [
   "First paragraph ",
   {
    "i": "with italic text"
   }, "."
  ],
  [
   "Second paragraph with a footnote.",
   {
    "fn": "This is the footnote."
   }
  ]
 ]
}

The equivalent to the above in HTML5/EPUB3 would look something like this:

<blockquote>

<p>First paragraph <em>with italic text</em>.</p>

<p>Second paragraph with a footnote.<a href="#ft1f" epub:type="noteref">1</a><p>

<blockquote>

<aside id="ft1f" epub:type="footnote">

<p>This is the footnote.</p>

</aside>

While it isn't an exact science, saving both of these examples (with unnecessary whitespace stripped out) in UTF-8 encoding, I get a JSON file size of 138 bytes and an HTML file size of 227 bytes (almost 40% larger) for the same content on OS X.

A lower file size for each article is great, but how would we introspect and parse an object such as this?

First, assuming we'd already drilled down to the blockquote and we we're aiming to display in HTML using JavaScript to parse, we'd

1. create and insert our empty blockquote into the document, giving it an incremented id
2. check if the value of the blockquote object is an array like we're expecting (if it's not then we can either extract all the text that we can anyway through further "introspection" or reject for its illegal formatting. Note: It might be that a quote with nothing other than plain text has been put in without square brackets, and we should probably allow this.)
3. check the length of the array, i.e. the number of items in it, so we know how many times to iterate across the array
4. check the type of the first item in the array
5. if we hit a string or an object, we know this is a single paragraph blockquote, so we build the paragraph string with <p> tags either side and make it the inner HTML of the blockquote.
6. If we hit an array first time we know this a multi-paragraph blockquote and so build a paragraph string for each array within the array, and place this in our blockquote inner HTML
7. Handling objects within paragraph arrays, we retrieve the key first of all (e.g. i, b, em, ct, fn) and then build the HTML accordingly

And why would we go to such effort?

While all of this parsing might sound like a lot of work for one blockquote, in actual fact the code will be generic enough to be reused across all content types. And we'll follow a similar pattern in whichever language(s) we choose to render the document.

The most important bit

If we create articles and books directly in HTML, they are not only larger and less optimised for modern APIs, but we have more work when it comes to updating them for HTML6/EPUB4/5 and so on: with JSON we just update the parser, but with direct creation in HTML we have to update every single document.

Likewise, if we are rendering, for example, the same book natively in Objective-C and iOS gets updated, we simply update the parser we've built. Rarely, if ever, will we need to change the (JSON) data.

What else?

This might sound radical and make some publishers nervous, but with JSON we don't actually need to be involved at a CSS or HTML level at all. These parsers could be built into ereader apps and be entirely the responsibility of the app developers. Imagine if the move from EPUB2 to EPUB3 had been like this? We could have sat back and done nothing.

But what about our choice of fonts and type sizes, our CSS layout?

There's nothing to stop us declaring these preferences in the JSON as global characteristics, but actually ereader apps still restrict fonts and let the readers dictate the font they wish to read in, often throwing out all the things we spend time on anyway. And if we're truthful most of the CSS we write is to fix inconsistencies and to control margins and indents (tedious stuff).

With the advent of sites such as Medium.com we see that reading can actually be better with clean consistency across an app rather than variation. But publishers knew this already, that's why books don't have a jumble of layouts or fonts. They have a consistent style.

Think about the advantages

Using JSON and an API to distribute books via the web (see this earlier post), and letting developers handle the development (and design) end of things, incentivised by the income they'll receive, there is the potential to have native apps across all platforms (major and minor) without doing anything more than spending the time to create simple JSON files.

Follow @sketchytech