Substance Document helps with the creation and transformation of digital documents. It ensures consistency, separates content from presentation and provides an easy to use API. It depicts the heart of the Substance Platform, a set of tools for content creation and distribution.

A Substance Document can range from loosly structured content involving headings and text, such as reports or articles to more complex things that you wouldn’t consider a traditional document anymore. The format is designed to be extensible, so you can create your own flavors of documents. We put a lot of thought into the design of this module. This first release is a result of more than 6 months of research and development, so we're looking forward to your feedback.

Play with the Console

Without too much talking, just have a look yourself. The Substance Console allows you to explore some examples and to mess around with the document manipulation protocol in a playful manner.

Design goals

  • A document consists of a sequence of content nodes of different types (e.g. heading, text, image)
  • A document is manipulated through atomic operations
  • The history is tracked, so users reconstruct previous document states at any time
  • Support for incremental text updates, using a protocol similar to Google Wave
  • Support for text annotations that are not part of the content, but rather an overlay
  • Support for comments to have dicussions that can stick on content elements or annotations.

Getting started

This section is intended to be a step to step guide on how to use the module to programmatically create and transform digital documents of any kind.

Start tracking a new document.

var doc = new Substance.Document({ id: "document:substance" });

First off, let's specify a human readable title for our document

var op = ["set", {"title": "Substance" }];

doc.apply(op, {"user": "michael"});

Add a first heading to our document.

var opA = ["insert", {
  "id": "heading:1",
  "type": "heading",
  "target": "back",
  "data": { "content": "Hello" }
}];

doc.apply(opA, {"user": "michael"});

Now let's add another node, this time a text node.

This operation is pretty similar to the previous one, except this time we specify text as our type.

var opB = ["insert", {
  "id": "text:2",
  "type": "text",
  "data": { "content": "Hey there." }
}];

doc.apply(opB, {"user": "michael"});

Update an existing node

There's a special API for incrementally updating existing nodes. This works by specifying a delta operation describing only what's changed in the text.

var opC = ["update", {
    "id": "heading:1",
    "data": {
      "content": [5, " world!"]
    }
}];

doc.apply(opC, {"user": "michael"});

Inspect the document state

Now after executing a bunch of operations, it is a good time to inspect the current state of the document.

doc.toJSON();

This is how the JSON serialization looks like:

{
  "properties": {"title": "Substance"},
  "nodes": {
    "heading:1": {
      "content": "Hello world!",
      "id": "heading:1",
      "type": "heading"
    },
    "text:2": {
      "content": "Hey there.",
      "id": "text:2",
      "type": "text"
    }
  },
  "views": {
    "content": ["heading:1", "text:2"]
  }
}

As you can see there are two nodes registered, which can be directly accessed by their id. In order to reflect the order of our nodes we keep an array of node id's in views.content.

History

But we don't only have access to the current state, the document model keeps track of all operations that have been executed.

doc.export();

Now this is the slightly more verbose output of the complete history of that same document. It wraps every operation in a commit object, each having a GUID as well as a reference to the previous commit.

{
  "id": "doc:hello",
  "refs": {
    "master": {
      "head": "f7bd8120adb16d179c454ce57b57b50d"
    }
  },
  "commits": {
    "1b6544d5f9724c33aa8dbb5becc51c0d": {
      "op": [
        "insert",
        {
          "id": "section:1",
          "type": "section",
          "target": "back",
          "data": {"content": "Hello"}
        }
      ],
      "user": "michael",
      "sha": "1b6544d5f9724c33aa8dbb5becc51c0d"
    },
    "b0595f2d3863db5ecd82d43e7ec05da0": {
      "op": [
        "insert",
        {
          "id": "text:2",
          "type": "text",
          "target": "back",
          "data": {"content": "Hey there."}
        }
      ],
      "user": "michael",
      "sha": "b0595f2d3863db5ecd82d43e7ec05da0",
      "parent": "1b6544d5f9724c33aa8dbb5becc51c0d"
    },
    "f7bd8120adb16d179c454ce57b57b50d": {
      "op": ["update", {"id": "section:1", "data": [5," world!"]}],
      "sha": "f7bd8120adb16d179c454ce57b57b50d",
      "parent": "b0595f2d3863db5ecd82d43e7ec05da0"
    }
  }
}

Time travel

Keep in mind, we can always look back. According to the commit graph shown above, we can checkout any reference in our document.

doc.checkout("master", "1b6544d5f9724c33aa8dbb5becc51c0d");

When using the Substance Console and exploring the history, this is what happens behind the curtain.

Construct an existing document

When constructing a document, you can pass in the history of an existing document, by providing all operations that happenend on that document, which are used to reconstruct the latest document state.

The document must come in this format.

var docSpec = {
  "id": "DOCUMENT_ID",
  "refs": {
    "master": {
      "head": "commit-sha"
    }
  },
  "commits": {...}, // Commit history containing all operations applied on that document
}

var doc = new Substance.Document(docSpec);

Annotations

So far, we have a bare-metal digital document, containing two different types of content nodes. Now we'd also like to store additional contextual information, relevant to a particular portion of text within the document.

Unlike in other systems with Substance annotations are not part of the content itself. They're completely separated from the text. Most text editors offer the ability to emphasize portions of text using markup. E.g. in HTML it looks like this.

<em>Emphasized term</em> in a text body.

In Substance however, we keep annotations external and remember the position of the first character, as well as an offset (how many characters are effected). An annotation emphasizing the "Hey" in "Hey there." looks like so:

{id: "annotation:23", type: "em", node: "text:2", "pos": [0,3]}

Insert annotation

Like document content, annotations are manipulated through operations.

var op1 = ["insert", {"id": "em:1", "type": "emphasis", "data": {"node": "section:1", "pos": [0,3]}}];
document.apply(op1);

Comments

Substance has in mind the usecase of collaborative composition of documents. Thus it allows a lively discussion during the editing process.

So if you wanted to support comments in your document model, you can can just define a new annotation type comment and stick them on either a content node or an annotation.

Insert comment

Like everything, adding a comment needs to be done by specifying an operation.

["insert", {"id": "comment:a", "type": "comment", "data": {"node": "section:1", "content": "Shouldn't you add an exclamation mark here?"}}]

API

Constructor

var doc = new Substance.Document({id: "document:hello-world"});

Checkout revision

Usage: doc.checkout('master', ref)

doc.checkout('master', 'head'); // Checkout head
doc.checkout('master', '1b6544d5f9724c33aa8dbb5becc51c0d'); // Checkout a given sha

Apply Operation

Usage: doc.apply(op, [options])

doc.apply(["delete", {"nodes": "comment:25"}]);

List annotations

Usage: doc.find(viewName, node)

To list annotations.

doc.find('annotations', 'text:25');

To list comments.

doc.find('comments', 'text:25'); // Comments for a given content node
doc.find('comments', 'idea:54'); // Comments for a particular text annotation

Document Operations

Insert Node

Parameters:

  • id - Unique ID of the element
  • type - Type of the new node
  • data (optional) - All properties that need to be stored on the node
  • target (optional) - Only for content nodes, can either be 'front', 'back' or the id of a target node. Defaults to 'back'.

Inserting a text node.

["insert", {
  "id": "text:25",
  "type": "type",
  "data": { "content": "Hello" }
}];

Insert a new comment.

["insert", {"id": "comment:a", "data" {"node": "text:2", "content": "A way of saying helo.", "node": "idea:1"}}]

Update Node

Parameters:

  • id - ID of the node to be updated
  • properties (optional) - Properties with new values
  • data (optional) - An object of key/value pairs. The key corresponds to the property name and the value can either be the new value or for strings, an array describing the delta of an incremental text update.

Updating properties.

["update", {
  "id": "text:2",
  "data": { "content": "Hello" }
}]

Incremental text update:

["update", {
  "id": "text:2",
  "data": {
    "content": [5, " world!"]
  }
}]

Move Node(s)

Parameters:

  • nodes - Node selection that should be moved to a new location
  • target - Target node, selection will be appended here

Moving two text nodes to their new position (only works for content nodes).

["move", {
  "nodes": ["text:4", "text:5"],
  "target": "text:25"
}]

Delete Node(s)

Parameters:

  • nodes - Node selection that should be removed from the document

Deleting a text node.

["delete", {"nodes": ["text:3"]}]