hypereffect

Specification

1Conventions

The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be interpreted as described in RFC 2119 and RFC 8174 when, and only when, they appear in capitals.

JSON refers to RFC 8259. Merge patch refers to RFC 7386. CEL refers to the Common Expression Language as specified at github.com/google/cel-spec.

The components and their attributes are specified on the Components page, which is part of this specification.

2Terminology

Server
The process that produces element, remove and signal messages and receives actions. It owns application state.
Renderer
The process or library that displays elements and sends actions. It has no knowledge of the application.
Host
The application that embeds a renderer and provides it with a drawing surface and a transport.
Session
One connection between a renderer and a server, from the renderer's hello until the transport closes.
Element
A JSON object describing one instance of a component.
Signals
A single JSON object holding the values that expressions read and controls write. Expressions read it as the variable s.
Local signal
A member of the signals object, at any depth, whose key begins with _ (U+005F).
Structural attribute
id, type, parent, before, bind, and every handler attribute. Their values MUST be literal.

3Framing

In both directions, the protocol is a sequence of lines encoded in UTF-8. Each line is one message, consisting of a tag, a single space (U+0020), and a JSON text, followed by a line feed (U+000A).

signal {"count":1}
element {"id":"a","type":"text","text":"Hello"}
  1. The tag MUST consist only of lowercase ASCII letters.
  2. The JSON text MUST be a JSON object and MUST NOT contain a line feed. Since JSON strings cannot contain unescaped control characters, any compact JSON serialization satisfies this.
  3. A receiver MUST accept a carriage return (U+000D) immediately before the line feed and ignore it.
  4. A receiver MUST ignore empty lines. Senders MAY send empty lines as keep-alives.
  5. A receiver MUST support lines of at least 1 MiB. It MAY reject longer lines.
  6. Messages MUST be processed in the order they are received.

4Sessions

  1. When the transport opens, the renderer MUST send a hello message before any other message.
  2. A session begins with an empty element tree and with signals equal to the signals member of the renderer's hello, or {} if absent.
  3. The server SHOULD respond to hello by sending the messages that build the initial interface.
  4. The server MAY send messages at any time during the session, whether or not they are a response to an action.
  5. When the transport closes, the session ends. The renderer SHOULD continue to display the last state with all controls disabled, and SHOULD indicate that it is disconnected.
  6. If the host opens a new transport, a new session begins. The renderer MUST discard the previous tree and signals no later than when it processes the first server message of the new session. It SHOULD keep displaying the previous state until then, to avoid an empty frame.

Resuming a session across transports is not part of this specification. Servers that need it can identify the user in the hello signals, or through the transport, and rebuild the interface.

5Server messages

5.1element

The JSON object is an element. It MUST have an id member (a non-empty string) and a type member (a component name). Processing is described in section 7.

element {"id":"title","type":"text","parent":"main","variant":"h1","text":"Inbox"}

5.2remove

The JSON object MUST have an id member. The renderer removes the element with that id and all its descendants. If no element has that id, the message has no effect.

remove {"id":"row-17"}

5.3signal

The JSON object is a merge patch. The renderer applies it to the signals as described in section 8.

signal {"unread":4,"draft":null}

6Renderer messages

6.1hello

Member Type Description
signals object Optional. Initial signals supplied by the host. Local signals in it are retained by the renderer and not sent.
hello {"signals":{"device":"kitchen"}}

The signals sent in hello MUST have local signals removed, as for actions.

6.2action

Member Type Description
name string Required. The action value of the handler.
payload any Present if and only if the handler has a payload; copied verbatim.
signals object Required. The signals at the time of sending, with all local signals removed.
action {"name":"archive","payload":{"contact":7},"signals":{"filter":"all"}}

Removing local signals means: for every object in the signals, at any depth, omit each member whose key begins with _. Arrays are copied as they are; objects inside arrays are filtered in the same way.

7The element tree

The renderer maintains a tree of elements. The root is implicit, has no id, and lays out its children as a column with default attributes. Dialogs are kept at the root but displayed as overlays (see dialog).

7.1Validation

An element message is rejected (ignored, with a diagnostic, see section 12) if any of the following hold:

  1. id is missing, not a string, or empty.
  2. type is not a known component name.
  3. parent is present and no element with that id exists.
  4. The parent, or the root if parent is absent, does not accept children of this type (see the parent attribute of each component).
  5. parent is the element's own id or the id of one of its descendants.
  6. A structural attribute is a CEL expression, or has the wrong JSON type.

Consequently a server MUST send a parent before its children.

7.2Insertion

If no element with the given id exists, the element is added as a child of its parent (or the root). If before names a current child of that parent, it is inserted immediately before that child; otherwise it is appended as the last child.

7.3Replacement

If an element with the given id exists, the new element replaces it:

  1. All attributes are replaced. Attributes absent from the new element take their defaults. Replacement is not a merge.
  2. If type is unchanged, the element keeps its children and the renderer SHOULD preserve widget state that is not described by attributes: keyboard focus, text selection and cursor, scroll position, and expanded or collapsed state.
  3. If type changed, all descendants are removed and widget state is discarded.
  4. If parent differs from the current parent, the element and its descendants move to the new parent, positioned according to before as in 7.2.
  5. If parent is unchanged and before names another current child of that parent, the element moves to immediately before it. Otherwise its position is unchanged.

7.4Removal

Removing an element removes its descendants. Ids of removed elements may be reused by later messages; a reused id creates a new element.

7.5Ids

Ids are opaque strings compared by code point. They are unique across the whole tree, not per parent. Ids are not shown to the user, except as the fallback label of a tab.

8Signals

  1. The signals object is a JSON object. A signal message is applied to it using the MergePatch algorithm of RFC 7386, section 2: members whose patch value is null are removed, object values are merged recursively, and any other value replaces the target value.
  2. Arrays are values: a patch replaces an array entirely. There is no way to patch an element of an array.
  3. Writes made by the renderer itself — from bind, from a handler's patch, or from setting an indicator — are also applied as merge patches.
  4. After applying a patch, the renderer MUST update the display as if every expression and every bound control had been evaluated again against the new signals. It need not actually do so: see 9.5. It MAY coalesce updates: it SHOULD process all messages that are available before redrawing.
  5. The server MAY write local signals. The renderer MUST NOT send them.

8.1Signal paths and binding

A signal path is a non-empty sequence of segments separated by ., each matching [A-Za-z_][A-Za-z0-9_]*. The path a.b.c denotes signals["a"]["b"]["c"].

A control with bind displays the value at its path, converted as its component describes. If the value is missing or has the wrong type, the control shows its empty state. When the user changes the control, the renderer writes the new value by applying the patch {"a":{"b":{"c":value}}}. This creates intermediate objects as needed and replaces any non-object value in the way.

When the bound value is changed by a signal message, the control MUST show the new value, even if it has focus.

9Expressions

An attribute value that is a JSON object with exactly one member, named cel, whose value is a string, is an expression. Any other value is literal. Expressions are permitted in every non-structural attribute and as leaf values of a handler's patch.

9.1Environment

  1. The environment declares one variable, s, of type dyn, bound to the current signals.
  2. Renderers MUST provide the CEL standard definitions and the strings extension as defined by cel-go. The complete list is on the Expressions page.
  3. Renderers MUST NOT provide functions with side effects or functions that read state other than their arguments.

9.2JSON to CEL

JSON CEL
null null_type
true, false bool
number with no fraction or exponent, in the range of int64 int
any other number double
string string
array list(dyn)
object map(string, dyn)

CEL does not mix int and double in arithmetic. A signal that may hold either should be converted with double() before arithmetic.

9.3CEL to attribute

The result of an expression is checked against the attribute's type:

For a handler's patch, the result is converted to JSON: numbers to numbers, maps with string keys to objects, lists to arrays. A result with no JSON representation (bytes, timestamp, duration, type values) is an error.

9.4Failure

If an expression fails to parse, fails to evaluate (for example because a signal is missing), or produces a value of the wrong type, the attribute takes its default value. In a handler's patch, the member is omitted from the patch. The renderer SHOULD report the failure as a diagnostic and MUST NOT report it to the user.

Renderers MAY limit the cost of evaluation using CEL's cost tracking, and treat an expression exceeding the limit as failed. The limit SHOULD be large enough for expressions of a few hundred characters over signals of a few hundred kilobytes.

9.5Dependencies

An expression's result is determined by the signal values it reads, since CEL has no side effects and the environment holds nothing else. A renderer can therefore re-evaluate only the expressions that read something a patch changed. This section defines what an expression reads, for renderers that do so.

  1. A read is a signal path, as in 8.1, extended to any keys. Selecting a field or a constant key from s, possibly repeatedly, reads the path of the selection: s.user.name and s['user'].name read user.name. Any other use of a value taken from the signals reads the whole value at its path: size(s.items) reads items, and s alone reads every signal. A selection that finds no key, including inside has(), still reads its path.
  2. A patch affects a read if the patch changes the value at the read's path, at a path that begins with it, or at a path it begins with. For example, a patch to user or to user.name.first affects the read user.name; a patch to user.email does not.
  3. The dependencies of an expression are the reads made during its most recent evaluation. Only the parts of the expression that were evaluated count. In s.a ? s.b : s.c, when s.a is true the dependencies are a and b, not c. The same holds for && and || when the left operand decides the result, and for macros that stop early, such as exists. An evaluation that fails has as dependencies the reads made before it failed.
  4. After a patch, an expression none of whose dependencies is affected has the same result as before, and need not be evaluated. An expression with an affected dependency MUST be evaluated again, and its dependencies are replaced by the reads of the new evaluation. If s.a above becomes false, the expression is evaluated again and from then on depends on a and c.
  5. A bound control depends on the path of its bind.
  6. A renderer MAY use a larger set of dependencies than these, for example every path that appears in the expression regardless of which branch runs, or every signal. It MUST NOT use a smaller set.

10Handlers

A handler is a JSON object with at least one of patch (an object) and action (a non-empty string). With action it MAY also have payload (any JSON value) and indicator (a signal path whose first segment begins with _). Handlers with none of patch or action, or with an invalid indicator, are invalid; an element with an invalid handler is rejected.

When the user triggers a handler, the renderer MUST perform these steps in order:

  1. If the element or any ancestor is disabled or not visible, stop.
  2. If the handler has patch, evaluate its expressions against the current signals and apply the resulting merge patch.
  3. If the handler has action and the transport is not open, stop.
  4. If the handler has indicator, write true at that path.
  5. If the handler has action, send an action message.

The renderer MUST NOT clear an indicator itself. The server is expected to clear it, usually in the same batch as its response.

The renderer MUST NOT change the display in anticipation of the server's response, beyond the effects of the patch and indicator. Servers SHOULD respond to every action with at least the indicator reset, if one was set.

11Transports

A transport carries lines in both directions, reliably and in order. Which transport is used, how a session is authenticated, and how the renderer finds the server are decided by the host application. This section defines mappings for common transports so that independent implementations interoperate.

Byte streams
Unix domain sockets, TCP, TLS, standard input and output of a child process, QUIC or iroh bidirectional streams: lines are written directly to the stream in both directions.
WebSocket
Each text frame contains one or more complete lines. A line MUST NOT span frames. The final line feed in a frame MAY be omitted. Binary frames are not used.
HTTP with Server-Sent Events
Server to renderer: a text/event-stream response in which each SSE event has its event field set to the tag and a single data field containing the JSON. Renderer to server: each message is sent as the body of an HTTP POST, as a single line. The URLs, and how the POST is associated with the stream, are application-defined. Because the hello is sent by POST, servers SHOULD accept hello data in the request that opens the stream (for example as a query parameter or header) and treat that as the hello.
In-process
A host that runs server and renderer in the same process MAY pass messages as values without serializing them, provided the semantics are identical.

12Errors and compatibility

  1. A renderer MUST NOT terminate the session because of an invalid message. It ignores the message and, if it has a diagnostic channel (a log, a developer console), reports it there.
  2. A line that does not match the grammar, or whose JSON is not an object, is ignored.
  3. A message with an unknown tag is ignored. This allows messages to be added without breaking existing implementations.
  4. Unknown attributes on an element are ignored. Unknown members in hello and action are ignored by the server.
  5. An unknown enumeration value is treated as the attribute's default.
  6. An element with an unknown type is rejected as in 7.1. A renderer MAY display a placeholder in its place in development builds.

There is no error message from server to renderer or from renderer to server. Application errors are shown to the user with ordinary elements, such as a notice.

13Security considerations

14Appendix: grammar

In ABNF (RFC 5234), over Unicode code points:

stream  = *( line / empty )
line    = tag SP json [CR] LF
empty   = [CR] LF
tag     = 1*%x61-7A                 ; a-z
json    = <a JSON object per RFC 8259, containing no LF>

Tags defined by this specification:

Tag Direction Section
element server → renderer 5.1
remove server → renderer 5.2
signal server → renderer 5.3
hello renderer → server 6.1
action renderer → server 6.2