1.1   Resolving Server Names

Each matrix homeserver is identified by a server name consisting of a DNS name and an optional TLS port.

server_name = dns_name [ ":" tls_port]
dns_name = <host, see [RFC 3986], Section 3.2.2>
tls_port = *DIGIT

If the port is present then the server is discovered by looking up an AAAA or A record for the DNS name and connecting to the specified TLS port. If the port is absent then the server is discovered by looking up a _matrix._tcp SRV record for the DNS name. If this record does not exist then the server is discovered by looking up an AAAA or A record on the DNS name and taking the default fallback port number of 8448. Homeservers may use SRV records to load balance requests between multiple TLS endpoints or to failover to another endpoint if an endpoint fails.

1.2   Retrieving Server Keys

{
    "old_verify_keys": {
        "ed25519:auto1": {
            "expired_ts": 922834800000,
            "key": "Base+64+Encoded+Old+Verify+Key"
        }
    },
    "server_name": "example.org",
    "signatures": {
        "example.org": {
            "ed25519:auto2": "Base+64+Encoded+Signature"
        }
    },
    "tls_fingerprints": [
        {
            "sha256": "Base+64+Encoded+SHA-256-Fingerprint"
        }
    ],
    "valid_until_ts": 1052262000000,
    "verify_keys": {
        "ed25519:auto2": {
            "key": "Base+64+Encoded+Signature+Verification+Key"
        }
    }
}

GET /_matrix/key/v2/query/${server_name}/${key_id}/?minimum_valid_until_ts=${minimum_valid_until_ts} HTTP/1.1

POST /_matrix/key/v2/query HTTP/1.1
Content-Type: application/json

{
    "server_keys": {
        "$server_name": {
            "$key_id": {
                "minimum_valid_until_ts": $posix_timestamp
            }
        }
    }
}

HTTP/1.1 200 OK
Content-Type: application/json
{
    "server_keys": [
       # List of responses with same format as /_matrix/key/v2/server
       # signed by both the originating server and this server.
    ]
}

{
    "server_name": "example.org",
    "signatures": {
        "example.org": {
            "ed25519:auto": "Base+64+Encoded+Signature"
        }
    },
    "tls_certificate": "Base+64+Encoded+DER+Encoded+X509+TLS+Certificate"
    "verify_keys": {
        "ed25519:auto": "Base+64+Encoded+Signature+Verification+Key"
    }
}

2.1   Transaction Fields

{
 "transaction_id":"916d630ea616342b42e98a3be0b74113",
 "ts":1404835423000,
 "origin":"red",
 "prev_ids":["e1da392e61898be4d2009b9fecce5325"],
 "pdus":[...],
 "edus":[...]
}

The prev_ids field contains a list of previous transaction IDs that the origin server has sent to this destination. Its purpose is to act as a sequence checking mechanism - the destination server can check whether it has successfully received that Transaction, or ask for a re-transmission if not.

The pdus field of a transaction is a list, containing zero or more PDUs.[*] Each PDU is itself a JSON object containing a number of keys, the exact details of which will vary depending on the type of PDU. Similarly, the edus field is another list containing the EDUs. This key may be entirely absent if there are no EDUs to transfer.

(* Normally the PDU list will be non-empty, but the server should cope with receiving an "empty" transaction, as this is useful for informing peers of other transaction IDs they should be aware of. This effectively acts as a push mechanism to encourage peers to continue to replicate content.)

3.1   Required PDU Fields

{
 "context":"#example:green.example.com",
 "origin":"green.example.com",
 "pdu_id":"a4ecee13e2accdadf56c1025af232176",
 "origin_server_ts":1404838188000,
 "pdu_type":"m.room.message",
 "prev_pdus":[
   ["blue.example.com","99d16afbc8",
       {"sha256":"abase64encodedsha256hashshouldbe43byteslong"}]
 ],
 "hashes":{"sha256":"thishashcoversallfieldsincasethisisredacted"},
 "signatures":{
   "green.example.com":{
     "ed25519:key_version:":"these86bytesofbase64signaturecoveressentialfieldsincludinghashessocancheckredactedpdus"
   }
 },
 "is_state":false,
 "content": {...}
}

In contrast to Transactions, it is important to note that the prev_pdus field of a PDU refers to PDUs that any origin server has sent, rather than previous IDs that this origin has sent. This list may refer to other PDUs sent by the same origin as the current one, or other origins.

Because of the distributed nature of participants in a Matrix conversation, it is impossible to establish a globally-consistent total ordering on the events. However, by annotating each outbound PDU at its origin with IDs of other PDUs it has received, a partial ordering can be constructed allowing causality relationships to be preserved. A client can then display these messages to the end-user in some order consistent with their content and ensure that no message that is semantically in reply of an earlier one is ever displayed before it.

3.2   State Update PDU Fields

PDUs fall into two main categories: those that deliver Events, and those that synchronise State. For PDUs that relate to State synchronisation, additional keys exist to support this:

{...,
 "is_state":true,
 "state_key":TODO-doc
 "required_power_level":TODO-doc
 "prev_state_id":TODO-doc
 "prev_state_origin":TODO-doc
}

8.1   Request Authentication

Every HTTP request made by a homeserver is authenticated using public key digital signatures. The request method, target and body are signed by wrapping them in a JSON object and signing it using the JSON signing algorithm. The resulting signatures are added as an Authorization header with an auth scheme of X-Matrix. Note that the target field should include the full path starting with /_matrix/..., including the ? and any query parameters if present, but should not include the leading https:, nor the destination server's hostname.

Step 1 sign JSON:

 {
     "method": "GET",
     "uri": "/target",
     "origin": "origin.hs.example.com",
     "destintation": "destination.hs.example.com",
     "content": { JSON content ... },
     "signatures": {
         "origin.hs.example.com": {
             "ed25519:key1": "ABCDEF..."
         }
     }
}

Step 2 add Authorization header:

GET /target HTTP/1.1
Authorization: X-Matrix origin=origin.example.com,key="ed25519:key1",sig="ABCDEF..."
Content-Type: application/json

{ JSON content ... }

Example python code:

def authorization_headers(origin_name, origin_signing_key,
                          destination_name, request_method, request_target,
                          content_json=None):
    request_json = {
         "method": request_method,
         "uri": request_target,
         "origin": origin_name,
         "destination": destination_name,
    }

    if content_json is not None:
        request["content"] = content_json

    signed_json = sign_json(request_json, origin_name, origin_signing_key)

    authorization_headers = []

    for key, sig in signed_json["signatures"][origin_name].items():
        authorization_headers.append(bytes(
            "X-Matrix origin=%s,key=\"%s\",sig=\"%s\"" % (
                origin_name, key, sig,
            )
        ))

    return ("Authorization", authorization_headers)

8.2   Response Authentication

Responses are authenticated by the TLS server certificate. A homeserver should not send a request until it has authenticated the connected server to avoid leaking messages to eavesdroppers.

8.3   Client TLS Certificates

Requests are authenticated at the HTTP layer rather than at the TLS layer because HTTP services like Matrix are often deployed behind load balancers that handle the TLS and these load balancers make it difficult to check TLS client certificates.

A homeserver may provide a TLS client certificate and the receiving homeserver may check that the client certificate matches the certificate of the origin homeserver.

14.1   Canonical JSON

Matrix events are represented using JSON objects. If we want to sign JSON events we need to encode the JSON as a binary string. Unfortunately the same JSON can be encoded in different ways by changing how much white space is used or by changing the order of keys within objects. Therefore we have to define an encoding which can be reproduced byte for byte by any JSON library.

We define the canonical JSON encoding for a value to be the shortest UTF-8 JSON encoding with dictionary keys lexicographically sorted by unicode codepoint. Numbers in the JSON must be integers in the range [-(2**53)+1, (2**53)-1].

We pick UTF-8 as the encoding as it should be available to all platforms and JSON received from the network is likely to be already encoded using UTF-8. We sort the keys to give a consistent ordering. We force integers to be in the range where they can be accurately represented using IEEE double precision floating point numbers since a number of JSON libraries represent all numbers using this representation.

import json

def canonical_json(value):
    return json.dumps(
        value,
        # Encode code-points outside of ASCII as UTF-8 rather than \u escapes
        ensure_ascii=False,
        # Remove unnecessary white space.
        separators=(',',':'),
        # Sort the keys of dictionaries.
        sort_keys=True,
        # Encode the resulting unicode as UTF-8 bytes.
    ).encode("UTF-8")

value     = false / null / true / object / array / number / string
false     = %x66.61.6c.73.65
null      = %x6e.75.6c.6c
true      = %x74.72.75.65
object    = %x7B [ member *( %x2C member ) ] %7D
member    = string %x3A value
array     = %x5B [ value *( %x2C value ) ] %5B
number    = [ %x2D ] int
int       = %x30 / ( %x31-39 *digit )
digit     = %x30-39
string    = %x22 *char %x22
char      = unescaped / %x5C escaped
unescaped = %x20-21 / %x23-5B / %x5D-10FFFF
escaped   = %x22 ; "    quotation mark  U+0022
          / %x5C ; \    reverse solidus U+005C
          / %x62 ; b    backspace       U+0008
          / %x66 ; f    form feed       U+000C
          / %x6E ; n    line feed       U+000A
          / %x72 ; r    carriage return U+000D
          / %x74 ; t    tab             U+0009
          / %x75.30.30.30 (%x30-37 / %x62 / %x65-66) ; u000X
          / %x75.30.30.31 (%x30-39 / %x61-66)        ; u001X

14.2   Signing JSON

We can now sign a JSON object by encoding it as a sequence of bytes, computing the signature for that sequence and then adding the signature to the original JSON object.

{
   "name": "example.org",
   "signing_keys": {
     "ed25519:1": "XSl0kuyvrXNj6A+7/tkrB9sxSbRi08Of5uRhxOqZtEQ"
   },
   "unsigned": {
      "age_ts": 922834800000
   },
   "signatures": {
      "example.org": {
         "ed25519:1": "s76RUgajp8w172am0zQb/iPTHsRnb4SkrzGoeCOSFfcBY2V/1c8QfrmdXHpvnc2jK5BD1WiJIxiMW95fMjK7Bw"
      }
   }
}

def sign_json(json_object, signing_key, signing_name):
    signatures = json_object.pop("signatures", {})
    unsigned = json_object.pop("unsigned", None)

    signed = signing_key.sign(encode_canonical_json(json_object))
    signature_base64 = encode_base64(signed.signature)

    key_id = "%s:%s" % (signing_key.alg, signing_key.version)
    signatures.setdefault(signing_name, {})[key_id] = signature_base64

    json_object["signatures"] = signatures
    if unsigned is not None:
        json_object["unsigned"] = unsigned

    return json_object

14.3   Signing Events

Signing events is a more complicated process since servers can choose to redact non-essential parts of an event. Before signing the event it is encoded as Canonical JSON and hashed using SHA-256. The resulting hash is then stored in the event JSON in a hash object under a sha256 key.

def hash_event(event_json_object):

    # Keys under "unsigned" can be modified by other servers.
    # They are useful for conveying information like the age of an
    # event that will change in transit.
    # Since they can be modifed we need to exclude them from the hash.
    unsigned = event_json_object.pop("unsigned", None)

    # Signatures will depend on the current value of the "hashes" key.
    # We cannot add new hashes without invalidating existing signatures.
    signatures = event_json_object.pop("signatures", None)

    # The "hashes" key might contain multiple algorithms if we decide to
    # migrate away from SHA-2. We don't want to include an existing hash
    # output in our hash so we exclude the "hashes" dict from the hash.
    hashes = event_json_object.pop("hashes", {})

    # Encode the JSON using a canonical encoding so that we get the same
    # bytes on every server for the same JSON object.
    event_json_bytes = encode_canonical_json(event_json_bytes)

    # Add the base64 encoded bytes of the hash to the "hashes" dict.
    hashes["sha256"] = encode_base64(sha256(event_json_bytes).digest())

    # Add the "hashes" dict back the event JSON under a "hashes" key.
    event_json_object["hashes"] = hashes
    if unsigned is not None:
        event_json_object["unsigned"] = unsigned
    return event_json_object

The event is then stripped of all non-essential keys both at the top level and within the content object. Any top-level keys not in the following list MUST be removed:

auth_events
depth
event_id
hashes
membership
origin
origin_server_ts
prev_events
prev_state
room_id
sender
signatures
state_key
type

A new content object is constructed for the resulting event that contains only the essential keys of the original content object. If the original event lacked a content object at all, a new empty JSON object is created for it.

The keys that are considered essential for the content object depend on the the type of the event. These are:

type is "m.room.aliases":
  aliases

type is "m.room.create":
  creator

type is "m.room.history_visibility":
  history_visibility

type is "m.room.join_rules":
  join_rule

type is "m.room.member":
  membership

type is "m.room.power_levels":
  ban
  events
  events_default
  kick
  redact
  state_default
  users
  users_default

The resulting stripped object with the new content object and the original hashes key is then signed using the JSON signing algorithm outlined below:

def sign_event(event_json_object, name, key):

    # Make sure the event has a "hashes" key.
    if "hashes" not in event_json_object:
        event_json_object = hash_event(event_json_object)

    # Strip all the keys that would be removed if the event was redacted.
    # The hashes are not stripped and cover all the keys in the event.
    # This means that we can tell if any of the non-essential keys are
    # modified or removed.
    stripped_json_object = strip_non_essential_keys(event_json_object)

    # Sign the stripped JSON object. The signature only covers the
    # essential keys and the hashes. This means that we can check the
    # signature even if the event is redacted.
    signed_json_object = sign_json(stripped_json_object)

    # Copy the signatures from the stripped event to the original event.
    event_json_object["signatures"] = signed_json_oject["signatures"]
    return event_json_object

Servers can then transmit the entire event or the event with the non-essential keys removed. If the entire event is present, receiving servers can then check the event by computing the SHA-256 of the event, excluding the hash object. If the keys have been redacted, then the hash object is included when calculating the SHA-256 instead.

New hash functions can be introduced by adding additional keys to the hash object. Since the hash object cannot be redacted a server shouldn't allow too many hashes to be listed, otherwise a server might embed illict data within the hash object. For similar reasons a server shouldn't allow hash values that are too long.