Monto Version 3 Specification, Draft 1

Abstract

This specification describes an improved iteration of the Monto protocol for Disintegrated Development Environments ([MONTO]) . These improvements allow for simpler implementations for Clients. They also make it feasible to have multiple Clients sharing a single Service, and for Services to be operated over the Internet (rather than on the local network or on a single machine).

1. Conventions and Terminology

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Other terms used in this specification are as follows.

Client
An IDE, text editor, or other software directly controlled by the end user.
Broker
A per-user piece of software that manages communication between Clients and Services.
Service
A piece of software that receives Products from a Broker, and uses them to produce other Products in response.
Message
A single JSON value sent as an HTTP request or response body.
Product
Structured data sent between Clients, Brokers, and Services.
Client Protocol
The protocol used to communicate between Clients and Brokers.
Service Protocol
The protocol used to communicate between Brokers and Services.
Monto Protocols
The Client Protocol and Service Protocol.

2. Introduction

Monto makes it easy to interface the information provided by a language’s compiler with various editors, without the compiler developer needing to implement language support for each editor. Unfortunately, the native APIs of various popular editors vary widely with respect to how well they support the features required by Monto, namely the ability to bind to ØMQ ([ZEROMQ]) and the relatively large amount of client-side “bookkeeping” to be done.

This document suggests changes to the Monto Protocols which are focused on removing the ZeroMQ requirement and simplifying the protocol that the client has to support as much as possible. As these changes are not backwards compatible with existing Clients and Services, these changes are collectively known as Monto Version 3.

3. Protocol Overview

The Monto Protocols are built on top of HTTP/2 ([RFC7540]) , with each request being a POST request to a special Monto endpoint. Both request and response bodies are JSON ([RFC7159]) . This allows for the reuse of the many technologies that are capable of debugging this relatively common protocol combination, such as mitmproxy ([MITMPROXY]) , Postman ([POSTMAN]) , and others. Furthermore, almost every mainstream programming language supports HTTP and JSON, meaning the wide variety of client programming languages (e.g. CoffeeScript, Emacs Lisp, Java, Python, etc.) can all interoperate with it.

Both the Client Protocol and Service Protocol are versioned according to Semantic Versioning ([SEMVER]) . This document describes Client Protocol version 3.0.0 and Service Protocol version 3.0.0.

Unless specified otherwise, a Message is serialized as JSON and sent with a Content-Type of application/json.

3.1. Common Messages

These Messages are shared by both the Client Protocol and the Service Protocol.

Messages are documented with JSON Schema ([JSONSCHEMA]) .

The schemas are also present in the “schemas” directory, which should accompany this document.

3.1.1. Identifier

3.1.1.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "Identifier",
	"description": "A reverse-hostname-style dotted identifier.",

	"type": "string",
	"pattern": "[a-zA-Z_][a-zA-Z_0-9]*(\\.[a-zA-Z_][a-zA-Z_0-9]*)*"
}

3.1.1.2. Example

"com.acme.foo"

3.1.2. MontoVersion

3.1.2.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "MontoVersion",
	"description": "The version number of the Monto protocol.",

	"type": "object",
	"properties": {
		"major": { "type": "integer" },
		"minor": { "type": "integer" },
		"patch": { "type": "integer" }
	},
	"additionalProperties": false,
	"required": ["major", "minor", "patch"]
}

3.1.2.2. Example

{
	"major": 3,
	"minor": 0,
	"patch": 0
}

3.1.3. NamespacedName

3.1.3.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "NamespacedName",
	"description": "A name after a dotted identifier.",

	"type": "string",
	"pattern": "[a-zA-Z_][a-zA-Z_0-9]*(\\.[a-zA-Z_][a-zA-Z_0-9]*)*/[a-zA-Z_][a-zA-Z_0-9]*"
}

3.1.3.2. Example

"com.acme.foo/bar"

3.1.4. Product

3.1.4.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "Product",
	"description": "A Product, along with its contents.",

	"type": "object",
	"properties": {
		"name": { "$ref": "ProductName.json#" },
		"language": { "$ref": "Identifier.json#" },
		"path": { "type": "string" },
		"contents": {}
	},
	"additionalProperties": false,
	"required": ["name", "contents"]
}

3.1.4.2. Example

{
	"name": "errors",
	"language": "python",
	"contents": []
}

3.1.5. ProductIdentifier

3.1.5.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ProductIdentifier",
	"description": "A Product's name and language.",

	"properties": {
		"name": { "$ref": "ProductName.json#" },
		"language": { "$ref": "Identifier.json#" },
		"path": { "type": "string" }
	},
	"additionalProperties": false,
	"required": ["name"]
}

3.1.5.2. Example

{
	"name": "errors",
	"language": "python"
}

3.1.6. ProductName

3.1.6.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ProductName",
	"description": "The name of a Product.",

	"anyOf": [
		{ "$ref": "NamespacedName.json#" },
		{
			"title": "BuiltInProductName",
			"description": "The name of a Product defined by the specification.",

			"enum": [
				"directory",
				"errors",
				"highlighting",
				"source"
			]
		}
	]
}

3.1.6.2. Example

"com.example.foo/bar"

4. The Client Protocol

The Client Protocol dictates communication between Clients and Brokers.

4.1. Connection Initiation

A Client SHALL initiate a connection to a Broker either when it starts, or when Monto capabilities are requested. Although Monto can operate over connections on any port, Clients SHOULD default to connecting to port 28888 on the current machine, and Brokers SHOULD default to serving on that port. Clients and Brokers SHOULD be able to connect to and serve on other ports, if configured to do so.

Upon initiating a connection to a Broker, a Client MUST attempt to use an HTTP/2 connection if the Client supports HTTP/2. If the Client does not, it SHALL use the same protocol, but over HTTP/1.1 ([RFC7230]) instead. If a Client is using HTTP/1.1, it MAY open multiple connections to the server in order to have multiple requests “in flight” at the same time.

4.2. Version Negotiation

After the HTTP connection is established, the Client SHALL make a POST request to the /monto/version path, with a ClientNegotiation Message as the body. The Broker SHALL check that it is compatible with the Client. The Broker SHALL respond with a ClientBrokerNegotiation Message. If the Broker is compatible with the Client, this response SHALL have an HTTP Status of 200. If the Broker and Client are not compatible, the response SHALL instead have an HTTP Status of 409.

If the HTTP Status is 200, the Client SHALL check that it is compatible with the Broker. If the HTTP Status is not 200 or the Client and Broker are not compatible as determined by the Client, the Client SHOULD inform the user and MUST close the connection.

Compatibility between versions of the Client Protocol SHALL be determined using the Semantic Versioning rules. Additionally, a Client MAY reject a Broker that is known to not follow this specification correctly, and vice versa.

If the intersection of the extensions field of the ClientNegotiation and ClientBrokerNegotiation Messages is nonempty, the corresponding extensions MUST be considered to be enabled by both the Client and the Broker. The semantics of an extension being enabled are left to that extension. All non-namespaced extensions are documented in the Client Protocol Extensions section below.

If a non-zero number of extensions are enabled, all requests from the Client to the Broker and all responses from the Broker to the Client MUST have a Monto-Extensions HTTP header with a space-separated list of the enabled extensions, sorted lexicographically. For example, if the extensions com.acme/foo and org.example/bar are enabled, the header would read Monto-Extensions: com.acme/foo org.example/bar. The header MAY be present with an empty value when no extensions are enabled.

4.3. Requesting Products

A Client SHALL request Products by making a POST request to the /monto/products path, with a ClientRequest Message as the body.

If the ClientRequest Message contains requests for Products which a Service does not expose, or a request for Products from a Service that does not exist, the Broker SHALL respond with an HTTP Status of 400 and the ClientSingleRequest Message corresponding to the illegal request as the body.

Otherwise, the Broker SHALL respond with an HTTP Status of 200 and a BrokerResponse Message as the body. Each member of the BrokerResponse Message corresponds to one of the requests from the ClientRequest Message. No particular order is enforced; a Client MUST be able to handle a BrokerResponse Message whose elements have a different order from the requests in the ClientRequest Message.

When a Service responds to the Broker with an HTTP Status of 200, the corresponding BrokerSingleResponse MUST be a BrokerProductResponse. Conversely, when the Service responds to the Broker with a ServiceErrors, the BrokerSingleResponse MUST be a BrokerErrorResponse. If another error occurs while retrieving the Product, the BrokerSingleResponse MUST be a BrokerErrorResponse. The value field SHOULD state that the error came from the Broker rather than the Service.

4.4. Client Protocol Messages

4.4.1. ClientNegotiation

4.4.1.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ClientNegotiation",
	"description": "The Message that a Client sends to a Broker during version negotiation.",

	"type": "object",
	"properties": {
		"monto": { "$ref": "MontoVersion.json#" },
		"client": {
			"title": "ClientVersion",
			"description": "The implementation and version of the Client.",

			"type": "object",
			"properties": {
				"id": { "$ref": "Identifier.json#" },
				"name": { "type": "string" },
				"vendor": { "type": "string" },
				"major": { "type": "integer" },
				"minor": { "type": "integer" },
				"patch": { "type": "integer" }
			},
			"additionalProperties": false,
			"required": ["id"]
		},
		"extensions": {
			"title": "ClientExtensions",
			"description": "The Client extensions supported by the Client.",

			"type": "array",
			"minItems": 1,
			"items": { "$ref": "NamespacedName.json#" },
			"uniqueItems": true
		}
	},
	"additionalProperties": false,
	"required": ["monto", "client"]
}

4.4.1.2. Example

{
	"monto": {
		"major": 3,
		"minor": 0,
		"patch": 0
	},
	"client": {
		"id": "com.acme.foo",
		"name": "Foo Client",
		"vendor": "ACME Inc.",
		"major": 0,
		"minor": 1,
		"patch": 0
	},
	"extensions": [
		"com.acme.foo/cool_thing"
	]
}

4.4.2. ClientBrokerNegotiation

4.4.2.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ClientBrokerNegotiation",
	"description": "The Message that a Broker sends to a Client during version negotiation.",

	"type": "object",
	"properties": {
		"monto": { "$ref": "MontoVersion.json#" },
		"broker": {
			"title": "BrokerVersion",
			"description": "The implementation and version of the Broker.",

			"type": "object",
			"properties": {
				"id": { "$ref": "Identifier.json#" },
				"name": { "type": "string" },
				"vendor": { "type": "string" },
				"major": { "type": "integer" },
				"minor": { "type": "integer" },
				"patch": { "type": "integer" }
			},
			"additionalProperties": false,
			"required": ["id"]
		},
		"extensions": {
			"title": "ClientBrokerExtensions",
			"description": "The Client extensions supported by the Broker.",

			"type": "array",
			"minItems": 1,
			"items": { "$ref": "NamespacedName.json#" },
			"uniqueItems": true
		},
		"services": {
			"title": "BrokerServices",
			"description": "The Services the Broker is connected to.",

			"type": "array",
			"items": { "$ref": "ServiceNegotiation.json#" },
			"uniqueItems": true
		}
	},
	"additionalProperties": false,
	"required": ["monto", "broker", "services"]
}

4.4.2.2. Example

{
	"monto": {
		"major": 3,
		"minor": 0,
		"patch": 0
	},
	"broker": {
		"id": "com.acme.bar",
		"name": "Bar Broker",
		"vendor": "ACME Inc.",
		"major": 0,
		"minor": 1,
		"patch": 0
	},
	"extensions": [
		"com.acme.foo/cool_thing",
		"com.acme.bar/other_thing"
	],
	"services": [
		{
			"monto": {
				"major": 3,
				"minor": 0,
				"patch": 0
			},
			"service": {
				"id": "com.acme.quux",
				"name": "Quux Service",
				"vendor": "ACME Inc.",
				"major": 0,
				"minor": 1,
				"patch": 0
			},
			"products": []
		}
	]
}

4.4.3. ClientRequest

4.4.3.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ClientRequest",
	"description": "A request for Products from a Client to a Broker.",

	"type": "object",
	"properties": {
		"products": {
			"title": "ClientProducts",
			"description": "Products being provided along with the request.",

			"type": "array",
			"items": { "$ref": "Product.json#" },
			"uniqueItems": true
		},
		"requests": {
			"title": "ClientRequests",
			"description": "The actual requests.",

			"type": "array",
			"items": { "$ref": "ClientSingleRequest.json#" },
			"uniqueItems": true
		}
	},
	"additionalProperties": false,
	"required": ["requests"]
}

4.4.3.2. Example

{
	"requests": [
		{
			"product": {
				"name": "source",
				"language": "python",
				"path": "/projects/hello/hello_world.py"
			},
			"service": "com.acme.quux"
		}
	],
	"products": [
		{
			"name": "source",
			"language": "python",
			"path": "/projects/hello/hello_world.py",
			"contents": "print(\"Hello, world!\")"
		}
	]
}

4.4.4. ClientSingleRequest

4.4.4.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ClientSingleRequest",
	"description": "A single request for a Product from a service.",

	"type": "object",
	"properties": {
		"product": { "$ref": "ProductIdentifier.json#" },
		"service": { "$ref": "Identifier.json#" }
	},
	"additionalProperties": false,
	"required": ["product", "service"]
}

4.4.4.2. Example

{
	"product": {
		"name": "errors",
		"language": "python"
	},
	"service": "com.acme.quux"
}

4.4.5. BrokerResponse

4.4.5.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "BrokerResponse",
	"description": "A response containing Products from the Broker to a Client.",

	"type": "array",
	"items": {
		"title": "BrokerSingleResponse",
		"description": "A single response from a Service.",
		
		"oneOf": [
			{
				"title": "BrokerErrorResponse",
				"description": "A BrokerSingleResponse representing an error.",

				"type": "object",
				"properties": {
					"type": { "const": "error" },
					"value": { "$ref": "ServiceErrors.json#" }
				},
				"required": ["type", "value"],
				"additionalProperties": false
			},
			{
				"title": "BrokerProductResponse",
				"description": "A BrokerSingleResponse representing the successful retrieval of a Product.",

				"type": "object",
				"properties": {
					"type": { "const": "product" },
					"value": { "$ref": "Product.json#" }
				},
				"required": ["type", "value"],
				"additionalProperties": false
			}
		]
	}
}

4.4.5.2. Example

[
	{
		"type": "product",
		"value": {
			"name": "errors",
			"language": "python",
			"contents": []
		}
	}
]

4.5. Client Protocol Extensions

Currently, there are no built-in extensions defined for the Client Protocol. However, a Client or Broker MAY support arbitrary extensions whose names are in the form of the NamespacedName above.

5. The Service Protocol

The Service Protocol dictates communication between Brokers and Services.

5.1. Connection Initiation

A Broker SHALL initiate a connection to the Services requested by the user when it starts. Brokers MUST be able to connect to a Service on any port, and Services MUST be able to serve on any port.

Brokers MUST first attempt to use HTTP/2, and MAY support HTTP/1.1 as well. Services SHOULD support HTTP/2 if at all possible, as the pipelining it allows is more useful than for the Client Protocol, as it is more likely that there are several in-flight requests at once.

5.2. Version Negotiation

After the HTTP connection is established, the Broker SHALL make a POST request to the /monto/version path, with a ServiceBrokerNegotiation Message as the body. The Service SHALL check that it is compatible with the Broker. The Service SHALL respond with a ServiceNegotiation Message. If the Service is compatible with the Broker, this response SHALL have an HTTP Status of 200. If the Service and Broker are not compatible, the response SHALL instead have an HTTP Status of 409.

If the HTTP Status is 200, the Broker SHALL check that it is compatible with the Service. If the HTTP Status is not 200 or the Broker and Service are not compatible as determined by the Broker, the Broker MUST close the connection. In this situation, the Broker SHOULD log this event, and MAY choose to ignore the Service’s existence.

Compatibility between versions of the Service Protocol SHALL be determined using the Semantic Versioning rules. Additionally, a Broker MAY reject a Service that is known to not follow this specification correctly, and vice versa.

If the intersection of the extensions field of the ServiceBrokerNegotiation and ServiceNegotiation Messages is nonempty, the corresponding extensions MUST be considered to be enabled by both the Client and the Broker. The semantics of an extension being enabled are left to that extension. All non-namespaced extensions are documented in the Service Protocol Extensions section below.

If a non-zero number of extensions are enabled, all requests from the Broker to the Service and all responses from the Service to the Broker MUST have a Monto-Extensions HTTP header with a space-separated list of the enabled extensions, sorted lexicographically. For example, if the extensions com.acme/foo and org.example/bar are enabled, the header would read Monto-Extensions: com.acme/foo org.example/bar. The header MAY be present with an empty value when no extensions are enabled.

5.3. Requesting Products

The broker SHALL request a Product from a Service by making a POST request to the /monto/service path, with a BrokerRequest as the body.

If the BrokerRequest Message contains a request for a Product which the Service does not expose, the Service MUST respond with an HTTP Status of 400 with the ProductIdentifier of the failed BrokerRequest as the body.

If the Service is unable to create the requested Product from the Products present in the BrokerRequest, the Service MUST respond with an HTTP Status of 500 and a ServiceErrors Message using the ServiceErrorUnmetDependency variant as the body.

If the Service encounters some other error, the Service MUST respond with an HTTP Status of 500 and a ServiceErrors Message using the ServiceErrorOther variant as the body.

Otherwise, the Service MUST respond with an HTTP Status of 200 and a ServiceProduct Message containing the requested Product as the Body.

5.4. Service Protocol Messages

5.4.1. ServiceBrokerNegotiation

5.4.1.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ServiceBrokerNegotiation",
	"description": "The Message that a Broker sends to a Service during version negotiation.",

	"type": "object",
	"properties": {
		"monto": { "$ref": "MontoVersion.json#" },
		"broker": {
			"title": "BrokerVersion",
			"description": "The implementation and version of the Broker.",

			"type": "object",
			"properties": {
				"id": { "$ref": "Identifier.json#" },
				"name": { "type": "string" },
				"vendor": { "type": "string" },
				"major": { "type": "integer" },
				"minor": { "type": "integer" },
				"patch": { "type": "integer" }
			},
			"additionalProperties": false,
			"required": ["id"]
		},
		"extensions": {
			"title": "ServiceBrokerExtensions",
			"description": "The Service extensions supported by the Broker.",

			"type": "array",
			"minItems": 1,
			"items": { "$ref": "NamespacedName.json#" },
			"uniqueItems": true
		}
	},
	"additionalProperties": false,
	"required": ["monto", "broker"]
}

5.4.1.2. Example

{
	"monto": {
		"major": 3,
		"minor": 0,
		"patch": 0
	},
	"broker": {
		"id": "com.acme.bar",
		"name": "Bar Broker",
		"vendor": "ACME Inc.",
		"major": 0,
		"minor": 1,
		"patch": 0
	},
	"extensions": [
		"com.acme.foo/cool_thing",
		"com.acme.bar/other_thing"
	]
}

5.4.2. ServiceNegotiation

5.4.2.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ServiceNegotiation",
	"description": "The Message that a Service sends to a Broker during version negotiation.",

	"type": "object",
	"properties": {
		"monto": { "$ref": "MontoVersion.json#" },
		"service": {
			"title": "ServiceVersion",
			"description": "The implementation and version of the Service.",

			"type": "object",
			"properties": {
				"id": { "$ref": "Identifier.json#" },
				"name": { "type": "string" },
				"vendor": { "type": "string" },
				"major": { "type": "integer" },
				"minor": { "type": "integer" },
				"patch": { "type": "integer" }
			},
			"additionalProperties": false,
			"required": ["id"]
		},
		"extensions": {
			"title": "ServiceExtensions",
			"description": "The Service extensions supported by the Service.",

			"type": "array",
			"minItems": 1,
			"items": { "$ref": "NamespacedName.json#" },
			"uniqueItems": true
		},
		"products": {
			"title": "BrokerProductList",
			"description": "The Products a Service can produce.",

			"type": "array",
			"items": { "$ref": "ProductIdentifier.json#" },
			"uniqueItems": true
		}
	},
	"additionalProperties": false,
	"required": ["monto", "service", "products"]
}

5.4.2.2. Example

{
	"monto": {
		"major": 3,
		"minor": 0,
		"patch": 0
	},
	"service": {
		"id": "com.acme.quux",
		"name": "Quux Service",
		"vendor": "ACME Inc.",
		"major": 0,
		"minor": 1,
		"patch": 0
	},
	"products": [
		{
			"name": "errors",
			"language": "python"
		},
		{
			"name": "com.acme/styleguide",
			"language": "python"
		}
	],
	"extensions": [
		"com.acme.quux/incremental"
	]
}

5.4.3. BrokerRequest

5.4.3.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "BrokerRequest",
	"description": "A request for a Product from the Broker to a Service.",

	"type": "object",
	"properties": {
		"products": {
			"title": "BrokerProducts",
			"description": "Products being provided along with the request.",

			"type": "array",
			"items": { "$ref": "Product.json#" },
			"uniqueItems": true
		},
		"request": { "$ref": "ProductIdentifier.json#" }
	},
	"additionalProperties": false,
	"required": ["request"]
}

5.4.3.2. Example

{
	"products": [],
	"request": {
		"name": "errors",
		"language": "python"
	}
}

5.4.4. ServiceErrors

5.4.4.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ServiceErrors",
	"description": "Errors encountered by a Service.",

	"type": "object",
	"properties": {
		"errors": {
			"type": "array",
			"items": { "$ref": "#/definitions/error" },
			"minItems": 1
		},
		"notices": {
			"type": "array",
			"items": { "$ref": "ServiceNotice.json#" }
		}
	},
	"required": ["errors"],
	"additionalProperties": false,

	"definitions": {
		"error": {
			"oneOf": [
				{
					"title": "ServiceErrorUnmetDependency",
					"description": "An error representing a dependency not being present.",
	
					"type": "object",
					"properties": {
						"type": { "const": "unmet_dependency" },
						"value": { "$ref": "ProductIdentifier.json#" }
					},
					"required": ["type", "value"],
					"additionalProperties": false
				},
				{
					"title": "ServiceErrorOther",
					"description": "A miscellaneous error.",
	
					"type": "object",
					"properties": {
						"type": { "const": "other" },
						"value": { "type": "string" }
					},
					"required": ["type", "value"],
					"additionalProperties": false
				}
			]
		}
	}
}

5.4.4.2. Example

{
	"errors": [
		{
			"type": "unmet_dependency",
			"value": {
				"name": "source",
				"path": "/home/example/foo.py",
				"language": "python"
			}
		},
		{
			"type": "unmet_dependency",
			"value": {
				"name": "source",
				"path": "/home/example/bar.py",
				"language": "python"
			}
		}
	],
	"notices": []
}

5.4.5. ServiceProduct

5.4.5.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ServiceProduct",
	"description": "A response containing Products from a Service to the Broker.",

	"type": "object",
	"properties": {
		"product": { "$ref": "Product.json#" },
		"notices": {
			"type": "array",
			"items": { "$ref": "ServiceNotice.json#" }
		}
	},
	"required": ["product"],
	"additionalProperties": false
}

5.4.5.2. Example

{
	"product": {
		"name": "errors",
		"language": "python",
		"contents": []
	},
	"notices": [
		{
			"type": "unused_dependency",
			"product": {
				"name": "errors",
				"language": "python",
				"path": "/home/example/old_file.py"
			}
		}
	]
}

5.4.6. ServiceNotice

5.4.6.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "ServiceNotice",
	"description": "A message from a Broker to the Service signalling a non-error special condition.",

	"oneOf": [
		{
			"title": "ServiceNoticeUnusedDependency",
			"description": "A notice that a dependency was unused when producing a Product.",

			"type": "object",
			"properties": {
				"type": { "const": "unused_dependency" },
				"product": { "$ref": "ProductIdentifier.json#" }
			},
			"additionalProperties": false,
			"required": ["type", "product"]
		}
	]
}

5.4.6.2. Example

{
	"type": "unused_dependency",
	"product": {
		"name": "errors",
		"language": "haskell",
		"path": "/home/example/.xmonad/xmonad.hs"
	}
}

5.5. Optimizations

The naïve Broker dependency-resolution algorithm is rather inefficient, and can be optimized in several ways.

5.5.1. Caching the Dependency Graph

Typically, a source file’s current dependencies are very similar to its past dependencies. This can be taken advantage of by caching the dependencies of a specific Product and requesting its dependencies before requesting the final Product. This potentially could result in more work being done that needed (as a no-longer needed dependency is still computed). Most edit actions don’t affect the dependency graph, though, so this approach is efficient in the general case.

5.5.2. Inferring Dependencies

The semantics specified here intentionally do not specify in what order the Broker should query Services to obtain a Product. This allows a Broker to attempt to infer the dependencies a particular Product will have. Although no heuristics are described here, they could in principle be developed and applied.

5.5.3. Caching Products

Most projects’ dependencies contains more source code than the projects themselves – the dependencies’ source code is unlikely to change, and it is wasteful to send it over the network with each request. A simple Service Protocol Extension that remedies this problem would be a caching mechanism, in which the Broker would send an opaque identifier (e.g. a SHA-256 hash) for the dependencies instead of the dependencies themselves. A Service could then either use a cached copy, if one exists, or fail with an error similar to the ServiceErrorUnmetDependency error if the dependency is not in the cache.

5.5.4. Incremental Product Transfer

The next step on the above would be to send all changes to files as deltas from a previous state. This would greatly decrease the amount of network bandwidth required, and would be a relatively minor variation on the above.

5.5.5. Incremental Compilation

Once an incremental transfer system exists, full incremental compilation is easy to support. A Service would only have to cache the last Product cooresponding to the input, and then could use it in the next compilation for that Product.

5.6. Service Protocol Extensions

Currently, there are no built-in extensions defined for the Service Protocol. However, a Broker or Service MAY support arbitrary extensions whose names are in the form of the NamespacedName above.

6. Products

Some fields are in common between most Product types. The main two are startByte and endByte. They represent the start and end of a selection of text from the corresponding source code. startByte is inclusive, while endByte is exclusive. The usage of “byte” in their names is significant – these MUST be the the byte indexes, rather than the character indexes.

6.1. directory

6.1.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "directory",
	"description": "A listing of a directory.",

	"type": "array",
	"items": {
		"type": "object",
		"properties": {
			"name": { "type": "string" },
			"absolutePath": { "type": "string" },
			"type": {
				"oneOf": [
					{ "const": "file" },
					{ "const": "directory" },
					{ "const": "symlink" },
					{ "const": "other" }
				]
			}
		},
		"required": ["name", "absolutePath", "type"],
		"additionalProperties": false
	}
}

6.1.2. Example

[
	{
		"name": "foo.c",
		"absolutePath": "/projects/bar/foo.c",
		"type": "file"
	},
	{
		"name": "bar.c",
		"absolutePath": "/projects/bar/bar.c",
		"type": "file"
	},
	{
		"name": "quux",
		"absolutePath": "/projects/bar/quux",
		"type": "directory"
	}
]

6.2. errors

6.2.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "errors",
	"description": "Syntactic or semantic errors detected in source code.",

	"type": "array",
	"items": {
		"type": "object",
		"properties": {
			"message": { "type": "string" },
			"startByte": { "$ref": "#/definitions/byteOffset" },
			"endByte": { "$ref": "#/definitions/byteOffset" },
			"severity": {
				"oneOf": [
					{ "const": "error" },
					{ "const": "warning" },
					{ "const": "info" }
				]
			}
		},
		"required": ["message", "startByte", "endByte"],
		"additionalProperties": false
	},

	"definitions": {
		"byteOffset": {
			"type": "integer",
			"minimum": 0
		}
	}
}

6.2.2. Example

[
	{
		"message": "Invalid import: foo.bar",
		"startByte": 0,
		"endByte": 19
	}
]

6.3. highlighting

6.3.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "highlighting",
	"description": "Token information to be used for highlighting source code.",

	"type": "array",
	"items": {
		"type": "object",
		"properties": {
			"startByte": { "$ref": "#/definitions/byteOffset" },
			"endByte": { "$ref": "#/definitions/byteOffset" },
			"token": { "$ref": "#/definitions/token" }
		},
		"required": ["startByte", "endByte", "token"],
		"additionalProperties": false
	},

	"definitions": {
		"byteOffset": {
			"type": "integer",
			"minimum": 0
		},
		"token": {
			"oneOf": [
				{ "const": "comment" },
				{ "const": "function" },
				{ "const": "identifier" },
				{ "const": "keyword" },
				{ "const": "literal" },
				{ "const": "operator" },
				{ "const": "punctuation" },
				{ "const": "type" }
			]
		}
	}
}

6.3.2. Example

[
	{
		"startByte": 0,
		"endByte": 1,
		"token": "punctuation"
	},
	{
		"startByte": 1,
		"endByte": 5,
		"token": "keyword"
	},
	{
		"startByte": 6,
		"endByte": 7,
		"token": "punctuation"
	},
	{
		"startByte": 7,
		"endByte": 10,
		"token": "identifier"
	},
	{
		"startByte": 10,
		"endByte": 11,
		"token": "punctuation"
	},
	{
		"startByte": 14,
		"endByte": 15,
		"token": "literal"
	},
	{
		"startByte": 15,
		"endByte": 16,
		"token": "punctuation"
	}
]

6.4. source

6.4.1. Schema

{
	"$schema": "http://json-schema.org/draft-06/schema#",
	"title": "source",
	"description": "Source code.",

	"type": "string"
}

6.4.2. Example

"(define (hello)\n  (println \"Hello, world!\"))\n\n(hello)\n"

7. Security Considerations

7.1. Remote Access To Local Files

The Broker sends arbitrary files to Services, which may be running on a different machine. A malicious Service could therefore request a sensitive file (for example, ~/.ssh/id_rsa). As a result, a Broker MAY claim such a file does not exist.

Furthermore, a security-conscious user MAY run the Broker in a virtual machine or container, only giving access to user files in specific directories.

7.2. Encrypted Transport

HTTP/2 optionally supports TLS encryption. Most HTTP/2 implementations require encryption, so Clients, Brokers, and Services SHOULD support TLS encryption. Due to the relative difficulty of obtaining a TLS certificate for a local Service, Clients SHOULD support connecting to a Broker that does not support TLS or uses a self-signed certificate.

8. Further Work

8.1. Binary Encoding instead of JSON

A speed boost could potentially be gained by using CBOR ([RFC7049]) , MessagePack ([MSGPACK]) or a similar format instead of JSON. This could be added as a simple protocol extension.

8.2. Asynchronous Communication

Re-adding support for asynchronous communication between Clients and Brokers as a protocol extension could be implemented as a protocol extension either by polling, which is relatively efficient in HTTP/2, or with a chunked response in HTTP/1.1.

8.3. Commands

Previous versions of Monto supported arbitrary commands being run by the Service, for example, renaming a function everywhere it appears (in all files). This is difficult to do while allowing Services to be run on remote machines. It could be achieved by allowing Services to request file writes in addition to reads, but would probably require a large amount of overhead, and come with its own security risks.

8.4. Stateful Services

Some services inherently have state, such as debuggers. Unfortunately, this model does not translate well to the Monto Version 3 Protocol – services may be shared with multiple brokers over the network. A possible solution would be to use a “state token,” a unique identifier for each session.

9. References

9.1. Normative References

[JSONSCHEMA]: Wright, A., Ed., and H. Andrews, Ed., “JSON Schema: A Media Type for Describing JSON Documents”, draft-wright-json-schema-01, April 2017.

[MITMPROXY]: “mitmproxy - home”, https://mitmproxy.org/.

[MONTO]: Keidel, S., Pfeiffer, W., and S. Erdweg., “The IDE Portability Problem and Its Solution in Monto”, doi:10.1145/2997364.2997368, November 2016.

[MSGPACK]: Furuhashi, S., “MessagePack: It’s like JSON, but fast and small.“, https://msgpack.org/.

[POSTMAN]: “Postman | Supercharge your API workflow”, https://www.getpostman.com/.

[RFC2119]: Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels”, BCP 14, RFC 2119, March 1997.

[RFC7049]: Bormann, C., and P. Hoffman, “Concise Binary Object Representation (CBOR)”, RFC 7049, October 2013.

[RFC7159]: Bray, T., “The JavaScript Object Notation (JSON) Data Interchange Format”, RFC 7159, March 2014.

[RFC7230]: Fielding, R., Ed., and J. Reschke, Ed., “Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing”, RFC 7230, June 2014.

[RFC7540]: Belshe, M., Peon, R., and M. Thomson, Ed., “Hypertext Transfer Protocol Version 2 (HTTP/2)”, RFC 7540, May 2015.

[SEMVER]: “Semantic Versioning 2.0.0”, http://semver.org/spec/v2.0.0.html.

[ZEROMQ]: “Distributed Messaging - zeromq”, http://zeromq.org/.