Backend protocol

This section describes the protocol used by DISCOS to communicate with external backends. Here you can find the protocol definition, the grammar definition and a simple example implementation of the protocol server as a twisted protocol. This definition is liberally inspired by the one used by KAT telescope for device communication ([KAT]) as we found that work of really good quality.

Protocol Version	1.2
Last revision	25/01/2016
Status	PRODUCTION

Introduction

DISCOS BACKEND PROTOCOL consits of newline separated text messages sent synchronously over a TCP/IP connection. Messages can be of two kinds: request and reply. Requests are sent from a client to a server, while replies are sent from the server in response to a client’s request. Each request must receive one and only one reply message back and in this kind of topology the client will be tipically represented by a DISCOS component, while the server is represented by the backend controller. This version of the protocol does not impose any constraint to the number of clients connecting to a server but leaves to the clients the responsibility of orchestrating their requests in a consistent way. Implementations consisting of a single client will be the first and foremost ones.

Here you can find a quick list of the requests defined by the protocol, which will be better described in the next sections:

Request	Description
status	get the backend status code
version	get the server protocol version
get-configuration	get the backend actual configuration
set-configuration	set a new backend configuration
get-integration	get the backend integration time
set-integration	set a new integration time
get-tpi	ask the backend a total power level read
get-tp0	ask the backend a zero level read
time	get the backend time
start [timestamp]	start the acquisition [at a given time]
stop [timestamp]	stop the acquisition [at a given time]
set-section	configure a section of the backend
cal-on [interleave=0]	(de)activate the calibration mark
set-filename	tell the backend the output file name
convert-data	instructs the backend to convert file

Handshake

The protocol is a connected protocol. This means that a client must establish a connection with the server and keep it open for message exchange. In this preliminary version a very simple handshake is required: on every new connection the server will write to the client a REPLY message containing the version of the protocol that the server is implementing. Further decisions are left to the client.

Example:

client.open_connection( server_ip, server_port )

server -> client
reply: "!version,1.0.1\r\n"

client now can continue communication or close it because it detects an incomptible server version

Note that the message is exactly the same one published in reply to the version command.

Messaging Protocol

Communication consists of a number of messages, each message consisting of a line of text. The protocol supports requests and replies messages, identified by the first character of the message: requests are indicated by ‘?’, replies by ‘!’. Each message is made up of a type code followed by the message name, and optionally by one or many message arguments; name and arguments are separated one each other using the character ‘,’ .

A reply is necessary for every request, however the nature of the reply may change depending on the request:

The reply message should have the same name as the request message.

The first argument of a reply message should always be a return code.

A return code of ok indicates successful processing of the request, while anything else indicates failure.

The recommended failure strings are invalid (for malformed requests) and fail (for valid requests which could not be processed).

On success, further reply arguments are specific to the type of request.

In the case of failure a second argument should describe the failure in more detail and in human-readable form.

Return Code	Description
ok	Request successfully processed. Further arguments are request specific.
invalid	Request malformed. Second argument is a human-readable description of the error.
fail	Valid request that could not be processed. Second argument is a human-readable description of the error.

Line Separation

Each message is terminated by the sequence CR LF (carriage-return + line feed) as specified by the TELNET standad ([TELNET]) . This will make the protocol easily usable also for debug purposes using simple telnet clients.

Message Grammar

The message grammar is described in extended BNF [EBNF] where:

Optional items are enclosed in square brackets

Items repeating 0 or more times are suffixed with a *

Items repeating 1 or more times are suffixed with a +

Alternative choices in a production are separated by the ‘|’ symbol

Set difference is indicated by the ‘/’ symbol

The grammar is defined as:

  <message> ::= <type> <name> <arguments> <eol>
     <type> ::= "?" | "!"
     <name> ::= alpha (alpha | digit | "-")*
<arguments> ::= ("" | <separator> <argument>) [<arguments>]
      <eol> ::= newline
<separator> ::= ","
 <argument> ::= (<plain> | <escape>)+
    <plain> ::= character / <special>
  <special> ::= backslash | null | newline | carriage-return | escape | tab
   <escape> ::= "\" <escapecode>
<escapecode>::= "\" | "t" | <separator>

Note that arguments can contain spaces and tabs and are limited only by commas and newlines at the end of the message.

Data Types

Being the protocol string based, whenever we need to transmit other data types they must be encoded into strings in an unequivocable manner. This is defined as per the table below:

integer as formatted by printf(“%d”,i). i.e. 10 -15

float as formatted by printf(“%f”,f) i.e. -1209087123.234234 1.0

boolean True as 1 and False as 0 i.e. 1, 0

timestamp is the number of centinanoseconds since epoch. It is the format used by the ACS framework and it can be obtained by multiplying the unix timestamp in floating point format by 10000000. All times are intended to be UT i.e. 143092278297088300

Request and Reply Messages

For each command we give a brief description of how the command can be used and the description of the reply to the command. We then provide a simple example.

status

Asks the status of the backend. The request message has no arguments. The Reply message has 3 arguments:

timestamp the timestamp of the answer message according to the backend clock

status code in normal working condition should be ok, any other value should be used for representing any possible failure state

acquiring is a boolean value indicating if the backend is performing an acquisition, can be 0 for false or 1 for true

Example communication:

request: "?status\r\n"
  reply: "!status,ok,1430922782.97088300,ok,0\r\n"

request: "?status\r\n"
  reply: "!status,ok,1430922782.97088300,clock error,0\r\n"

version

Asks the backend server what version of the protocol it is implementing. The Request message has no argument. The Reply message has 1 argument:

version id a string representing the protocol version

Example communication:

request: "?version\r\n"
  reply: "!version,1.0.1\r\n"

get-configuration

Asks the backend server what configuration is loaded at the moment. Request message has no argument. The Reply message has 1 argument:

configuration id a string representing the loaded configuration

If the backend has not yet been configured a special value of unconfigured is returned as reply argument.

Example communication:

request: "?get-configuration\r\n"
  reply: "!get-configuration,K2000\r\n"

request: "?get-configuration\r\n"
  reply: "!get-configuration,unconfigured\r\n"

set-configuration

Instruct the backend to configure itself according to the specified configuration code given as argument. Reply message has no argument. Request message has one argument:

configuration id a string identifying the configuration to be loaded

Example communication:

request: "?set-configuration,K2000\r\n"
  reply: "!set-configuration,ok\r\n"

request: "?set-configuration,nonexistent\r\n"
  reply: "!set-configuration,fail,cannot find configuration 'nonexistent'\r\n"

get-integration

Asks the backend server what integration time is configured. Request message has no argument. The Reply message has 1 argument:

integration time an integer representing the integration time configured in ms.

If the backend has not yet been configured a special value of 0 is returned as reply argument.

Example communication:

request: "?get-integration\r\n"
  reply: "!get-integration,20\r\n"

request: "?get-integration\r\n"
  reply: "!get-configuration,0\r\n"

set-integration

Instruct the backend to configure itself with the specified integration time given as argument. Reply message has no argument. Request message has one argument:

integration time an integer representing the integration time to be set in ms.

Example communication:

request: "?set-integration,20\r\n"
  reply: "!set-integration,ok\r\n"

request: "?set-integration,wrong\r\n"
  reply: "!set-integration,fail,integration time must be an integer number"

get-tpi

Ask the backend a total power level read. Request message has no argument. Reply message has as many arguments as the number of sections configured in the backend:

TPI_sec0, TPI_sec1, … a sequence of floating point numbers representing the total power intensity level of each section of the backend

Example communication:

request: "?get-tpi\r\n"
  reply: "!get-tpi,ok,900.00,1240.00\r\n"

get-tp0

Ask the backend a total power read with zero level input. Request message has no argument. Reply message has as many arguments as the number of sections configured in the backend:

TP0_sec0, TP0_sec1, … a sequence of floating point numbers representing the total power intensity level of each section of the backend with terminated input.

Example communication:

request: "?get-tp0\r\n"
  reply: "!get-tp0,ok,00.00,00.00\r\n"

time

Asks the backend to return its own timestamp, this command should be used to verify that the backend has an acceptable clock working before issuing time tagged acquisition commands. Request has no argument. The reply has one only argument:

timestamp the timestamp of the answer message according to the backend clock

Example communication:

request: "?time\r\n"
  reply: "!time,ok,1430922782.97088300\r\n"

start [timestamp]

Tell the backend to start the acquisition. The reply has no parameter. The reqeust has one optional parameter:

timestamp the exact time at which the acquisition should start

If given with a timestamp the backend should continue to accept commands while waiting for the start time. A stop command will cancel any further pending acquisition. If a new start command is issued while waiting for a start time, the most recent start command will overwrite the pending one.

Example communication:

request: "?start\r\n"
  reply: "!start,ok\r\n"

request: "?start,1430922782.97088300\r\n"
  reply: "!start,ok\r\n"

request: "?start,1430922782.97088300\r\n"
  reply: "!start,fail,cannot start at given time\r\n"

stop [timestamp]

Tell the backend to stop the acquisition. The reply has no parameter. The reqeust has one optional parameter:

timestamp the exact time at which the acquisition should stop

If given with a timestamp the backend should continue to accept commands while waiting for the stop time. If a new stop command is issued while waiting for a stop time, the most recent stop command will overwrite the pending one.

Example communication:

request: "?stop\r\n"
  reply: "!stop,ok\r\n"

request: "?stop,1430922782.97088300\r\n"
  reply: "!stop,ok\r\n"

request: "?stop,1430922782.97088300\r\n"
  reply: "!stop,fail,cannot stop at given time\r\n"

Note

In general we note that the correct behaviour of time tagged commands is left as a responsibility to the backend itself and not to the protocol. It will be duty of the particoular implementation to keep track of pending start and stop timestamps during the acquisition process. For example it is possible to have both a start timestamp and a stop timestamp issued in the future, and these should work as expected.

set-section

Configure a section of the backend, just as explained in Total Power - focus selector. The command expects 7 parameters, all of which are mandatory, but can be substituted by an asterisk ‘*’ meaning that the corresponding parameter should be left unchanged. The synthax is:

set-section=sect,startFreq,bw,feed,mode,sampleRate,bins

The server should respond ok if the operation succeds, and fail if the server could not perform the operation or the backend does not support this kind of operation.

Example communication:

request: "?set-section,1,50.0,200.0,1,CP,10,2048\r\n"
  reply: "!set-section,ok\r\n"

request: "?set-section,1,*,*,*,*,*,*\r\n"
  reply: "!set-section,ok\r\n"

request: "?set-section,1,*\r\n"
  reply: "!set-section,fail,set-section needs 7 arguments\r\n"

request: "?set-section,1,badparam,200.0,1,CP,10,2048\r\n"
  reply: "!set-section,fail,wrong parameter format\r\n"

cal-on [interleave=0]

Activate the noise diode for calibration purposes using a cal switching acquisition mode. The command takes in input one mandatory parameter:

interleave the number of backend samples to be acquired with the mark on and off alternatively. If 0 is given (default) as parameter the mark is always off, this can be used to turn the fast mark off.

Example communication:

request: "?cal-on\r\n"
  reply: "!cal-on,ok\r\n"

request: "?cal-on,10\r\n"
  reply: "!cal-on,ok\r\n"

request: "?cal-on,-10\r\n"
  reply: "!cal-on,fail,interleave samples must be a positive int\r\n"

set-filename

Tell the backend where to find the telescope file with metadata to be merged with the backend data.

filename absolute path of the file to be merged

Example communication:

request: "?set-filename,/hi/im/a/file.fits\r\n"
  reply: "!set-filename,ok\r\n"

convert-data

New in version 1.2.

Instructs the backend to convert data. This means that the control system has successfully closed the auxiliary metadata file which can now be completed with collected data from the backend.

Example communication:

request: "?convert-data\r\n"
  reply: "!convert-data,ok\r\n"

Handling Errors

As specified above, the protocol permits to distinguish between two kinds of errors, both of which are identified in the response messages:

protocol errors are identified by the response argument invalid

application errors are identified by the response argument fail

Both responses permit a second argument to specify a description of the error.

Example communication:

request: "?nonexistentcommand\r\n"
  reply: "!nonexistentcommand,invalid,cannot find command\r\n"

request: "?--asdf\r\n"
  reply: "!--asdf,invalid,invalid characters in command name\r\n"

request: "ciao\r\n"
  reply: "!ciao,invalid,requests must start with '?'\r\n"

request: "?start,0\r\n"
  reply: "!start,fail,invalid timestamp\r\n"

request: "?start,0\r\n"
  reply: "!start,fail,invalid timestamp\r\n"

Reference Implementation

You can find a reference implementation of the protocol at https://github.com/discos/discos-backend . This package implements all the logics related to the protocol, including parsing and serialization of messages, transmission, checks for correctness and error management. The package also defines a server implementation which enables a pluggable protocol to be used. The developer can just look at the tests (as described in the package docs) in order to define its own protocol implementation.

Considerations

The proposed protocol is intentionally very simple and little powerful; this choice is derived from the specs given in the meeting held the 13 / 04 / 2015 at OaC. More complex interactions would require a different protocol resulting in a more complex definition and different technologies involved. In particular the actual definition does not :

Version the protocol in the protocol itself

Permit to send asynchronous messages

Permit biderectional requests

Permit to send the same message to multiple recipients

Enable any security mechanism

References

[EBNF]

http://www.cl.cam.ac.uk/~mgk25/iso-14977.pdf

[KAT]

https://casper.berkeley.edu/wiki/images/1/11/NRF-KAT7-6.0-IFCE-002-Rev4.pdf

[TELNET]

http://www.freesoft.org/CIE/RFC/1123/31.htm