2.9 RapidSense API
- Kurt Dudek
States
RapidSense is comprised of 4 states: discovery, calibrating, operating, and fatal. The transitions are coordinated by users of the system based on what commands they issue to RapidSense. If at any point an error is encountered that requires user-intervention to resolve, RapidSense transitions into a fatal state and will not operate until the cause is resolved. The following figure illustrates the transitions.
Messages
The following messages define the RapidSense 2.x API.
For REST interface endpoints, use port number 11235. For example, http://localhost:11235/volumes.
Volumes
General description
Users can create, read, update, and delete volumes defined in the project. This is a purely REST interface.
Endpoint | /volumes |
|---|---|
Supported Operations | GET, POST, PUT, and DELETE |
Base Endpoint | <hostname>:<port>/volumes |
Create
To create a volume, users POST to the /volumes endpoint.
Operation | POST |
|---|---|
Description | Creates a volume. |
Request | POST <hostname>:<port>/volumes |
Response Body Format | {
"volume": {
"id": integer,
"bounds": {
"lower": [double, double, double],
"upper": [double, double, double]
}
} |
Example Response Body | {
"volumes": [
{
"id": 1,
"bounds": {
"lower": [-10.0, -10.0, -10.0]
"upper": [10.0, 10.0, 10.0]
}
}
]
} |
Read
To read a list of all volumes configured in RapidSense, users can perform a GET on the /volumes endpoint and receive a list of all volumes with their IDs and bounds.
Operation | GET |
|---|---|
Description | Returns a list of all configured volumes in the system. |
Request | GET <hostname>:<port>/volumes |
Response Body Format | {
"volumes": [
{
"id": integer,
"bounds": {
"lower": [double, double, double],
"upper": [double, double, double]
},
{
"id": integer,
"bounds": {
"lower": [double, double, double],
"upper": [double, double, double]
}
}
}
]
} |
Example Response Body | {
"volumes": [
{
"id": "/volumes/1",
"bounds": {
"lower": [-0.5, -0.7, 0]
"upper": [0.5, 0.7, 1.0]
}
}
]
} |
Update
To update a volume, users PUT to the /volumes/<id> endpoint providing the volume ID. The volume ID must exist already otherwise, the request errors out with a bad request error code. Updates must be idempotent, i.e., updates must be a full replacement of the volume details.
Operation | PUT |
|---|---|
Description | Updates a volume or creates a new volume if the ID does not match any existing volume IDs. |
Request | PUT <hostname>:<port>/volumes/<id> |
Request Body Format | {
"volume": {
"id": integer,
"bounds": {
"lower": [double, double, double],
"upper": [double, double, double]
}
} |
Response Body Format | {
"volume": {
"id": integer,
"bounds": {
"lower": [double, double, double],
"upper": [double, double, double]
}
} |
Example Request Body | {
"volume": {
"id": 2,
"bounds": {
"lower": [0.0, 0.0, 0.0]
"upper": [1.2, 1.2, 1.4]
}
}
]
} |
Example Response Body | {
"volume": {
"id": 2,
"bounds": {
"lower": [0.0, 0.0, 0.0]
"upper": [1.2, 1.2, 1.4]
}
}
]
} |
Delete
To remove / delete a volume, users DELETE on the /volumes/<id> endpoint providing the ID to remove.
Operation | DELETE |
|---|---|
Description | Removes a volume. |
Request | DELETE <hostname>:<port>/volumes/<id> |
Request Body Format | None. |
Response Body Format | None. |
Status
Users of the system can query the status of RapidSense either via the ASCII interface through the GetRapidSenseStatus command or via the REST interface through the /status endpoint.
ASCII Message Definition
Structure | {topic: GetRapidSenseStatus} |
|---|---|
Argument Description | N/A |
Function Description | This command can be called as soon as the RTR controller accepts a socket. It returns the latest status of RapidSense. All returned values are strings. |
Response Details | The response details contains a list of rapid sense state and error codes. The format is “GetRapidSenseStatus,<version>,<state>,<num-sensors><sensor-status><num-errors>,<list-of-error-codes>” Parameter descriptions: RapidSense Version
RapidSense State
Sensor Status(es)
Active RapidSense Error Code(s)
|
Example CSV Request | GetRapidSenseStatus |
Example YAML Response | {topic: GetRapidSenseStatus, type: Response, data: {rapidsense_version: 0.2.0, rapidsense_mode: IDLE, sensor_status: [inbound: 1, pallet-left: 1, pallet-right: 1], error_list: []}} |
Example CSV Responses | GetRapidSenseStatus,2.0.0,2,inbound,1,pallet-left,1,pallet-right,1,2,5002,5031
GetRapidSenseStatus,2.0.0,1,inbound,1,pallet-left,1,pallet-right,1,1,0
GetRapidSenseStatus,2.0.0,1,inbound,1,pallet-left,1,pallet-right,1,2,5102,5103 |
REST Endpoint & JSON Message Definition
Endpoint | /status |
|---|---|
Operation | GET |
Description | Returns the status of RapidSense. |
Response Body Format | {
"version": major.minor.patch,
"state": string,
"sensors": [{"serial_number": string, "calibration_status": string}],
"error_codes": [integer, ...]
} |
Example Request Body | GET localhost:11235/status |
Example Response Body | {"version": "0.1.2", "state": "operating", "sensors": [{"serial_number": "213509183b", "calibration_status": "calibrated"}, {"serial_number": "53134952f", "calibration_status": "uncalibrated"}], "error_codes": [5001, 5505]} |
Scan Scene
Users can scan the scene using 4 modes: snapshot, streaming, off, and clear. These operations affect the motion planning, enabling dynamic collision avoidance.
ASCII Message Definition
Structure | ScanScene |
|---|---|
CSV Arguments + Description | The mode by which to scan the scene:
|
YAML Arguments + Description |
|
Function Description | This command changes the scanning mode for RapidSense. |
Response Details | Whether the system is set to the requested mode. |
Example CSV Request | ScanScene,1 |
Example CSV Response | ScanScene,1,0 // success
ScanScene,1,5102 // failure |
Example YAML Request | {topic: ScanScene, data: {type: "start_streaming"}} |
Example YAML Response | {topic: ScanScene, response: {}} |
Calibration Setup
Users can setup RapidSense for calibration through the sensors and markers interfaces.
Markers
Markers are persisted between usages of RapidSense. Users can create, read, update, and delete markers for the system. Not all requests require the {id}field. The marker ID is the resource ID, as far as REST semantics is concerned.
Endpoint | /markers/{id} |
|---|---|
Supported Operations | GET, PUT, POST, PATCH, DELETE |
Base Endpoint | <hostname>:<port>/markers/{id} |
Read
To receive a list of all markers configured for RapidSense, users can perform a GET on the /markers endpoint and receive a list of all marker information. To receive a specific marker, users can perform a GET providing the marker’s ID in the {id} field.
Operation | GET |
|---|---|
Description | Returns a list of all markers in the system. |
Request | GET <hostname>:<port>/markers |
Response Body Format | {
"markers": [
{
"id": integer, // marker id
"length": float, // meters
"robot"?: string // optional name of the robot this marker is mounted to
},
...
]
} |
Example Response Body | {
"markers": [
{
"id": 319, // marker id
"length": 0.254,
"robot"?: "lab_1_mitsu" // optional name of the robot this marker is mounted to
},
{
"id": 125,
"length": 0.254,
// no robot parameter indicates a stationary target
},
]
} |
HTTP Code Responses | N/A - returns an empty list if no markers exist. |
Operation | GET |
|---|---|
Description | Returns a single marker by its ID. |
Request | GET <hostname>:<port>/markers/{id} |
Response Body Format | {
"marker": {
"id": integer,
"length": float, // meters
"robot": string // optional name of the robot this marker is mounted to
}
} |
Example Response Body | {
"marker": {
"id": 319,
"length": 0.254,
"robot": "lab_1_mitsu" // optional key-value pair for the robot to which this marker is mounted
}
} |
HTTP Code Responses |
|
Create, Update, or Delete
The above data format can also be used to create new markers, update existing markers, or delete markers from the system.
Operation | POST |
|---|---|
Description | Create a single marker. |
Request | POST <hostname>:<port>/markers |
Request Body Format | {
"markers": [
{
"id": integer,
"length": float, // meters
"robot"?: string // optional name of the robot this marker is mounted to
},
]
} |
Example Response Body | { // a list of all markers is returned
"markers": [
{
"id": 319,
"length": 0.254,
"robot"?: "lab_1_mitsu" // optional name of the robot this marker is mounted to
},
{
"marker_id": 125,
"length": 0.254,
// no robot parameter indicates a stationary target
},
]
} |
HTTP Code Responses |
|
Sensors
The sensors endpoint is automatically populated with detected sensors. Any sensors that are not configured meaning not setup for calibration, return default values regarding their calibration setup. The following table details the general REST endpoint information needed to setup calibration for a sensor.
Endpoint | /sensors/{id} |
|---|---|
Supported Operations | GET and PATCH |
Base Endpoint | <hostname>:<port>/sensors/{id} |
Read
To read a list of all sensor configurations for RapidSense, users can perform a GET on the /sensors endpoint and receive a list of all sensor information including the resource’s ID. Additionally, users can perform a GET on a single sensor using the sensor’s resource ID.
Operation | GET |
|---|---|
Description | Returns a list of all detected sensors in the system. |
Request | GET <hostname>:<port>/sensors/{id} |
Response Body Format | {
"sensors": [{
"id": integer,
"name": string,
"mfg_info": {
"manufacturer": string,
"model": string,
"serial_number": string
},
"pose": double array of 16-elements,
"frustum": {
"hfov": float,
"vfov": float,
"near_clip": float,
"far_clip": float
},
"active": boolean,
"calibration": {
"last_calibrated": string, // format YYYY-MM-DD HH:mm
"markers": [
{
"marker_id": integer,
"cal_target": string, // only returned if marker has robot field populated
},
...
]
}
}
}
]
} |
Example Response Body | {
"sensors": [{
"id": 1,
"name": "pallet",
"mfg_info": {
"manufacturer": "Intel",
"model": "D455",
"serial_number": "123abe45613f"
},
"pose": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0],
"frustum": {
"hfov": "240",
"vfov": "320",
"near_clip": 0.5,
"far_clip": 2.5
},
"active": true,
"calibration": {
"last_calibrated": "2024-01-02 11:48",
"markers": [
{
"cal_target": "lab1_mitsu_calibration_12345",
"marker_id": 6,
},
{
"marker_id": 2,
}
]
}
]
} |
Update
To update a sensor’s calibration setup information, users can perform a PATCH on /sensors/<id>. Users associate a marker to a sensor. If a stationary marker is not being used or set, users do not provide that object.
Operation | PATCH |
|---|---|
Description | Updates a sensor’s calibration parameters. |
Request | PATCH <hostname>:<port>/sensors/<id> |
Request Body Format | {
"markers": {
{
"marker_id": integer,
"cal_target": string, // only provided if marker is associated with a robot
},
}
} |
Example Request Body | {
"markers": [
{
"marker_id": 6,
"cal_target": "lab1_mitsu_calibration_12345",
},
{
"marker_id": 2,
}
]
} |
Example Response Body | {
"id": 1,
"name": "pallet",
"mfg_info": {
"manufacturer": "Intel",
"model": "D455",
"serial_number": "123abe45613f"
},
"pose": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0],
"frustum": {
"hfov": "240",
"vfov": "320",
"near_clip": 0.5,
"far_clip": 2.5
},
"active": true,
"calibration": {
"last_calibrated": "2024-01-02 11:48",
"markers": [
{
"marker_id": 6,
"cal_target": "lab1_mitsu_calibration_12345",
},
{
"marker_id": 2,
}
]
}
} |
Calibrating
A user of the system (e.g., a script) can move the robot with a calibration standard to a certain location and initiate the calibration of a sensor, then move onto the next sensor until complete. The calibration process is coordinated by the calibration service.
Height Check (formerly Query Clearance)
General description
An area within a volume can be queried for its maximum height. This can be used for querying a placement location for a package on a pallet, as an example. The caller in this case would compute the difference in the height of the expected landing spot for that package and the actual landing spot returned by height check and adjust the place location height.
Users can create, read, update, and delete height check filter objects and adjust their parameters. Creation of filter objects and adjusting the filter parameters can only be performed via the REST interface. To query the filter for heights, users have the option of using the ASCII interface or the REST interface.
Endpoint | /heights |
|---|---|
Supported Operations | GET, PATCH, POST, PUT, and DELETE |
Base Endpoint | <hostname>:<port>/heights |
Create
To create an instance of the height check filter, users must POST on the /heights endpoint, providing the volume ID within the request body.
Operation | POST |
|---|---|
Description | Creates a volume. |
Request | POST <hostname>:<port>/heights |
Request Body Format | {"volume_id": integer} |
Response Body Format | {
"height_filter": {
"id": integer,
"parameters": {
"integration_time": double, // units in milliseconds, default of 100ms
"erosion": double // units in meters
}
}
} |
Example Response Body | {
"height_filter": {
"id": 1,
"parameters": {
"integration_time": 100.0,
"erosion": 0.02
}
}
]
} |
Read
To read a list of all height filters configured in RapidSense, users can perform a GET on the /heights endpoint and receive a list of all volumes with their IDs and bounds.
Operation | GET |
|---|---|
Description | Returns a list of all configured volumes in the system. |
Request | GET <hostname>:<port>/heights |
Response Body Format | {
"height_filters": [{
"id": integer,
"parameters": {
"integration_time": double,
"erosion": double
}
}
]
} |
Example Response Body | {
"height_filters": [
{
"id": 1,
"parameters": {
"integration_time": 100.0,
"erosion": 0.02
}
}
]
} |
Update
To update a filter’s parameter(s), users can perform a PUT on /heights/<id>.
Operation | PUT |
|---|---|
Description | Updates a height filter’s parameters. |
Request | PUT <hostname>:<port>/heights/<id> |
Request Body Format | {
"height_filter": {
"id": integer,
"parameters": {
"integration_time": double,
"erosion": double,
}
}
} |
Response Body Format | {
"height_filter": {
"id": integer,
"parameters": {
"integration_time": double,
"erosion": double
}
}
} |
Example Request Body | {
"height_filter": {
"id": integer,
"parameters": {
"integration_time": double,
"erosion": double
}
}
} |
Example Response Body | {
"height_filter": {
"id": 1,
"parameters": {
"integration_time": 123.0,
"erosion": 0.04
}
}
} |
Delete
To remove / delete a height filter, users DELETE on the /heights/<id> endpoint providing the ID to remove.
Operation | DELETE |
|---|---|
Description | Removes a height filter. |
Request | DELETE <hostname>:<port>/heights/<id> |
Request Body Format | None. |
Response Body Format | None. |
To query heights
Users can query the maximum height of a specified area via the ASCII interface or the REST interface. Users specify the centroid (the x / y center point) along with the length (distance along the x-axis) and width (distance along the y-axis) of the area to query for height. The maximum height in that specified area is returned in units of meters. Below are details regarding both the ASCII and REST interfaces.
ASCII Message Definition
Structure | {topic: HeightCheck, data: {id: string, centroid: [double, double], width: double, length: double}} |
|---|---|
Argument Description | id: the ID of the filter instance (returned when a user POSTs on centroid: the center of the area as an x, y position relative to the volume’s origin (lowest x and y values). width: the width of the area (along the x-axis). length: the height of the area (along the y-axis). NOTE: Units are set by |
Function Description | Height check uses the heuristic of maximum value within the specified area within the volume. NOTE: objects that are to be placed with significant overhang will bias the measurements so care should be taken in computing this value to allow overhangs but not bias the calculations to negatively impact system operations. This command can be run at any time while RapidSense is in OPERATING state. |
Response Details | The response contains a status for the operation and the height according to the heuristic (currently maximum value) in the units specified (by the Status indicates whether the query command was successful or whether an error was encountered. A status of 0 indicates the operation was a success. Otherwise, the error code is returned. If an error code is encountered, the clearance is not returned. |
Example Request | {topic: HeightCheck, data: {id: 1, centroid: [500, 200], width: 100, length: 220} |
Example Responses | {topic: HeightCheck, data: {status: 0, height: 130}}
{topic: HeightCheck, data: {status: 5001}} |
Example CSV Responses | HeightCheck,0,130
HeightCheck,5001 |
REST Endpoint & JSON Message Definition
Endpoint | /heights/<id>/?cx={x}&cy={y}&width={width}&length={length} |
|---|---|
Supported Operations | GET |
Function Description | The heights endpoint provides callers the height according to the heuristic of maximum value within a specified area. The parameters are the centroid 2D point (cx, cy) and the width and length of the area of interest (all units in meters). |
Response Details | The response is the maximum height in meters. |
Example Request | GET localhost:11235/heights/1/?cx=0.450&cy=0.789&width=0.110&length=0.120 |
Example Response | {"height": 0.123} |
Change Detection(under development)
General description
Users can inspect a volume for deviations from a expected template built by adding objects to a volume and then querying for the deviations.
Users can create, read, update, and delete change detection filter objects and adjust their parameters. Creation of filter objects and adjusting the filter parameters can only be performed via the REST interface. To query the filter for changes, users have the option of using the ASCII interface or the REST interface.
Endpoints | /changes
/changes/<id>/objects |
|---|---|
Supported Operations | GET, PATCH, POST, PUT, and DELETE |
Base Endpoints | <hostname>:<port>/changes
<hostname>:<port>/changes/<id>/objects |
Create (filter)
To create an instance of the change detection filter, users must POST on the /changes endpoint, providing the volume ID within the request body.
Operation | POST |
|---|---|
Description | Creates a volume. |
Request | POST <hostname>:<port>/changes |
Request Body Format | {"volume_id": integer} |
Response Body Format | {
"change_detection_filter": {
"id": integer,
"parameters": {
"timeout": double // units in milliseconds, default of 100ms
}
}
} |
Example Response Body | {
"change_detection_filter": {
"id": 1,
"parameters": {
"timeout": 100.0
}
}
]
} |
Read (filter)
To read a list of all change detection filters configured in RapidSense, users can perform a GET on the /changes endpoint and receive a list of all filters with their IDs and attributes and objects added to the volume.
Operation | GET |
|---|---|
Description | Returns a list of all configured filters in the system. |
Request | GET <hostname>:<port>/changes
GET <hostname>:<port>/changes/<id> |
Response Body Format | {
"change_detection_filters": [{
"id": integer,
"parameters": {
"timeout": double // timeout in milliseconds
},
"objects": [
{
"id": integer,
"centroid": [double, double], // all values in meters
"length": double,
"width": double,
"height": double
}
]
}
]
} |
Example Response Body | {
"change_detection_filters": [
{
"id": 1,
"parameters": {
"timeout": 3000.0
},
"objects": []
}
]
} |
Update (filter)
To update a filter’s parameter(s), users can perform a PUT on /changes/<id>.
Operation | PUT |
|---|---|
Description | Creates a volume. |
Request | POST <hostname>:<port>/changes |
Request Body Format | {"volume_id": integer} |
Response Body Format | {
"change_detection_filter": {
"id": integer,
"parameters": {
"timeout": double // units in milliseconds, default of 100ms
}
}
} |
Example Response Body | {
"change_detection_filter": {
"id": 1,
"parameters": {
"timeout": 100.0
}
}
]
} |
Delete (filter)
To remove / delete a change detection filter, users DELETE on the /changes/<id> endpoint providing the ID to remove.
Operation | DELETE |
|---|---|
Description | Removes a change detection filter. |
Request | DELETE <hostname>:<port>/changes/<id> |
Request Body Format | None. |
Response Body Format | None. |