Tracking Protocol Specification
CinCraft Scenario SDK Lite 1.0.1
This page describes the SDK Lite tracking protocol, it is also available as a downloadable file here:
Description
The idea behind the SDK Lite is to provide our customers with a much simpler way to interface with CinCraft Scenario as a data source.
This is achieved by means of a consistent UDP connection with a single MTU for each frame containing all of the essential information usually needed for integration with 3rd party software.
The server type can be set to either UDP broadcast or UDP unicast.
The server type, port and IP address can all be modified in CinCraft Scenario.
The total size of each message/packet will always be smaller than the typical 1500 Bytes in UDP communications and its format will conform to the specification described by the version of the SDK Lite Protocol.
Further extensions are planned for the SDK Lite, including a serial implementation.
Packet Format
After a 2 characters control flag (0x0A, 0x0D), the next 8 bytes will always be fixed whatever the version of the SDK Lite protocol. They define the version of the protocol that is streamed and the length of the dynamic section in bytes.
# | Size | Type (x86_64) | Data |
|---|---|---|---|
0 | 2 Bytes | char [2] | Control Flag [0x0A, 0x0D] |
The next section is static and has a fixed size of 8 bytes | | | |
1 | 1 Byte | uint8_t | Checksum (Two’s complement) |
2 | 3 Bytes | uint8_t [3] | SDK Lite Version [M, m, r] |
3 | 4 Bytes | uint32_t | Length in bytes of the dynamic section |
Dynamic section starts here | | | |
4 | 4 Bytes | uint32_t | Packet Number |
5 | 4 Bytes | uint32_t | Timecode |
6 | 12 Bytes | float [3] | Translation [x, y, z] |
7 | 12 Bytes | float [3] | Rotation [ox, oy, oz] |
8 | 12 Bytes | float [3] | Normalized Focus, Iris and Zoom |
9 | 12 Bytes | float [3] | Mapped Focus, Iris and Zoom |
10 | 4 Bytes | float | Nodal Shift in Meters |
11 | 64 Bytes | float [16] | Lens Parameters |
The packets must be checked against their checksum at arrival. This is achieved by computing the sum of all bytes as int8_t starting with the checksum and ending with the last byte of the dynamic section which length is given by #3. A valid packet would sum to zero.
Total size of a packet is 134 Bytes. For a 1080p60 video signal, the required bandwidth is 60x134 = 8040 Bytes/sec so under 8KB/sec.
Making sense of the SDK Lite’s information:
Packet Number
A counter of sent packets since the SDK Lite began streaming.
Timecode
The SDI source’s timecode.
Translation
The translation vector[3] in meters.
Rotation
The rotation uses a compact vector[3] Rodrigues representation:
- The angle of rotation is given by the module of the input vector.
- The axis of rotation is given by the normalized input vector.
OpenCV provides a convenient method to convert to and from a 3x3 rotation matrix:
void Rodrigues(InputArray src, OutputArray dst, OutputArray jacobian=noArray())
Normalized FIZ
Focus, Iris and Zoom encoder values normalized between 0.0 and 1.0.
Mapped FIZ
World scale values for Focus (in meters), Iris (in T-stops) and Zoom (in mm).
Nodal Shift
Distance from the camera sensor’s center to the nodal point in mm.
Lens Parameters
This is a vector[16] with 3 distinct sections:
- The first four values are affine parameters: Normalized H Focal, Normalized V Focal, Normalized H Optical center and Normalized V Optical center.
- The two that follow are radial distortion: k1, k2
- The two that follow are radial undistortion: inv_k1, inv_k2
C++ Pseudo-code for Packet Deserialization
Low-level Read Helpers
uint8_t ReadUInt8(BYTE* data, uint32_t& pos, int maxLen = -1)
{
if (maxLen > -1 && static_cast<int>(pos) > maxLen)
{
return 0;
}
uint8_t ret = static_cast<uint8_t>(data[pos]);
pos += 1;
return ret;
}
uint32_t ReadUInt32(BYTE* data, uint32_t& pos, int maxLen = -1)
{
if (maxLen > -1 && static_cast<int>(pos + 3) > maxLen)
{
return 0;
}
uint32_t ret = static_cast<uint32_t>(data[pos]);
pos += 4;
return ret;
}
float ReadFloat32(BYTE* data, uint32_t& pos, int maxLen = -1)
{
if (maxLen > -1 && static_cast<int>(pos + 3) > maxLen)
{
return 0;
}
float ret;
uint8_t bytes[] = { data[pos], data[pos+1], data[pos+2], data[pos+3] };
memcpy(&ret, &bytes, sizeof(ret));
pos += 4;
return ret;
} Conversions and Coordinate Utilities
// in degrees
float NormalizedFocalLengthToFov(float focalLengthNorm)
{
return (180.0 / M_PI) * 2.0 * atan2(1.0, 2 * focalLengthNorm);
}
// Zeiss SDK streams CameraFromWorld
Eigen::Vector3f GetPosition(float tX, float tY, float tZ, float rotX, float rotY, float rotZ)
{
Eigen::Vector3f rodriguesVec(rotX, rotY, rotZ); // read directly from SDK Lite
Eigen::AngleAxisf angleAxis(rodriguesVec.norm(), rodriguesVec.normalized());
Eigen::Matrix3f rotationMatrix = angleAxis.toRotationMatrix();
auto inverseRotation = rotationMatrix.transpose();
Eigen::Vector3f translationVec(tX, tY, tZ);
Eigen::Vector3f position = -inverseRotation * translationVec;
return position;
}
void rodriguesToEuler(float rotX, float rotY, float rotZ, float& panOut, float& tiltOut, float& rollOut)
{
Eigen::Vector3f rodriguesVec(rotX, rotY, rotZ); // read directly from SDK Lite
Eigen::AngleAxisf angleAxis(rodriguesVec.norm(), rodriguesVec.normalized());
Eigen::Matrix3f rotationMatrix = angleAxis.toRotationMatrix();
auto rotationMatrixInv = rotationMatrix.transpose();
// Choose the angle order as needed for the target system
Eigen::Matrix3f ZeissCS;
ZeissCS.setZero();
ZeissCS(0, 0) = 1.0;
ZeissCS(1, 1) = -1.0;
ZeissCS(2, 2) = -1.0;
const auto lRotation = ZeissCS * rotationMatrixInv;
Eigen::Vector3f eulerAngles = lRotation.eulerAngles(1, 0, 2);
panOut = eulerAngles[0] * -1;
tiltOut = eulerAngles[1];
rollOut = eulerAngles[2];
} Packet Extraction Example
void ExtractTrackdata(char* tsname, char* RxBuffer)
{
// Content of RxBuffer is expected to be checked before entering
// Control flags must be valid
// Checksum must also be valid
BYTE* data = (BYTE*)RxBuffer;
// =========
// Header
pos += 2; // control flag
// ==============
// Static section
pos += 1; // checksum
uint8_t version_major = ReadUInt8(data, pos);
uint8_t version_minor = ReadUInt8(data, pos);
uint8_t version_revision = ReadUInt8(data, pos);
uint32_t maxLen = ReadUInt32(data, pos) + pos; // maximum length of packet
// ==============
// Dynamic section
float packet_number = static_cast<float>(ReadUInt32(data, pos, maxLen));
float timecode = static_cast<float>(ReadUInt32(data, pos, maxLen));
float tX = ReadFloat32(data, pos, maxLen);
float tY = ReadFloat32(data, pos, maxLen);
float tZ = ReadFloat32(data, pos, maxLen);
float rotX = ReadFloat32(data, pos, maxLen);
float rotY = ReadFloat32(data, pos, maxLen);
float rotZ = ReadFloat32(data, pos, maxLen);
// Position of pinhole / projection center / non-parallax point in meters from world origin
// ZEISS uses a right-handed Y-up coordinate system
auto position = GetPosition(tX, tY, tZ, rotX, rotY, rotZ);
// Rotation in radians
float pan, tilt, roll;
rodriguesToEuler(rotX, rotY, rotZ, pan, tilt, roll);
// Rotation in degrees
pan = 180.f * pan / static_cast<float>(M_PI);
tilt = 180.f * tilt / static_cast<float>(M_PI);
roll = 180.f * roll / static_cast<float>(M_PI);
// Normalized FIZ values in [0, 1]
float focusNorm = ReadFloat32(data, pos, maxLen);
float irisNorm = ReadFloat32(data, pos, maxLen);
float zoomNorm = ReadFloat32(data, pos, maxLen);
// Mapped FIZ values
float focusMapped = ReadFloat32(data, pos, maxLen); // meters
float irisMapped = ReadFloat32(data, pos, maxLen); // T-stops
float zoomMapped = ReadFloat32(data, pos, maxLen); // mm
// Distance of the sensor from pinhole / projection center / non-parallax point
float nodalShift = ReadFloat32(data, pos, maxLen);
const float hFov = NormalizedFocalLengthToFov(ReadFloat32(data, pos, maxLen));
const float vFov = NormalizedFocalLengthToFov(ReadFloat32(data, pos, maxLen));
// ZEISS values are normalized relative to image width and height
// (0.0, 0.0) is top-left and (1.0, 1.0) is bottom-right
const float hCenter = ReadFloat32(data, pos, maxLen);
const float vCenter = ReadFloat32(data, pos, maxLen);
// Distortion values from Zeiss / Cinelens are normalized along the half diagonal of the sensor
// If values are normalized along the full diagonal, scale factors must be adjusted accordingly
float k1 = ReadFloat32(data, pos, maxLen) / 4.0f;
float k2 = ReadFloat32(data, pos, maxLen) / 16.0f;
// Inverse parameters
float k1Inv = ReadFloat32(data, pos, maxLen) / 4.0f;
float k2Inv = ReadFloat32(data, pos, maxLen) / 16.0f;
} Sample for one frame of data
const uint8_t buf[] = { 0x0a, 0x0d, 0xb2, 0x01, 0x00, 0x01, 0x7c, 0x00, 0x00, 0x00, 0xec, 0xa4, 0x00, 0x00, 0x05, 0x47, 0x26, 0x08, 0x1d, 0x55, 0x14, 0x3f, 0x0b, 0x6d, 0x2f, 0x40, 0xd5, 0x23, 0x32, 0x3f, 0x10, 0x8b, 0xa4, 0x3f, 0xe7, 0x31, 0x6c, 0x3c, 0x8c, 0x2f, 0x93, 0xbd, 0x76, 0x90, 0x7e, 0x3f, 0x00, 0x00, 0x00, 0x00, 0x34, 0x28, 0x7f, 0x3f, 0x7b, 0x66, 0x1d, 0x41, 0x33, 0x33, 0xf3, 0x3f, 0x32, 0x34, 0x6e, 0x42, 0x93, 0xa6, 0x72, 0x3d, 0xf0, 0xac, 0xc5, 0x40, 0x35, 0xd9, 0x2f, 0x41, 0x6b, 0x66, 0x01, 0x3f, 0x1d, 0xe8, 0x05, 0x3f, 0x97, 0xf0, 0xd3, 0x3c, 0xf9, 0x8c, 0xcd, 0x38, 0x8b, 0x53, 0xd3, 0xbc, 0xd7, 0xc5, 0xdc, 0x3a, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff, 0xff, };