TUIO SKELETON (SDK V. 1.0.3.191)
Scheme TUIO SKELETON
TUIO SERVER sends data TUIO CLIENT receives data
How It Works
Thanks to the features of the Tuio protocol, it has been implemented a client-server system for the transmission of the skeleton data kinect “real time”. The results have been excellent opening new scenarios for the TUIO protocol use.The data transmission is fast and efficiently, the protocol TUIO is redundant and provides reliability
On the left there is the server On the right the Client
The Tuio Server divides the points of the skeleton in 5 groups and each group is assigned an id.
BodySegment(0,skeletons[k].Joints[JointType.ShoulderLeft],skeletons[k].Joints[JointType.ElbowLeft], skeletons[k].Joints[JointType.WristLeft], skeletons[k].Joints[JointType.HandLeft]);
BodySegment(10, skeletons[k].Joints[JointType.ShoulderRight], skeletons[k].Joints[JointType.ElbowRight], skeletons[k].Joints[JointType.WristRight], skeletons[k].Joints[JointType.HandRight]);
BodySegment(30,skeletons[k].Joints[JointType.HipLeft],skeletons[k].Joints[JointType.KneeLeft], skeletons[k].Joints[JointType.AnkleLeft], skeletons[k].Joints[JointType.FootLeft]);
BodySegment(50,skeletons[k].Joints[JointType.HipRight],skeletons[k].Joints[JointType.KneeRight], skeletons[k].Joints[JointType.AnkleRight], skeletons[k].Joints[JointType.FootRight]);
BodySegment(80,skeletons[k].Joints[JointType.Spine],skeletons[k].Joints[JointType.ShoulderCenter], skeletons[k].Joints[JointType.Head], skeletons[k].Joints[JointType.ShoulderCenter]);
The server sends add the TuioCursor (id, [joint.x, joint.y]) and commit the Frame:
_server.AddTuioCursor(id, (s_tmp));
_server.CommitFrame();
The client, thanks to the id, understand the type of point received.
if (tcur.getSessionID() == xxx){
…
…
}
The client graphics without problems the skeleton using five polyline.
Detail of the Client which displays the received messages in the console and the graphics rendering of points received
You can see a Demo of my application here:
OSC
Open Sound Control is a transmission protocol that allows musical instruments, computers and other devices to exchange multimedia creators that make it easy to “music performance data” in real time through a simple internal network (TCP / IP, Ethernet) or the Internet.
OSC works on high-speed network (broadband network speed), to exchange more data compared to MIDI, also giving more flexibility to the type of data available, this implies the possibility of a higher level of interaction between machines / software. The protocol for data exchange network used is the UDP.
A software can choose any number of ports, only limited by bandwidth, and also to send data to receive, and each signal has a path that can be assigned to an associated URL.
This technology allows to send different types of data including 32-bit packages, provides bandwidth for the exchange of data much larger, 1024 simultaneous messages against 128 of the MIDI, low latency and 32-bit data resolution.
TUIO PROTOCOL
The TUIO protocol is encoded using the format Open Sound Control, which provides an effective method of encoding binary data for transmission. The default transport method for the TUIO protocol is the encapsulation of binary data in OSC message within UDP packets sent to port 3333. The TUIO protocol defines two main classes of messages: SET messages and ALIVE messages. SET messages are used to communicate information about an object’s state such as position, orientation, and other recognized states. ALIVE messages indicate the current set of objects present on the surface using a list of unique Session IDs. To avoid possible errors evolving out of packet loss, no explicit ADD or REMOVE messages are included in the TUIO protocol. The receiver deduces object lifetimes by examining the difference between sequential ALIVE messages. In addition to SET and ALIVE messages, FSEQ messages are defined to uniquely tag each update step with a unique frame sequence ID. An optional SOURCE message identifies the TUIO source in order to allow source multiplexing on the client side.
Efficiency & Reliability
In order to provide low latency communication our implementation of the TUIO protocol uses UDP transport. When using UDP the possibility exists that some packets will be lost. For efficiency reasons set messages are packed into a bundle to completely use the space provided by a UDP packet. Each bundle also includes a redundant alive message to allow for the possibility of packet loss. For larger object sets, a bundle with a series of set messages, each including an alive message, are transmitted. When the surface is quiescent, alive messages are sent at a fixed rate dependent on the channel quality, for example once every second, to ensure that the receiver eventually acquires a consistent view of the set of alive objects. The state of each alive but unchanged object is periodically resent with additional set messages. This redundant information is resent at a lower rate, and includes only a subset of the unchanged objects at each update. The subset is continuously cycled so that each object is periodically addressed. Finally, each packet is marked with a frame sequence ID (fseq) message: an incrementing number that is the same for all packets containing data acquired at the same time. This allows the client to maintain consistency by identifying and dropping out-of-order packets.
——————————————————————————————————-
References:
Rosario Barbagallo 
Leave a comment