New methodologies of human-robot interaction through kinect and graphics tablet.
A system has been realized to allow immediate and secure interaction with an industrial manipulator through gestures, voice commands, applications, touch and pen input. The system is structured with a multi-threaded concurrency control on shared variables, in order to manage parallel activities and ensure an exchange of information with the robot controller every 12 ms. The system is designed in every detail to be used by the final user without a mouse and a keyboard.
The project aim is to make a robotic arm easy to use with devices that do not need to be worn. This system provides a simple front end that hides the complexity in the programming of a robotic arm.
It’s a system that can work with existing applications, but provides new usage scenarios.
If the user lifts only a finger, any kind of events can be generated. For example , drawing or marking specific paths to be tracked can be easily done just by moving the index finger in the space.
If a user would like a higher accuracy on the path, a tablet can be used to refine the trajectories. The user by means of the hand has the ability to indicate the point where the robot takes an object and the destination where the robot leaves it.
Appropriate checks will be implemented to ensure the safety of users.
The main objective is to create a collection of devices and software tools allowing a simple, fast but also safe human-machine interaction.
Entity Framework is an object-relational mapper. This means that it can return the data in your database as an object (e.g.: a Person object with the properties Id, Name, etc.) or a collection of objects.
The Entity Framework provides facilities to do the following: materialize data returned from the database as entity objects; track changes that were made to the objects; handle concurrency; propagate object changes back to the database; and bind objects to controls.
“Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications.
Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.”
The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.
Here’s an excerpt:
A San Francisco cable car holds 60 people. This blog was viewed about 2,500 times in 2013. If it were a cable car, it would take about 42 trips to carry that many people.
The developed software allows the doctor to record specific body movements that the patient have to do. The system is responsible to analyze, evaluate and report on every move made by the patient,has been developed a metric to evaluate the movements performed. The system provides graphical replay of the movements performed, and also there is a tool for remote monitoring.
The project aim is to make a robotic arm easy to use with the kinect. This system provides a simple front end that hides the complexity in the programming of a robotic arm. It’s a system that can work with existing applications, but provides new usage scenarios. If the user lifts only a finger, any kind of events can be generated. For example , drawing or marking specific paths to be tracked can be easily done just by moving the index finger in the space. The Kinect Power!
This software, based on the Microsoft Kinect, uses hand motion detection based on a clustering hand detection algorithm.
The coordinates of the hands are sent to an application that computes the inverse kinematics and the desired robot trajectory.
A raised finger sends an input. With the hand closed, the cursor is displayed on the screen without transmitting touch input.
Fig. 1 Working Area of Kuka.
The method is used to send the coordinates when the finger is detected, then, when the hand is closed the trajectory is considered concluded. It is necessary to keep the hand closed (first) because when the user lifts a finger, the system sends an input signal (touch).
Fig.2 Software C# Multithread “Kinect Writer”.
To improve the quality of the trajectory, interpolation and moving average has been implemented.
A system of events linked to voice commands is also integrated with multiple features. To avoid the activation of unwanted commands, the user must keep both hands closed while saying the command. All controls to be tested must have a confidence level greater than a given threshold (to be decided experimentally).
I have developed a compiler&interpreter using the tools: FLEX and BISON (C Language) from MIL to MIN.
The lexical analyzer should take as input a MINI-L program, parse it, and output the sequence of lexical tokens associated with the program.
The software takes a syntactically-correct MINI-L program (a program that can be parsed without any syntax errors), verify that it has no semantic errors, and then generate its corresponding intermediate code.
A source program in MINI-L will be translated by the compiler into an intermediate code program, and that intermediate code program will be the input to the interpreter which will take the intermediate code, execute it and give the final results.
MIL CODE
The MINI-L Language has the following features:
Integer scalar variables.
Unidimensional arrays of integers.
Assignment statements.
While loops.
If-then-else statements.
Read/Write statements.
Comments.
In MINI-L a comment is introduced by “- -” and goes to the end of the line. All tokens are separated by blanks, tabs, newlines, delimiters, operators or comments. A valid identifier begins with a letter and may be followed by additional letters, digits or underscores. The length of an identifier does not exceed 8 characters. Finally, MINI-L is case sensitive with all reserved words written in lower case.
Errors handled:
Error Handling for the use of variables that have not been declared at the beginning of the program
Error Handling for a declaration of the same variable multiple times
Error Handling in the use of an array variable as a normal integer variable
Error Handling negative array size
Error management use of an array index greater than the maximum size declared (in the case where the index is a number)
Management error identifier with less than 8 characters long and ends with ‘_’
Management error identifier with more than 8 characters
Syntax diagrams describing the syntax of MINI-L are given to you. Your first task in this assignment
is to develop a context-free grammar from the syntax diagrams. The parser will get the necessary information about the tokens from the lexical analyzer.
REST is an architectural style which is based on web-standards and the HTTP protocol.
In a REST based architecture everything is a resource. A resource is accessed via a common interface based on the HTTP standard methods.. GET, POST.. PUT!
JAX-RS provides some annotations to aid in mapping a resource class (POJO) as a web resource.
Jersey framework is more than the JAX-RS Reference Implementation.
Thanks to the features of the Tuio protocol, it has been implemented a client-server system for the transmission of the skeleton data kinect “real time”. The results have been excellent opening new scenarios for the TUIO protocol use.The data transmission is fast and efficiently, the protocol TUIO is redundant and provides reliability
On the left there is the server On the right the Client
The Tuio Server divides the points of the skeleton in 5 groups and each group is assigned an id.
The server sends add the TuioCursor (id, [joint.x, joint.y]) and commit the Frame:
_server.AddTuioCursor(id, (s_tmp));
_server.CommitFrame();
The client, thanks to the id, understand the type of point received.
if (tcur.getSessionID() == xxx){
…
…
}
The client graphics without problems the skeleton using five polyline.
Detail of the Client which displays the received messages in the console and the graphics rendering of points received
You can see a Demo of my application here: OSC
Open Sound Control is a transmission protocol that allows musical instruments, computers and other devices to exchange multimedia creators that make it easy to “music performance data” in real time through a simple internal network (TCP / IP, Ethernet) or the Internet.
OSC works on high-speed network (broadband network speed), to exchange more data compared to MIDI, also giving more flexibility to the type of data available, this implies the possibility of a higher level of interaction between machines / software. The protocol for data exchange network used is the UDP.
A software can choose any number of ports, only limited by bandwidth, and also to send data to receive, and each signal has a path that can be assigned to an associated URL.
This technology allows to send different types of data including 32-bit packages, provides bandwidth for the exchange of data much larger, 1024 simultaneous messages against 128 of the MIDI, low latency and 32-bit data resolution.
TUIO PROTOCOL
The TUIO protocol is encoded using the format Open Sound Control, which provides an effective method of encoding binary data for transmission. The default transport method for the TUIO protocol is the encapsulation of binary data in OSC message within UDP packets sent to port 3333. The TUIO protocol defines two main classes of messages: SET messages and ALIVE messages. SET messages are used to communicate information about an object’s state such as position, orientation, and other recognized states. ALIVE messages indicate the current set of objects present on the surface using a list of unique Session IDs. To avoid possible errors evolving out of packet loss, no explicit ADD or REMOVE messages are included in the TUIO protocol. The receiver deduces object lifetimes by examining the difference between sequential ALIVE messages. In addition to SET and ALIVE messages, FSEQ messages are defined to uniquely tag each update step with a unique frame sequence ID. An optional SOURCE message identifies the TUIO source in order to allow source multiplexing on the client side.
Efficiency & Reliability
In order to provide low latency communication our implementation of the TUIO protocol uses UDP transport. When using UDP the possibility exists that some packets will be lost. For efficiency reasons set messages are packed into a bundle to completely use the space provided by a UDP packet. Each bundle also includes a redundant alive message to allow for the possibility of packet loss. For larger object sets, a bundle with a series of set messages, each including an alive message, are transmitted. When the surface is quiescent, alive messages are sent at a fixed rate dependent on the channel quality, for example once every second, to ensure that the receiver eventually acquires a consistent view of the set of alive objects. The state of each alive but unchanged object is periodically resent with additional set messages. This redundant information is resent at a lower rate, and includes only a subset of the unchanged objects at each update. The subset is continuously cycled so that each object is periodically addressed. Finally, each packet is marked with a frame sequence ID (fseq) message: an incrementing number that is the same for all packets containing data acquired at the same time. This allows the client to maintain consistency by identifying and dropping out-of-order packets.
Rosario Barbagallo 