New methodologies of human-robot interaction through kinect and graphics tablet.
A system has been realized to allow immediate and secure interaction with an industrial manipulator through gestures, voice commands, applications, touch and pen input. The system is structured with a multi-threaded concurrency control on shared variables, in order to manage parallel activities and ensure an exchange of information with the robot controller every 12 ms. The system is designed in every detail to be used by the final user without a mouse and a keyboard.

The project aim is to make a robotic arm easy to use with devices that do not need to be worn. This system provides a simple front end that hides the complexity in the programming of a robotic arm.

It’s a system that can work with existing applications, but provides new usage scenarios.

If the user lifts only a finger, any kind of events can be generated. For example , drawing or marking specific paths to be tracked can be easily done just by moving the index finger in the space.

If a user would like a higher accuracy on the path, a tablet can be used to refine the trajectories.
The user by means of the hand has the ability to indicate the point where the robot takes an object and the destination where the robot leaves it.

Appropriate checks will be implemented to ensure the safety of users.
The main objective is to create a collection of devices and software tools allowing a simple, fast but also safe human-machine interaction.

Posted by: Rosario Barbagallo | June 1, 2014

Node.js

I  started to study node.js…

Why?

#event-driven #non-blocking I/O #real-time

Here, there is the server…  only 3 rows …. :D

Node.js Server

 

From the website we can read:

“Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications.
Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.”

 

Posted by: Rosario Barbagallo | January 18, 2014

2013 in review

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

A San Francisco cable car holds 60 people. This blog was viewed about 2,500 times in 2013. If it were a cable car, it would take about 42 trips to carry that many people.

Click here to see the complete report.

Posted by: Rosario Barbagallo | February 19, 2013

coding4fun

Thanks, Mr Greg Duncan! w coding4fun!

Posted by: Rosario Barbagallo | February 18, 2013

Rehabilitation Software

The developed software allows the doctor to record specific body movements that the patient have to do. The system is responsible to analyze, evaluate and report on every move made by the patient,has been developed a metric to evaluate the movements performed. The system provides graphical replay of the movements performed, and also there is a tool for remote monitoring.

more info:

http://www.rosariobarbagallo.com/kinect-doctor/

Posted by: Rosario Barbagallo | February 7, 2013

Kinect Writer

The project aim is to make a robotic arm easy to use with the kinect. This system provides a simple front end that hides the complexity in the programming of a robotic arm. It’s a system that can work with existing applications, but provides new usage scenarios. If the user lifts only a finger, any kind of events can be generated. For example , drawing or marking specific paths to be tracked can be easily done just by moving the index finger in the space. The Kinect Power!

This software, based on the Microsoft Kinect, uses hand motion detection based on a clustering hand detection algorithm.

The Kinect Writer

The coordinates of the hands are sent to an application that computes the inverse kinematics and the desired robot trajectory.
A raised finger sends an input. With the hand closed, the cursor is displayed on the screen without transmitting touch input.

KinectWriter

Fig. 1 Working Area of Kuka.

The method is used to send the coordinates when the finger is detected, then, when the hand is closed the trajectory is considered concluded. It is necessary to keep the hand closed (first) because when the user lifts a finger, the system sends an input signal (touch).

SwStructure

Fig.2 Software C# Multithread “Kinect Writer”.

To improve the quality of the trajectory, interpolation and moving average has been implemented.

A system of events linked to voice commands is also integrated with multiple features. To avoid the activation of unwanted commands, the user must keep both hands closed while saying the command. All controls to be tested must have a confidence level greater than a given threshold (to be decided experimentally).

The User writes: “Ciao”

The User writes: “Rosario” (My name! :-D)

Posted by: Rosario Barbagallo | February 5, 2013

Compiler C

I have developed a compiler&interpreter using the tools: FLEX and BISON (C Language) from MIL to MIN.

CompilerScheme

The lexical analyzer should take as input a MINI-L program, parse it, and output the sequence of lexical tokens associated with the program.
The software takes a syntactically-correct MINI-L program (a program that can be parsed without any syntax errors), verify that it has no semantic errors, and then generate its corresponding intermediate code.
A source program in MINI-L will be translated by the compiler into an intermediate code program, and that intermediate code program will be the input to the interpreter which will take the intermediate code, execute it and give the final results.

MIL CODE

The MINI-L Language has the following features:

  1. Integer scalar variables.
  2. Unidimensional arrays of integers.
  3. Assignment statements.
  4. While loops.
  5. If-then-else statements.
  6. Read/Write statements.
  7. Comments.

In MINI-L a comment is introduced by “- -” and goes to the end of the line. All tokens are separated by blanks, tabs, newlines, delimiters, operators or comments. A valid identifier begins with a letter and may be followed by additional letters, digits or underscores. The length of an identifier does not exceed 8 characters. Finally, MINI-L is case sensitive with all reserved words written in lower case.

Errors handled:

  1. Error Handling for the use of variables that have not been declared at the beginning of the program
  2. Error Handling for a declaration of the same variable multiple times
  3. Error Handling in the use of an array variable as a normal integer variable
  4. Error Handling negative array size
  5. Error management use of an array index greater than the maximum size declared (in the case where the index is a number)
  6. Management error identifier with less than 8 characters long and ends with ‘_’
  7. Management error identifier with more than 8 characters
Error

Syntax diagrams describing the syntax of MINI-L are given to you. Your first task in this assignment
is to develop a context-free grammar from the syntax diagrams. The parser will get the necessary information about the tokens from the lexical analyzer.

syntax1 syntax2 syntax3 syntax4 syntax5

MIN CODE

Each line in the MIN file

1. contains at most one MIL instruction

2. Each line is at most 254 characters long

3. All variables are defined before they are used

MinCode

MinCode2

Code Source Example:

program test_er;

n,x: integer;

d: integer;

a,b : array(1000) of integer;

c : array(10) of integer;

beginprogram — main program

– test op

n:= x+10*20/n-d%40+(2+20)*a(2);

– test if

if x>0 and n==d or a(10)>=c(n)

then d:=d*d*d*d; endif;

–test =

d:=11; d:=x; a(10):=2; a(10):=c(2);

– test if-else

if x+20/n>x-1*n

then n:=10+d;

if a(10)>=c(n)

then n:=d*x;

else n:=20*n;

endif;

endif;

– test while

while x<-10 or not c(2)!=0 loop

x:=x+1;

endloop;

– test read

read n;

read n,x,a(10);

endprogram

Output Code:

.n
.x
.d
.[] a, 1000
.[] b, 1000
.[] c, 10
.t0
.t1
.t2
.t3
.t4
.t5
.t6
.t7
.t8
.p0
.p1
.p2
.t9
.t10
.p3
.p4
.p5
.t11
.t12
.t13
.t14
.t15
.t16
.t17
.t18
.p6
.p7
.t19
.t20
.t21
.p8
.p9
.t22
.t23
.t24
.p10
.t25
.p11
.p12
.p13
.p14
.p15
.t26
: START
/ t0, 20,n
* t1,10,t0
% t2,d,40
– t3,t1,t2
+ t4, x,t3
+ t5,2,20
=[] t6,a,2
* t7,t5,t6
+ t8,t4,t7
= n,t8
> p0,x,0
== p1, n,d
&& p2,p0,p1
=[] t9,a,10
=[] t10,c,n
>= p3,t9,t10
|| p4,p2,p3
== p5,p4,0
?:= L000 ,p5
* t11, d,d
* t12, t11,d
* t13, t12,d
= d,t13
: L000
= d,11
= d,x
[]= a,10,2
=[] t14,c,2
[]= a,10,t14
/ t15, 20,n
+ t16, x,t15
* t17,1,n
– t18, x,t17
> p6,t16,t18
== p7,p6,0
?:= L001 ,p7
+ t19,10,d
= n,t19
=[] t20,a,10
=[] t21,c,n
>= p8,t20,t21
== p9,p8,0
?:= L002 ,p9
* t22, d,x
= n,t22
:= L003
: L002
* t23, 20,n
= n,t23
: L003
: L001
: L004
– t24,10
< p10, x,t24
=[] t25,c,2
!= p11, t25,0
! p12, p11
|| p14,p10,p12
== p15,p14,0
?:= L005 ,p15
+ t26,x,1
= x,t26
:= L004
: L005
.< n
.< n
.< x
.[]< a,10
.> n
.> x
.[]> a,10

Posted by: Rosario Barbagallo | January 19, 2013

RESTful Webservices Java (Jersey / JAX-RS)

REST is an architectural style which is based on web-standards and the HTTP protocol.

jax-rs tutorials

In a REST based architecture everything is a resource. A resource is accessed via a common interface based on the HTTP standard methods.. GET, POST.. PUT!

JAX-RS provides some annotations to aid in mapping a resource class (POJO) as a web resource.

Jersey framework is more than the JAX-RS Reference Implementation.

Start: https://jersey.java.net/

The response can be: Text,Html,Json, XML and file…

Your web services works on desktop browser, on mobile app … use Eclipse+API and write the code… can find example everywhere and they are very simple!

Posted by: Rosario Barbagallo | May 9, 2012

TUIO SKELETON (SDK V. 1.0.3.191)

TUIO SKELETON (SDK V. 1.0.3.191)

Scheme TUIO SKELETON


 TUIO SERVER sends data                            TUIO CLIENT receives data

 

How It Works

Thanks to the features  of the Tuio protocol, it has been implemented a client-server system for the transmission of the skeleton data  kinect “real time”. The results have been excellent opening new scenarios for the TUIO protocol  use.The data transmission is fast and efficiently, the protocol TUIO is redundant and provides reliability

On the left there is the server                   On the right the Client

 

The Tuio Server divides the points of the skeleton in 5 groups and each group is assigned an id.

BodySegment(0,skeletons[k].Joints[JointType.ShoulderLeft],skeletons[k].Joints[JointType.ElbowLeft], skeletons[k].Joints[JointType.WristLeft], skeletons[k].Joints[JointType.HandLeft]);

BodySegment(10, skeletons[k].Joints[JointType.ShoulderRight], skeletons[k].Joints[JointType.ElbowRight], skeletons[k].Joints[JointType.WristRight], skeletons[k].Joints[JointType.HandRight]);

BodySegment(30,skeletons[k].Joints[JointType.HipLeft],skeletons[k].Joints[JointType.KneeLeft], skeletons[k].Joints[JointType.AnkleLeft], skeletons[k].Joints[JointType.FootLeft]);

BodySegment(50,skeletons[k].Joints[JointType.HipRight],skeletons[k].Joints[JointType.KneeRight], skeletons[k].Joints[JointType.AnkleRight], skeletons[k].Joints[JointType.FootRight]);

BodySegment(80,skeletons[k].Joints[JointType.Spine],skeletons[k].Joints[JointType.ShoulderCenter], skeletons[k].Joints[JointType.Head], skeletons[k].Joints[JointType.ShoulderCenter]);

The server sends add the TuioCursor  (id, [joint.x, joint.y]) and commit the Frame:

_server.AddTuioCursor(id, (s_tmp));

_server.CommitFrame();

The client, thanks to the id, understand the type of point received.

if (tcur.getSessionID() == xxx){

}

The client graphics without problems the skeleton using five polyline.

Detail of the Client which displays the received messages in the console and the graphics rendering of points received

You can see a Demo of my application here:

OSC
Open Sound Control is a transmission protocol that allows musical instruments, computers and other devices to exchange multimedia creators that make it easy to “music performance data” in real time through a simple internal network (TCP / IP, Ethernet) or the Internet.

OSC works on high-speed network (broadband network speed), to exchange more data compared to MIDI, also giving more flexibility to the type of data available, this implies the possibility of a higher level of interaction between machines / software. The protocol for data exchange network used is the UDP.

A software can choose any number of ports, only limited by bandwidth, and also to send data to receive, and each signal has a path that can be assigned to an associated URL.

This technology allows to send different types of data including 32-bit packages, provides bandwidth for the exchange of data much larger, 1024 simultaneous messages against 128 of the MIDI, low latency and 32-bit data resolution.

TUIO PROTOCOL

The TUIO protocol is encoded using the format Open Sound Control, which provides an effective method of encoding binary data for transmission. The default transport method for the TUIO protocol is the encapsulation of binary data in OSC message within UDP packets sent to port 3333. The TUIO protocol defines two main classes of messages: SET messages and ALIVE messages. SET messages are used to communicate information about an object’s state such as position, orientation, and other recognized states. ALIVE messages indicate the current set of objects present on the surface using a list of unique Session IDs. To avoid possible errors evolving out of packet loss, no explicit ADD or REMOVE messages are included in the TUIO protocol. The receiver deduces object lifetimes by examining the difference between sequential ALIVE messages. In addition to SET and ALIVE messages, FSEQ messages are defined to uniquely tag each update step with a unique frame sequence ID. An optional SOURCE message identifies the TUIO source in order to allow source multiplexing on the client side.

Efficiency & Reliability

In order to provide low latency communication our implementation of the TUIO protocol uses UDP transport. When using UDP the possibility exists that some packets will be lost. For efficiency reasons set messages are packed into a bundle to completely use the space provided by a UDP packet. Each bundle also includes a redundant alive message to allow for the possibility of packet loss. For larger object sets, a bundle with a series of set messages, each including an alive message, are transmitted. When the surface is quiescent, alive messages are sent at a fixed rate dependent on the channel quality, for example once every second, to ensure that the receiver eventually acquires a consistent view of the set of alive objects. The state of each alive but unchanged object is periodically resent with additional set messages. This redundant information is resent at a lower rate, and includes only a subset of the unchanged objects at each update. The subset is continuously cycled so that each object is periodically addressed. Finally, each packet is marked with a frame sequence ID (fseq) message: an incrementing number that is the same for all packets containing data acquired at the same time. This allows the client to maintain consistency by identifying and dropping out-of-order packets.

 ——————————————————————————————————-

References:

http://www.tuio.org/

http://opensoundcontrol.org/

Posted by: Rosario Barbagallo | May 9, 2012

TUIO based MultiTouch Kinect (SDK V1.0) for Windows7

This article describes both the implementation of an input provider, based on the Kinect for Windows SDK v1.0 and the algorithm of hand detection. The input provider sends the coordinates to the windows driver using a clustering hand detection algorithm, which analyzes data in depth at a distance between 800 mm and 1100 mm. The driver sends the data to a virtual Human Interface Driver that translates the “touch messages”.

This input providers allows us to use any application (past and future) that works on touch devices. The developer can use this type of software to do also different applications like for example home automation applications.

The use of the TUIO protocol guarantees not only low latency but also a certain robustness because it includes redundant information to correct any lost data packets.
The possibility of transmitting data over the network allows us to use the kinect without a direct connection to the PC where the final application is installed. This way the control can be distributed over multiple machines. The Kinect has made faster processing, and it has combined the use of gestures and voice control that extends the usage scenario.

Project Information URL: http://www.barbagallogiuseppe.com/Prof/MultiTouchDriverKinect.pdf

Older Posts »

Categories

Follow

Get every new post delivered to your Inbox.