Fischertechnik+smartphone

[justins]

I'm happy to annonce a new project I've been working on which could be pretty usefull to some robotics fans. I was looking into alternate ways of controlling fischertechnik robots and I figured it would be interesting to control robots using a smartphone. I've been working on a nice solution for controlling robots with a smartphone. I've written an iOS and android app which connects to the robo tx controller through a desktop application I wrote. In this way its possible to control fischertechnik models using your smartphone's accelerometer/touchscreen.
Here's a demo video: http://www.youtube.com/watch?v=9I6dy1hK9vE

The software will be avaiilable soon, I just have to finish up a few more things.
I would have wrote this in german, but my german isn't that good...
Feel free to ask me any questions if anything isn't clear (I can read german, dutch and english)

[McDoofi]

Hello,

I think it's a really great idea to use a smartphone or an iPod touch to control robots witht the TX-C, I miss(ed) it, so I am really happy to read this. As I saw in the video it works great, but is it possible to change the motor outputs in your program of the motors, which drive the car? And what are your ideas for the future? Do you will programm something as 'RoboPro' for smartphones?

Thanks for answering.

Regards McDoofi

[vleeuwen]

Can you be more clear about the architecture of your connection?
You are using the phone as a simple controller.
If I understood well:
-) TX-C in the online mode, connected with a desktop (under Windows?)
-) The smartphone is communicating with the PC.
But were you are running the implemention of the control algoritmes?

In general, it is not so difficult to bring the TX-C under control of a webserver or SOAP server which is running under Windows.
MS robotics Developer Studio does offerr for these kind of application a very good frame work.
Event for distribution over the internet, this means that several (Robot) devices can be connected with different WIndow machine and controlled in one MS-RDS program.
There are probably Services for the integration of resource like mobile phones into the MS-RDS system.
Like there are also services for the use of WII controllers, XBOX Kinect controller, etc.

[justins]

I have a server application which runs on windows, this application receives basic, commands like, left, right, stop, etc. from the smartphone. This application then reads a configuration file containing which outputs should be on and off and sends the correct commands to the robo tx interface. I'm not using MS-RDS at the moment, I still have to explore it's possibilities. The server application is just a C# .net application using the fishface libraries from ftcomputing.de. I built the server in a way that only one robot can be connected and only one client can be connected at once. The server also supports control from a webbrowser using websockets.

[justins]

Hello,

I think it's a really great idea to use a smartphone or an iPod touch to control robots witht the TX-C, I miss(ed) it, so I am really happy to read this. As I saw in the video it works great, but is it possible to change the motor outputs in your program of the motors, which drive the car? And what are your ideas for the future? Do you will programm something as 'RoboPro' for smartphones?

Thanks for answering.

Regards McDoofi

At the moment there's just a text file where you can add commands like M1Stop, M3Right, etc.
For the future I'm looking into a system where the device connects to the tx controller directly over Bluetooth. Though this will be pretty complicated and I don't know if I'll succeed in doing that.

[vleeuwen]

My suggestion is to add the dotNet application that is responsable for the connection with the smartphone into the MS-Robotics Developer Studio system as a Joystick like device.
See for example the Virtual Desktop Joystick.

For the algorithmes which are controlling the (Fichertechnik) Robot, it does not matter if the control commands are coming from the smartphone, IPad, Ipot, WII controller or balance board, XBOX Kinect, web application, etc.
This is the nice and powerful concept in MS-RDS.

[justins]

Thanks for the suggestion, I'll try out some stuff in MS-RDS. I just have one question though, can I easily distribute something made in MS-RD, will the end-user need to install any dependencies to use my application?

[vleeuwen]

Distribution is easy.
You can create a MS-RDS install package with
.\bin\dssdeploy.exe deployment tool.

The end user will only need the service binaries.
The tool need only the contract name of your service and it create the package with all the alternated services in it.

[justins]

I was wondering, what is the real advantage of using MS-RDS instead of just calling the functions from the ftscmlib dll?

[ftDirk]

@justins:
Did you already use the new ftMscLib.dll for the TX-C firmware 1.24 for your project?

[justins]

I'm currently using the old FtMscLib for TX-C firmware 1.18, did allot change in version 1.24?

[vleeuwen]

With MS-RDS you get a complete infrastructure for robot interfacing and sing other resources.
VPL is offering a way to control robot on a very abstract level.
A nice visual programming language. Message flow oriented concept is nice and more powerful then work flow oriented concept,
A 3D simulation environnement.
The CCR and DSS technologie.
Real concurrency both in case of more cores or more CPU’s (also distribution over a network).
Because of the layer concept of MS-RDS, you don’t notice these technologies at the top level.
The ability to develop you own Visual elements. In fact the Visual element is a kind of a summary from your text based development.
Notification (event) driven programming.
etc

[vleeuwen]

1.24 firmaware is part of RoboPro 3.1.2.
1.24 differens on some points from 1.18.
See the new FtMscLib documentation on the Fischertechnik download site.

[ft-ninja]

Hello justins,

I like your idea. I'm also working on application that allow to control the robotx with the handy directly through bluetooth. Check this link - http://robotxcontrol.sourceforge.net.

[vleeuwen]

Suggestion:
This problem can be solved very simple in case the cell phone is able to act as a node in the TX-C Bluetooth message system.
I that case the cell phone can send and receive the normal command messages as they are in use inside RoboPro.
I that case a RoboPro program is able to dela with the messages.

I did this before white the RoboInterface on RS232 level and the message system that runs on the Robo Interface

[justins]

@ft-ninja, wow great job, I see you managed to make a direct connection from your handy to the robot. Pretty soon I won't need to bring my computer to exhibitions anymore. I've been trying to do something like this with the iPhone, but I think I probably have to gain some more knowledge about Bluetooth before I'll get anything working.

I guess my server solution has one great advantage though, I could have anyone control my robot over the internet using this little control panel.


@vleeuwen having the cell phone act as a node in the TX-C Bluetooth message system is a good idea, but still very difficult for me to accomplish I think. Though I was also thinking of making my server application hook onto robo pro and send it messages, so people can integrate the smartphone control into their existing robo pro programs.

[vleeuwen]

@Justin
You need to determine the right BT protocol but that seems to be rather difficult.

Do you have an idea of how to hook onto your program to RoboPro?
Do you now how RoboPro operates?

WIth the Robo Inerface FtLib (FtLib) there were two calls which enable data exchange in the offline mode.
This call does not exist in the FtMscLib.
Before the integration of the FT(Msc)Lib into the executable there was an other possibility but this is past.

A good web server to connect the FtMscLib is not new. In the past there were more example of Ipad etc. doing the same.
The MS-RDS build in web server HTTP part is also offering bidirectional HTTP traffic, this enables you to see the actual Input state on you virtual joystick. In case of the use of AJAX and SIlverlight this can gice a very nice virtual control screen.

The main problem with two BT connection (TX-C<->PC PC<-> smartphone) is that the double latency would be a problem in case of a closed control loop that is running on the smartphone.

[justins]

Yes, I managed to hook my program onto Robo pro, using a couple windows APIs to send it messages, simulating button presses. This isn't the perfect sollution yet, but it seems to work pretty well. Here's a small demo: http://youtu.be/vuiZ4zEf_m0

[vleeuwen]

Looks intresting and this could also opens another way to a connection with the XBOX Kinect.
With MS-RDS this is very easy.

[justins]

I'll be open sourcing all the software I wrote pretty soon.