Künstlich Intelligenz und fischertechnik

[zhengtzer]

Hi


its great to see friends working on same thing
i am also just starting to study [Quality AI 9V]

have background on AI, but the documentation is just sloppy
such as file path mistake in doc

wrong
[python test-image.py -d “build/test-dataset“ -i “test-dataset/triangle/triangle01.png]

correct
[python test-image.py -d “build/test-dataset“ -i “test-dataset/triangle/triangle-01.png"]

Question:
by the way, how do you take a snapshot and save as jpeg through webcam?
in order to train own custom model


Jonathan

[Techum]

Hello Jonathan,

for the picture transmission you can use the websocket approach introduced by vleeuven. You see an example here https://www.youtube.com/watch?v=9_eZDaHBpmg, which transmits a picture stream which will allow You to isolate single pictures or You use a similar approach to transmit only single pictures.
Python code for video feed at 1:49 of the video or here:
https://git.fischertechnik-cloud.com/FH ... deo_feed_1

I had started with the tensorflow object recognition topic of AI e.g. here:
https://www.youtube.com/watch?v=IGhEEVPgGdI or here: https://www.ftcommunity.de/bilderpool/v ... alk/48063/
This is not even using the txt - the model for the recognition runs on a PC, but it is a self-trained model. I only used 30 pictures of 10 different fischertechnik parts (multiple parts on the same picture, which I tagged). The very similar parts were not reliably identified by the AI.
The video shows how to do the training part, but probably You are familiar with that and also I expect, that in the meantime there are more simple ways how to do that.

Here https://www.youtube.com/watch?v=llnVXh4ysPs I use an existing model of tensorflow - again the model runs on the PC, this time it is a mobile TXT controlled robot with Torsten's great ftrobopy.

Then I got more daring and guess what - I got stuck. I tried to use Keras with a stupid simple marble "racer" model. A bit inspired by a AI model that was presented on a fischertechnik community fair in Dreieich 2018. (shocking, how long that is back...)
The dynamics of the fast rolling ball was killing all my efforts and I had trouble to find anything comparable or a how-to.
I should have started with something more simple...

As You say, You have experience with AI... I would be interested in exploring e.g. keras with fischertechnik.
Does anybody in the forum have experience or tried how to do that?
Keras could even run locally on the txt4.0, even though I am not sure, how much sense that would make, as normally AI is quite CPU hungry.
So training would better work on a connected PC, I think.

I have here still a model of a motor that shall explain, how a ESC works, and inspired by this thread I was thinking, that that could be a very good case for a simple machine learning with fischertechnik. It is a trivial case. However, one could explore different learning and experiment e.g.
- get the maximum speed from stand still
- run with a defined rotation speed (here one could try, how the model reacts, in case of letting the motor do some work)
- start, run a certain number of rotations and stop with a maximum number of rotations

Have a nice evening
Techum

[Techum]

Hi all,
not sure, if anyone interested.

I have build this little e-motor, which could be suitable for the above mentioned experiment.

Machine

IMG-20231002-WA0000.jpg (94.38 KiB) 2333 mal betrachtet

Also, I have created a gym environment for the e-motor using ftrobopy. The env passes the gym self-check and finally it can be used by Keras reinforcement learning eg RL2. - [Remark 08.10.2023: It turns out, that it is simpler to use e.g. stable_baselines3. Keras RL2 not maintained anymore.]

I did do that with my txt (not the txt4.0) using online mode, so there could possibly be a large audience, who have a txt and eg. studied or played around with ML with the classic virtual env like cartpole or mountain_car.

I used some existing ML code to test the environment and I passed properly (random) test data to the motor.
If the right data comes, the motor rotates.
As observation I provide the total turns per epoch and the rotation speed (if there was rotation).

Now, what next?
I expect it to be quite difficult to apply the right learning method.
A child can easily make the machine rotate. E.g. if one adds three power switches - one for each of the magnets - pressing one of the switches long enough, the machine does rotate up to 40 degrees. Now if applying power to the next magnet in sequence, the rotation continues.

The ML training, that I have found though, starts by randomly applying changing values immediately, without applying a single action for more time. This does not come with much result. It makes the e-motor vibrate, but that is it. Not much to learn from.
[Remark 08.10.2023:my fault: ML was just finding the "trivial solution" vibrating around the sensor - back and fro]

Certainly one could change the env, that only every second or so a new action could be applied but that would constraint rotation speeds achievable with the model.

If anyone interested I could provide the example env in gitlab or here.
[Remark 08.10.2023 - or at Sinsheim Convention, if I can still get a space - registered only now...]

Have a nice day
Techum