TXT Wie eine Shared Library neu laden?

Alles rund um TX(T) und RoboPro, mit ft-Hard- und Software
Computing using original ft hard- and software
Forumsregeln
Bitte beachte die Forumsregeln!
Antworten
hamlet
Beiträge: 332
Registriert: 12 Jan 2011, 21:41

TXT Wie eine Shared Library neu laden?

Beitrag von hamlet » 17 Dez 2019, 19:04

Hallo Forum,
Ich spiel gerade etwas mit dem RoboPro Shared Library Interface rum.
Hat jemand eine Idee wie man die TxtControlMain Applikation elegant dazu veranlassen kann eine neue Version einer Shared Library zu laden?

Einmal geladen verbleibt die beim ersten RoboPrp-Programm-Start geladene Version im Speicher und wird bei weiteren Programmstarts wiederverwendet, auch wenn man zwischenzeitlich eine neue Versions ins Flash geladen hat. (dlopen(…) aber kein dlclose?)

Bislang ist mir nichts Besseres eingefallen als den entsprechenden Prozess ("994 root {screen} SCREEN -A -m -d -S ROBOPRO /opt/knobloch/run.sh") zu killen (run.sh startet die TxtControlMain) und dann über "screen -A -m -d -S ROBOPRO /opt/knobloch/run.sh" genauso wieder zu starten. Wenn ich run.sh einfach im Vordergrund starte, hängt sich der TXT komischerweise komplett auf. Mit der 4.4.4 Firmware ging das noch, mit der aktuellen 4.6.6 anscheinend nicht mehr.

Man kann einen Neustart der TxtControlMain auch dadurch veranlassen, dass man ein kleines fieses "exit(0)" im finalen Library-Aufruf hinterlegt. Die TxtControlMain startet dann automatisch neu.

Richtig blöd dabei ist, dass der Neustart mit der 4.6.6 auch den SSH Server stoppt und direkt neu startet. Alle SSH Sessions werden dadurch beendet und man muss sich jedes Mal neu einloggen. … Argh, Schreiben hilft! Wenn man "run.sh" im Vordergrund startet und sich dann die shell weghängt, reißt das dann auch den ganzen TXT mit sich.

So, viel BlaBla, jetzt nochmal die Frage:
Hat jemand eine Idee wie man die TxtControlMain Applikation elegant dazu veranlassen kann eine neue Version einer Shared Library zu laden? Es wäre klasse wenn jemand Rat wüsste, meine Linux-Kenntnisse sind noch ausbaufähig (-;

Grüße,
Helmut

vleeuwen
Beiträge: 1557
Registriert: 31 Okt 2010, 22:23
Wohnort: Enschede (NL)
Kontaktdaten:

Re: TXT Wie eine Shared Library neu laden?

Beitrag von vleeuwen » 17 Dez 2019, 23:06

I am using SLI's since this possibility was available since 4.4.3. and I have the experience that after changing an SLI I need only to restart the RoboPro program 2 times to get rid of the older SLI version. And that is all.
The SLI has as objective to offer an extension to RoboPro only.
With the new web upload possibility I even don't need to make use of a tool like FIleZilla to upload the SLI.
I never have used the root account in combination with SLI's. Only a Putty terminal with screen -r for debugging under ROBOPro account.

====Google translate, I am not responsible for the quality and accuracy of these translations ==========

Ik gebruik SLI's sinds deze mogelijkheid beschikbaar was sinds 4.4.3. en ik heb de ervaring dat ik na het wijzigen van een SLI het RoboPRo-programma slechts 2 keer opnieuw hoef te starten om van de oudere SLI-versie af te komen. En dat is alles.
Het doel van de SLI is om alleen RoboPro uit te breiden.
Met de nieuwe web upload mogelijkheid hoef ik zelfs geen gebruik te maken van een tool zoals FIleZilla om de SLI te uploaden.
Ik heb nooit het root-account in combinatie met SLI's gebruikt. Alleen een Putty-terminal met scherm -r voor foutopsporing onder ROBOPro-account.
===
J'utilise SLI depuis que cette possibilité était disponible, depuis 4.4.3. et j'ai l'expérience qu'après avoir changé un SLI, je n'ai qu'à redémarrer le programme RoboPRo 2 fois pour me débarrasser de l'ancienne version SLI. Et c'est tout.
Le SLI a pour objectif de proposer une extension à RoboPro uniquement.
Avec la nouvelle possibilité de téléchargement sur le Web, je n'ai même pas besoin d'utiliser un outil comme FIleZilla pour télécharger le SLI.
Je n'ai jamais utilisé le compte root en combinaison avec les SLI. Seul un terminal Putty avec écran -r pour le débogage sous le compte ROBOPro.
===
Ich benutze SLIs, da diese Möglichkeit verfügbar war, seit 4.4.3. und ich habe die Erfahrung gemacht, dass ich nach dem Ändern eines SLI das RoboPRo-Programm nur zweimal neu starten muss, um die ältere SLI-Version zu entfernen. Und das ist alles.
Das SLI hat zum Ziel, eine Erweiterung nur für RoboPro anzubieten.
Mit der neuen Web-Upload-Möglichkeit muss ich nicht einmal ein Tool wie FIleZilla verwenden, um den SLI hochzuladen.
Ich habe das Root-Konto nie in Kombination mit SLIs verwendet. Nur ein Putty-Terminal mit screen -r zum Debuggen unter ROBOPro-Account.

Rei Vilo
Beiträge: 94
Registriert: 19 Dez 2015, 15:39

Re: TXT Wie eine Shared Library neu laden?

Beitrag von Rei Vilo » 18 Dez 2019, 08:34

It seems the SLI libraries are limited to send or receive one value at a time.

This is a serious limitation for building an interface to MQTT, with at least two values: one for the variable (e.g. 1 for temperature) and another for the value (e.g. 19,8 °C).

Let's hope a next release of RoboPro will feature either additional functions for MQTT similar to those for Bluetooth, or an extension of SLI with functions with multiple parameters.

vleeuwen
Beiträge: 1557
Registriert: 31 Okt 2010, 22:23
Wohnort: Enschede (NL)
Kontaktdaten:

Re: TXT Wie eine Shared Library neu laden?

Beitrag von vleeuwen » 18 Dez 2019, 10:33

Hi Rei,
I am using the SLI's already since they were available.
What you wrote is not completely right.
As to see in TxtSharedLibraryInterfaceDemo01 there is a mechanism to set and to get more variables but step for step.
But the limitation is that only numbers (float and int) can be used and not a string.
And the function call itself can not be indexed, like for example a call to the Motor element. With the motor element the index is M1..M4.
The variable are globale, they are not part of the function itself.

The SLI are working in the on-line mode too, but with the only restriction that the Motor lib thread is not running or can be activated.
But this can be detect in the SLI with MotorIOLib_ThreadIsRunning

Code: Alles auswählen

   if(!MotorIOLib_ThreadIsRunning()) return -2;// return with an error
To develop a dedicate MQTT client (the shared libraries are available on the TXT), IP/TCP socket server, websocket/SOAP server with civetweb, will be with a SLI not so a problem.
For the moment I am busy with developing a CivetWeb websocket based SLI.
===== Translation needed? Use this
=============================================================
===== Benötigen Sie eine Übersetzung? Verwenden Sie dies zum Beispiel. =============================
Zuletzt geändert von vleeuwen am 18 Dez 2019, 10:47, insgesamt 1-mal geändert.

hamlet
Beiträge: 332
Registriert: 12 Jan 2011, 21:41

Re: TXT Wie eine Shared Library neu laden?

Beitrag von hamlet » 18 Dez 2019, 10:47

Hi Carel,
Thanks for your answer!
vleeuwen hat geschrieben:
17 Dez 2019, 23:06
I am using SLI's since this possibility was available since 4.4.3. and I have the experience that after changing an SLI I need only to restart the RoboPro program 2 times to get rid of the older SLI version. And that is all.
Yes I can kind of reproduce this. In the first attempt to start the RoboPro program (offline/downloaded) after replacing the Shared lib in Flash, the program just hangs. This is kind of weird because on one side I don't see the log from

Code: Alles auswählen

static void __attribute__((constructor)) SharedLib_initialize(){
  printf( "########################################## libTxtTweak.so LOADED V%u\n", version);
}
I've added to get some logging when the library is loaded, and on the other side, something has definitely changed as the program just hangs. … Is the library code executed directly from flash??
However, if I then push the Stop button in the RoboPro UI, TxtControlMain logs a segmentation fault, bails out and restarts including the nasty SSH stop/start glitch closing all open remote SSH shells.
After this on the second attempt to start the program, the new version of the library is loaded correctly and the program runs smoothly.
The SLI has as objective to offer an extension to RoboPro only.
With the new web upload possibility I even don't need to make use of a tool like FIleZilla to upload the SLI.
I never have used the root account in combination with SLI's. Only a Putty terminal with screen -r for debugging under ROBOPro account.
Yes I've tried the web server, but changed back to FileZilla as for me it's more comfortable and for the console I prefer Terra Term. It took me some time to find out that "screen -ls / screen –r"-magic to get access to the logs. Now I usually watch them in a serial console. It's slower but survives TxtControlMain's SSH server restart. And I agree, root access isn't needed.

Kind Regards,
Helmut
Zuletzt geändert von hamlet am 18 Dez 2019, 12:24, insgesamt 1-mal geändert.

hamlet
Beiträge: 332
Registriert: 12 Jan 2011, 21:41

Re: TXT Wie eine Shared Library neu laden?

Beitrag von hamlet » 18 Dez 2019, 12:12

Hello SLI Fans,
I did some timing measurements on the Shared Library interface:
The test program performs every 10ms a six times burst call of the SLI function "chkTm" that just measures the time between its calls in microseconds and enters the result into a histogram. So we get 5 timing measurement each burst every 10ms. This approach simulates a real life application and shouldn't overload the TXT.
The time needed for plain clock and histogram access has also been measured: 3µs-4µs

Code: Alles auswählen

  
int chkTm( short add2hist){
    uint32_t us = tmDiff_us();
    if(add2hist) histo->add(us);
    // do it just a second time to measure the time for plain clock and histogram access
    // us = tmDiff_us();
    // if(add2hist) histo->add(us);
    return 0;
}



static uint32_t tmDiff_us(){
	timespec tm;
	static timespec tmLst = tm;
	static bool firstCall = true;

	clock_gettime( CLOCK_MONOTONIC, &tm);
	if(firstCall){
	    firstCall= false;
	    tmLst=tm;
	}
	uint32_t dtus = tmDiff_us( tmLst, tm);
	tmLst=tm ;
	return dtus;
}
static uint32_t tmDiff_us(const timespec& start, const timespec& end){
	uint32_t s = end.tv_sec-start.tv_sec;
	uint32_t us = end.tv_nsec-start.tv_nsec;
	if ( end.tv_nsec < start.tv_nsec ) {
		--s;
		us += 1000*1000*1000;
	}
	us += 500;
	us /= 1000;
	us += s * 1000*1000;
	return us;
}
SLIRoboPro.png
SLIRoboPro.png (22.02 KiB) 5067 mal betrachtet
The result doesn't look that promising:
SLIHisto.png
SLIHisto.png (23.25 KiB) 5067 mal betrachtet
About 600µs-1ms for one call to the Shared Lib!
So transmitting five parameters to the lib would take about 3ms-5ms.
This is rather painful. I guess that the task switch and the massive logging by the TxtControlMain is responsible for that.

Code: Alles auswählen

SharedLibraryInterface_WriteINT16 libTxtTweak.so chkTm 0
SharedLibraryInterface Thread wakeup libTxtTweak.so chkTm
SharedLibraryInterface_ExecuteWriteINT16 libTxtTweak.so chkTm 0
SharedLibraryInterface_ExecuteReadINT16 lib 0x4dd28
SharedLibraryInterface_ExecuteReadINT16 func 0xb404a741
SharedLibraryInterface_ExecuteWriteINT16 libTxtTweak.so chkTm RESULT 0 0
That are the logs for just one function call!

An interface that could transmit the contents of a RoboPro list or a string to the Shared lib would very useful.
Is anyone in the position to convince ft to implement something like this?

Kind Regards,
Helmut

vleeuwen
Beiträge: 1557
Registriert: 31 Okt 2010, 22:23
Wohnort: Enschede (NL)
Kontaktdaten:

Re: TXT Wie eine Shared Library neu laden?

Beitrag von vleeuwen » 18 Dez 2019, 13:47

@Hamlet
Are you running this in the on-line or off-line mode?

hamlet
Beiträge: 332
Registriert: 12 Jan 2011, 21:41

Re: TXT Wie eine Shared Library neu laden?

Beitrag von hamlet » 18 Dez 2019, 14:15

Hi Carel,
This was Offline mode, with the logging screen detached from the expensive serial console.
I've also tried to activate interface locking in the RoboPro Lib access elements, but that doesn't make any difference.
Online the situation is even worse, there it takes a few milliseconds:
SLIHisto2.png
SLIHisto2.png (31.53 KiB) 5034 mal betrachtet

Code: Alles auswählen

histogram entries, average, standard deviation, median, minimum and maximum
  n=5000
avg=3749.835200
sgm=1243.389060
mdn=3409.500000
min=929.000000
max=11585.000000
Kind Regards,
Helmut

vleeuwen
Beiträge: 1557
Registriert: 31 Okt 2010, 22:23
Wohnort: Enschede (NL)
Kontaktdaten:

Re: TXT Wie eine Shared Library neu laden?

Beitrag von vleeuwen » 18 Dez 2019, 14:54

What is your reference?
The conclusion it is worst or bad or good will only makes sense if there is an expectation and a reference.
How fast is a normal call to an RoboPro-element, for example the "Const" or "Var" element?
Now you are measuring the time to call the RoboPro "chkTm"-element only.

If there is complete closed control loop inside an SLI, this could be rather faster then the same loop build with RoboPro elements.
This because the leak of RoboPro overhead.

But the target group are RoboPro users.

hamlet
Beiträge: 332
Registriert: 12 Jan 2011, 21:41

Re: TXT Wie eine Shared Library neu laden?

Beitrag von hamlet » 18 Dez 2019, 16:16

Hi Carel,

My experience is that RoboPro is running quite performant on the TXT and the plain C/C++ code of a shared library will definitely do so. The point is that the interface between the two worlds seems to be so ridiculously slow, that it is hard to believe and I'm still looking/hoping for a bug in my test code. One short per 700us is four times slower than a 100kHz i2c link.

With the RoboPro thread calling period and the measurement update period of 10ms in mind one should keep the number of SLI calls really low.
In your PID example you have to transmit the set point and the actual measurement into the SLI and then get the result back: 3 SLI calls, 3ms worst case … for one PID controller …that sucks!
Think of an i2c driver, that has to transmit some data, e.g. the for the Pixy Cam. The interface would be way too slow.
I hoped that I could implement functions, that are painful to model in the RopoPro own Mondrian style, easily in plain C/C++. But if this comes with a severe speed penalty, this is a pity.

Kind Regards,
Helmut

vleeuwen
Beiträge: 1557
Registriert: 31 Okt 2010, 22:23
Wohnort: Enschede (NL)
Kontaktdaten:

Re: TXT Wie eine Shared Library neu laden?

Beitrag von vleeuwen » 18 Dez 2019, 18:01

In the past I have used SLI in relation with a BNO055. I had no reason to check the command speed at that moment.

I don't know the mechanism that has been used to discover and register (invoke) the functions but this is more an initialization thing.
I did it in the past in C# with reflection mechanisme.
Maybe something like this.
Why do you not save a set of system times (usec) in a area, stores this area after a loop in a log file and do the evaluation later?
This limits the overhead.

For the moment I am rather busy with exploring the CivetWeb server (CGI, websocket, LUA server site, ECMA-script server site, SQLite) , which is available on the TXT and MQTT-client. So I have no time to perform some performance test.

hamlet
Beiträge: 332
Registriert: 12 Jan 2011, 21:41

Re: TXT Wie eine Shared Library neu laden?

Beitrag von hamlet » 18 Dez 2019, 22:12

Hi Carel,
Good point, a slow reflection mechanism might be the root cause for the delay.
Many thanks for your helpful responses and good progress for your project!
Kind Regards,
Helmut

Rei Vilo
Beiträge: 94
Registriert: 19 Dez 2015, 15:39

Re: TXT Wie eine Shared Library neu laden?

Beitrag von Rei Vilo » 19 Dez 2019, 18:38

vleeuwen hat geschrieben:
18 Dez 2019, 10:33
there is a mechanism to set and to get more variables but step for step.
Thank you for the clarification. Yes, SLI can handle multiple variables, but RoboPro can only read one or write one at a time.

So sending a MQTT message requires two boxes: one to set the channel, another the set the value. Same for reading a message: set the channel, get the value.

So a possible implementation could include 3 functions:
  • set channel,
  • send value, send MQTT message /<channel>/<value>
  • receive value, read previously received MQTT message /<channel>/<value>
However, I don't know whether SLI is capable on running MQTT on background to subscribe to the messages and store them.

It is not very clean but it should work fine.

vleeuwen
Beiträge: 1557
Registriert: 31 Okt 2010, 22:23
Wohnort: Enschede (NL)
Kontaktdaten:

Re: TXT Wie eine Shared Library neu laden?

Beitrag von vleeuwen » 19 Dez 2019, 19:22

Because the SLI can not work with strings, it is needed to develop dedicated SLI's.
That is what fischertechnik does with their environment sensor.
Another solution could be to make use of a text based configuration file.
Uploading files to the TXT is now also part of the TXT, it is the default functionality of the TXT web server.
It is also possible to use USB-stick with the TXT and for example write log and data files to it.

I will make use of a configuration file for my embedded Websocket based SLI too.
I prefer to make use of a Websocket interface, because it will be probably must faster then the MQTT, this because the leak of overhead.
Raw communication instead of using REST (JSON) or SOAP (XML)
This is no problem in a local network.

Developing an SLI is not so complex, and certain not when there is a good template available.
SLI demands only basical knowledge of C/C++.

Antworten