Hi everyone,

Just wanted to pick your experts brains on something.

I'm having some system issues with my current software every 24-72 hours the software just slows down so much but I am not sure why as I only open the connects to the the database as and when needed and once I'm doing with them I close them.

And when the terminal go to sleep and is woken up again the software crawls (you can press sign on and it take 45-50 seconds for the sign on screen to show)

So could my issues be what is running in the background.

I have the following running on the pos

Xfce desktop (and some machines run just x11)
MySQL full database server
My printer controller program
My customer display control program
My status program (printer open etc)
My database sync program (only kicks in when it detects a update to the updatetable on the server)
And of course the pos application.

The pos database hold everything that is downloaded from the main server at boot up (I have a small sync program that does this)

I have been thinking about remove the MySQL server from the PoS Terminals and using something else that would be a lot lighter but I would need to make sure that I could have 2 programs access the database at any time (they would be the PoS itself and the sync program)

Can SQLite do this or would it be best to just stick with MySQL?


I have been told about these issues by 2 customers (and the software is running on 2 different machine but they are running on xfce debian 12)

I welcome any ideas as how to sort the slowing of the program and the database side (I could be on the totally wrong path here but I need to start somewhere)

 Instead of letting us guess what might be wrong based on your assumptions, have you tried monitoring the systems and see what is eating up memory or cpu time.

You think that would be an option?

gbWilly said

Instead of letting us guess what might be wrong based on your assumptions, have you tried monitoring the systems and see what is eating up memory or cpu time.

You think that would be an option?

This is why I'm asking for advise as I have no idea how to show that information on Linux

I have used the top command is that the one I need to use?

AndyGable said

This is why I'm asking for advise as I have no idea how to show that information on Linux

Luckily DuckDuckGo came up with multiple ideas to the question: how to monitor performance issues linux

1. https://www.tecmint.co…onitor-linux-performance/
2. https://geekflare.com/linux-performance-commands/
3. https://serverauth.com…ing-a-comprehensive-guide

Just the 3 first ideas of DuckDuckGo…
There where many more…

Enjoy…

AndyGable said

And when the terminal go to sleep and is woken up again the software crawls (you can press sign on and it take 45-50 seconds for the sign on screen to show)

So could my issues be what is running in the background.

As a sheer blind guess I'd say you are on the completely wrong track.

When a program goes to sleep, its active memory becomes available for other processes to use and they will progressively use more and more. Closing database connections frequently is one way of achieving this as every time you re-open the connection it will grab new memory to process cursors, sorts, etc etc.

When your program re-awakes it has to get all its active memory back which can take some time depending on how much has been re-used by something else.

Don't bother running a memory usage utility and reporting back that only a few % of memory is used. That means nothing regarding this. How and when memory is swapped out is entirely a matter for the operating system.

How much of your data is held in active memory? Why? How many images/pictures? How are you using them, as Pictures or Images?
Swapping to sqlite will only make things worse. It is a memory based dbms.

b

thatbruce said

As a sheer blind guess I'd say you are on the completely wrong track.

When a program goes to sleep, its active memory becomes available for other processes to use and they will progressively use more and more. Closing database connections frequently is one way of achieving this as every time you re-open the connection it will grab new memory to process cursors, sorts, etc etc.

When your program re-awakes it has to get all its active memory back which can take some time depending on how much has been re-used by something else.

Don't bother running a memory usage utility and reporting back that only a few % of memory is used. That means nothing regarding this. How and when memory is swapped out is entirely a matter for the operating system.

How much of your data is held in active memory? Why? How many images/pictures? How are you using them, as Pictures or Images?
Swapping to sqlite will only make things worse. It is a memory based dbms.

b

How much of your data is held in active memory? Why? How many images/pictures? How are you using them, as Pictures or Images?
No data is stored in active Memory (to my knowledge) everything is stored and recalled from the database, images IF they are used are stored as BASE64 text and is converted by the software once it has been recalled from the database

From what I have just read i fell that I could be doing my Database connections wrong as at the moment i am opening the connection to the database, doing what i need with the Database and then closing the connection

Should I instead open the connection at the start of the program and keep the same one open all the time and close it when my app exit? (I do have another Database application that access the same database on the local machine (this app is used to keep the local database in sync with the Main mysql database Server - would I need to update this as well to open the connection when starting and not close it until the app exits?)

Also I have noticed when the System it self turns off the screen (blanks it and then powers it down) it take for ever for my Gambas app to respond (if it does most of the time I have to reboot the system)

The tills are as follows
Toshiba ST-A10 2GB RAM 2GHz CPU running Debian 12 With XFCE desktop
Epson TM-88 Thermal Printer

When the system boot it created a 20mb RAM folder so the interface modules (my apps that control hardware etc) can talk and not over work the SSD that i have put in the machine

right now I can not connect to the Machine EVEN though I can see it online on my AnyDesk portal (it even slows that down to a point where it times out from the connections but once reboot everything works fine for 2-3 days and then it happens all over again)

Does anyone know of a command I can use that would log application memory etc to a file so when I have to reboot I can see what was the last data before the reboot

As I am not sure if it MySQL or my own Gambas applications that is doing this.

Andy

xfce4-taskmanager may help


<IMG src="https://community.linuxmint.com/img/screenshots/xfce4-taskmanager.png"> </IMG>

Could also use "top" from command line.

AndyGable said

How much of your data is held in active memory? Why? How many images/pictures? How are you using them, as Pictures or Images?
No data is stored in active Memory (to my knowledge) everything is stored and recalled from the database, images IF they are used are stored as BASE64 text and is converted by the software once it has been recalled from the database
That's not what I meant. From what I've seen most POS systems display grids of pretty pictures of cheese, chalk, cabbages etc to the operator. For that to happen, they must be in memory (whether that is core or GPU). So how much image information is in memory while the program is running. How they are stored is immaterial.

From what I have just read i fell that I could be doing my Database connections wrong as at the moment i am opening the connection to the database, doing what i need with the Database and then closing the connection

Should I instead open the connection at the start of the program and keep the same one open all the time and close it when my app exit? Yes! Let the dbms manage and reuse its own memory (as most genuine rdbms do these days even pschatk MySQL. Also you could learn how the dbms does reuse memory for its caches, cursors etc and maybe there is some tuning that can be done. Otherwise, every time you open the connection, it will have to create a new area to manage that client. Depending on mystical things inside the O/S like "staleness" negotiation of the memory allocations can take a finite time. Not much but depending on the number of re-connections per minute it can add up. There is no reason to work this way. This bit is not true but think about it this way. Suppose your system closed down the filesystem every time you close all the files that you are using. Then you re-open a file so the system has to start up the filesystem again. Crazy eh!

(I do have another Database application that access the same database on the local machine (this app is used to keep the local database in sync with the Main mysql database Server - would I need to update this as well to open the connection when starting and not close it until the app exits?)  That's a bit different. It would depend on how frequently the synching occurs. Once a minute? Several times a day? Once a month? Every Michaelmas? But within the sync app, I'd keep it open while it does its work and only close it when it is really not doing anything. However I dont think that the memory usage for this on the local system would be a large part of your problem. Now the memory on the central server may be a different matter.

Also I have noticed when the System it self turns off the screen (blanks it and then powers it down) it take for ever for my Gambas app to respond (if it does most of the time I have to reboot the system)
How is it arriving at the decision to "turn off the screen"? IOW what deamon is monitoring usage and turning off screen power? The system itself isn't going into some sleep mode too, is it? And God forbid it's not "hibernating" or some such silliness.


When the system boot it created a 20mb RAM folder so the interface modules (my apps that control hardware etc) can talk and not over work the SSD that i have put in the machine How have you achieved this? I doubt this is part of the issue, just interested.

right now I can not connect to the Machine EVEN though I can see it online on my AnyDesk portal (it even slows that down to a point where it times out from the connections but once reboot everything works fine for 2-3 days and then it happens all over again) No idea. Never used it. I only ever ssh into remotes. Is it as good as sliced bread?

Does anyone know of a command I can use that would log application memory etc to a file so when I have to reboot I can see what was the last data before the reboot
None that I can think of, sorry.
As I am not sure if it MySQL or my own Gambas applications that is doing this.
My money is on "Itsalottathings"

Someone mentioned using top. It's a very sophisticated tool that may give you some vision. Print the man page - there is tooo much to comprehend reading it on the screen. Then understand the Linux memory and process management parts so you can see what to display.

hth
b

thatbruce said

Someone mentioned using top. It's a very sophisticated tool that may give you some vision. Print the man page - there is tooo much to comprehend reading it on the screen. Then understand the Linux memory and process management parts so you can see what to display.

hth
b

Hi B,

Below are my reply's to your questions

That's not what I meant. From what I've seen most POS systems display grids of pretty pictures of cheese, chalk, cabbages etc to the operator. For that to happen, they must be in memory (whether that is core or GPU). So how much image information is in memory while the program is running. How they are stored is immaterial.
Yes I do this myself I use the load from string function for the picturebox and load the image to that.

Yes! Let the dbms manage and reuse its own memory (as most genuine rdbms do these days even pschatk MySQL. Also you could learn how the dbms does reuse memory for its caches, cursors etc and maybe there is some tuning that can be done. Otherwise, every time you open the connection, it will have to create a new area to manage that client. Depending on mystical things inside the O/S like "staleness" negotiation of the memory allocations can take a finite time. Not much but depending on the number of re-connections per minute it can add up. There is no reason to work this way. This bit is not true but think about it this way. Suppose your system closed down the filesystem every time you close all the files that you are using. Then you re-open a file so the system has to start up the filesystem again. Crazy eh!
Thats ok I shall update the code to just open the database connection at Boot and then close it when exits

That's a bit different. It would depend on how frequently the synching occurs. Once a minute? Several times a day? Once a month? Every Michaelmas? But within the sync app, I'd keep it open while it does its work and only close it when it is really not doing anything. However I dont think that the memory usage for this on the local system would be a large part of your problem. Now the memory on the central server may be a different matter.
The app polls a table on the main database called updateTable and if it finds anything it will process that function. It does this pooling every 60 seconds but this can be changed

How is it arriving at the decision to "turn off the screen"? IOW what deamon is monitoring usage and turning off screen power? The system itself isn't going into some sleep mode too, is it? And God forbid it's not "hibernating" or some such silliness.
in my start up scripit I have the following


Everything online says the xset stuff should turn off the sleep of the screen etc but I can confirm it does not work (I am going to need to find a better way to control it - AS I have a full Desktop install I may start to use the Display control program in settings)

How have you achieved this? I doubt this is part of the issue, just interested.
This is from my Start up scripit that does the RAM Drive setup


No idea. Never used it. I only ever ssh into remotes. Is it as good as sliced bread?
I use AnyDesk on all my GUI PoS terminals it allows me easier access to update the applications and get backups of the databases etc

None that I can think of, sorry.
No worries I am just grateful that I can bounce Ideas off people who know Linux a little better then myself.

My money is on "Itsalottathings"
I have the same feeling but I need to sort this soon as I can not really release a slow commercial program


Just incase anyone wants to see how we start our systems I have included all 3 scripits

Startup.sh

RAMDrive.sh


PoSStartUp.sh

That loop at the bottom looks wrong.


until should work on a "condition"  I can't quite see what "until RetailPos;" is evaluating?

maybe try pgrep -f to check if a program is running
something like this…

BruceSteers said

Code

until [ -n $(pgrep -f RetailPoS) ]; do
    echo 'RetailPoS has exited with a error code $?. Restarting now..' >&2
    sudo chmod +x /algPoS/Apps/RetailPoS/RetailPoS.gambas
    /algPoS/Apps/RetailPoS/RetailPoS.gambas /fullscreen /nomouse > /dev/null 2>&1
    sleep 1
done

Hi Bruce,
That stops my complete script if I exit out of my app (so that works i just need to see if I can force a crash of the app to see if it will reload the app)

I have also been doing some digging and it seems that my actual RetailPoS application is growing in memory every minute

When it starts up it is at 47.1MB in memory but after 2 mins it has gone to  76.3MB and all the app is doing is sitting on a "Closed" Screen

Below is the Code for my frmbackground (this is the main form showing)

I have 2 timers on this form one to display the time and date and the other handles files coming in from the add on modules (timer2)


and the code for the frmsigned off



frmSignedoff is showing inside a workspace on frmBackground (the full app is based on a workspace holder)

Is there a way I can see what my app is placing into memory all the time as I have checked my code and I can not see what is happening (and I have rewitten the database interface modules so it opens the connection at start of the program and not to close them until the end of the program)

This is what this section does

 Out of all that, the only thing I'd suggest trying is to get rid off all the Show/Hides and use Visible instead.
I don't know if it will help but I think setting visible on and off involves less effort than Show/Hide. You'd need to look at what happens inside the GUI to find out how they are different, probably easier just to try.

b
p.s. By the way, from what I said before, just because a control is hidden doesn't mean it is not using memory, so if any of those controls have images/pictures in them they are loaded, no matter if they are visible.

How is it arriving at the decision to "turn off the screen"? IOW what deamon is monitoring usage and turning off screen power? The system itself isn't going into some sleep mode too, is it? And God forbid it's not "hibernating" or some such silliness.
in my start up scripit I have the following

Code (gambas)

1. #!/bin/bash
2.  
3. #Action to be for running PoS System
4. xset s off
5. xset s noblank
6. xset -dpms s off
7.  
8. #Closes the XFCE Panels
9. killall xfce4-panel
10.  
11.  

Everything online says the xset stuff should turn off the sleep of the screen etc but I can confirm it does not work (I am going to need to find a better way to control it - AS I have a full Desktop install I may start to use the Display control program in settings)

Try using
You can see if this works in a terminal by using "xset q" before and after the above as it takes effect immediately.
I tried out your xsets in a terminal here. The first tWo don't affect  dpms but that simplified 3rd line does work here.

b

Now, lets get back to the actual dpms itself. I presume you are using xfce4-power-manager? What are the settings there? Particularly the System ones.

(Click to enlarge)

Also

(Click to enlarge)

Also
From the top.
Key.Code is an integer value. I see no need to waste time converting it to a string internally.
Use

(and I have rewitten the database interface modules so it opens the connection at start of the program and not to close them until the end of the program)

Did it make a difference? You may need a decent dbms monitor like dBeaver or something to see it. I think MySQL has a html front end but I have no idea how sophisticated it is.
b

thatbruce said

Also

Code (gambas)

1. Public Sub Form_KeyPress()
2.     KeyFunction(Key.Code)
3. End
4.  
5. Public Sub KeyFunction(KeyCode As String)
6.     Select Case KeyCode
7.         Case Global.Key_SignOn ' lower case A
8.             If
9.              blah blah
10.              
11.         Case Global.Key_PriceCheck

From the top.
Key.Code is an integer value. I see no need to waste time converting it to a string internally.
Use

Code (gambas)

1. Public Sub KeyFunction(KeyCode As Integer)
2.     Select Case KeyCode
3.         Case Key["a"] ' N.B. upper and lower case alphas are the same. If you really need to see if its one then you need to use Key.Shift as well.
4.              blah blah
5.  

The keys are stored in the system as numbers but stored as string. Should I change them to be integers?

thatbruce said

(and I have rewitten the database interface modules so it opens the connection at start of the program and not to close them until the end of the program)

Did it make a difference? You may need a decent dbms monitor like dBeaver or something to see it. I think MySQL has a html front end but I have no idea how sophisticated it is.
b

Yes it did the database has not gone over 15mb memory usage yet

 With all the updates that have been recommend and implemented I can say now on my development machine I have a memory staying around 65MB

But on the customers machine it has gone to 120mb

The only difference is I have on my machine 3.19.5 and on the customers machine is installed 3.19.4 runtime files.

Could the different versions be playing a part in this?

thatbruce said

How is it arriving at the decision to "turn off the screen"? IOW what deamon is monitoring usage and turning off screen power? The system itself isn't going into some sleep mode too, is it? And God forbid it's not "hibernating" or some such silliness.
in my start up scripit I have the following

Code (gambas)

1. #!/bin/bash
2.  
3. #Action to be for running PoS System
4. xset s off
5. xset s noblank
6. xset -dpms s off
7.  
8. #Closes the XFCE Panels
9. killall xfce4-panel
10.  
11.  

Everything online says the xset stuff should turn off the sleep of the screen etc but I can confirm it does not work (I am going to need to find a better way to control it - AS I have a full Desktop install I may start to use the Display control program in settings)

Try using

Code (gambas)

1. xset -dpms

You can see if this works in a terminal by using "xset q" before and after the above as it takes effect immediately.
I tried out your xsets in a terminal here. The first tWo don't affect  dpms but that simplified 3rd line does work here.

b

Now, lets get back to the actual dpms itself. I presume you are using xfce4-power-manager? What are the settings there? Particularly the System ones.Power Manager_001.png

Once I'm at the customers machine this afternoon I shall let you know the settings

Severely doubt it.

<COLOR color="#BF0080">(OOPS, this needs a quote so I and you need to know what we are talking about here. I'll be back after a little lie down now.)</COLOR>

One monitor I have looked at here today is vmstat. It does seem to give better information that may provide some pertinent data. You will need to run the following on a live pos for an entire day, which will give a LOT of data we need to look at.
 On a POS in a virtual terminal run this after starting the pos system. Let it run for an entire day
    vmstat -w -n 30 103680 -Sm
This will generate a big output to stdout, so you need to copy it somehow (piping doesn't seem to oblige vmstat, unless someone can convince me otherwise.) The things we are interested in are the swap values which might get us a little further.

thatbruce said

Severely doubt it.

<COLOR color="#BF0080">(OOPS, this needs a quote so I and you need to know what we are talking about here. I'll be back after a little lie down now.)</COLOR>

One monitor I have looked at here today is vmstat. It does seem to give better information that may provide some pertinent data. You will need to run the following on a live pos for an entire day, which will give a LOT of data we need to look at.
 On a POS in a virtual terminal run this after starting the pos system. Let it run for an entire day
    vmstat -w -n 30 103680 -Sm
This will generate a big output to stdout, so you need to copy it somehow (piping doesn't seem to oblige vmstat, unless someone can convince me otherwise.) The things we are interested in are the swap values which might get us a little further.

This has been set up on the customers machine

I've also checked and the setting for the display etc are all set off.

 Hi Everyone,

just wanted to ask a silly question while I am still getting the Memory data files for you as you have requested

Why in the xfce4-taskmanager does it show my app as org.gambas. then the PID and not the application name?
Do I need to change a setting somewhere so this can be seen?

Also I have done the following in my code

Delete every time I close the database connection and now it opens when it boot up and then closes when it exits
set all DataResult to null when I am done with them
set all global.pc (picture) as null when I am done once they have all loaded onto the from

And on my development PC my memory usage sits around 55MB but on the actual customers machine it starts at 49.4MB and it keeps increasing by 0.02MB every 2 seconds

SO I am not sure what else to search for when I am looking for memory usage now

 ****** W T F ******

Well I tried a little experiment on the customers machine just now and I shit you not i have the software sitting happily at 52.5MB of RAM usage

ALL I did was NOT launch the PoS application from the boot up script

SO does anyone know how I can still use my start up script to start my apps BUT not have them spawn from the script (as I think that is what is making it use memory)

I am thinking about making a small start up program in Gambas to launch a list of set programs and then exit (is this a good idea or should i stick with the bash scripit but make it work)