Get the last record in a CSV file

Post

Posted
Rating:
#1 (In Topic #531)
sarpomira
Avatar
Regular
sarpomira is in the usergroup ‘Regular’
Hello,
Boy am I happy I found Gambas3 and this forum :-)
After programming VB6 and VB.net for many years, moved to Python for many of the control and MQTT "internet of things" projects that I have.
I also run Linux, which limited me from good GUI applications. Until now.
Gambas is awesome, light, fast, and perfect for building GUI applications.
I run into some snags with the slight nuance differences in the BASIC syntax but that's not a big issue.
The current project is the building of a GUI dashboard to display various parameters in a greenhouse.
Light, Temperature, pumps on/off.

Currently am struggling with a CSV file and was hoping to find a few members with some experience with what I'm trying to do.
One of the greenhouse sensors publishes data to a .csv file.
The text format is:     date, time, temperature  (with no header headings).
Like this….
27/12/2020,18:23:33,13.62
27/12/2020,18:23:49,13.62
27/12/2020,18:24:04,13.62
27/12/2020,18:24:20,13.62

I was planning to use the CsvFile library from the internal gb.util
but I can't find any documentation on how to use it.
I have included the cb.lib in my project and coded the Opening of my file like this….

===============================
  Public myCSVfile As CsvFile

' Load file
  myCSVfile = New CsvFile(Application.Path & "/" & "Greehouse_Sensor_Data.txt")
  Do Until myCSVfile.Eof
     < SOME CODE HERE>
  Loop

The goal of this project is to grab the last line (specifically the last temperature reading) in the csv file.
The file gets updated every 15 seconds, but I just need to grab the last temperature reading every 10 minutes or so in order
to check it against alert/alarm settings.

Simply put:
How do I grab the last data item in a CSV file regardless of the number of records in the file?

It is not important to me whether or not the the CsvFile / gb.util method is used as long as I can read the last data point in the file.

thanks kindly
Online now: No Back to the top

Post

Posted
Rating:
#2
stevedee
Avatar
Regular
stevedee is in the usergroup ‘Regular’

sarpomira said

…How do I grab the last data item in a CSV file regardless of the number of records in the file?

It is not important to me whether or not the the CsvFile / gb.util method is used as long as I can read the last data point in the file…

Take a look at this post, but do come back if you still have a problem: Gambas One - Gambas ONE
Online now: No Back to the top

Post

Posted
Rating:
#3
cogier
Avatar
Guru
cogier is in the usergroup ‘Guru’
Hi sarpomeira and welcome to the forum.

Have a look at the attached program and see if that helps.

<IMG src="https://www.cogier.com/gambas/LastCSV.png"> </IMG>

Attachment

EDIT

Here are a few comments for the code.

Code (gambas)

  1. Public Sub Form_Open()
  2.  
  3.   Timer1.Trigger  'The timer is set for 10 mins so this just gets it run once
  4.  
  6.  
  7. Public Sub Timer1_Timer()
  8.  
  9.   Dim sFile As String[] = Split(File.Load(Application.Path &/ "Greehouse_Sensor_Data.txt"), gb.NewLine, "", True) 'This splits the file by line and ignores any blank lines
  10.  
  11.   With GridView1
  12.     .Clear
  13.     .Columns.Count = 3
  14.     .Rows.Count = 1
  15.     .Columns[0].Title = "Date"
  16.     .Columns[1].Title = "Time"
  17.     .Columns[2].Title = "Temp"
  18.     .[0, 0].Text = Split(sFile[sFile.Max])[0] 'Split the last line in the file and display the 1st item (.Max is the same as .Count -1. "," is the default split separator)
  19.     .[0, 1].Text = Split(sFile[sFile.Max])[1]
  20.     .[0, 2].Text = Split(sFile[sFile.Max])[2]
  21.     .Columns.Width = -1                       'Auto adjusts the Column widths
  23.  
  24.   Label1.Text = "Last checked at " & Str(Time(Now))
  25.  
  27.  
Online now: No Back to the top

Post

Posted
Rating:
#4
stevedee
Avatar
Regular
stevedee is in the usergroup ‘Regular’

sarpomira said

…but I just need to grab the last temperature reading every 10 minutes or so in order
to check it against alert/alarm settings…

It also occurs to me that if you only read the last (most recent) record, it might be easier to build the csv file the other way up, i.e. add the most recent record to the top of the file each time.
Online now: No Back to the top

Post

Posted
Rating:
#5
sarpomira
Avatar
Regular
sarpomira is in the usergroup ‘Regular’
 Thanks for the speedy response guys.

cogier, that example you constructed works great. Wow.

cheers
Online now: No Back to the top

Post

Posted
Rating:
#6
sarpomira
Avatar
Regular
sarpomira is in the usergroup ‘Regular’
Stevedee,

With a few mods, your code works also.
Here's how I did it.

Thanks


Code (gambas)

  1.  'Requires the addition of the gb.util library
  2. Public myCSVfile As CsvFile
  3.  
  4. Public Sub cmdRead_Click()
  5.  
  6.   Dim filepath As String
  7.   Dim aFields As String[]
  8.   Dim iRecords As Integer
  9.   Dim colRecords As New Collection
  10.   Dim thisRecord As Integer
  11.  
  12.   '************** LOAD FILE *************
  13.   myCSVfile = New CsvFile(Application.Path & "/" & "Greenhouse_Sensor_Data.txt")
  14.   Do Until myCSVfile.Eof
  15.     'read every line into a collection
  16.     colRecords[myCSVfile.Line] = myCSVfile.Read()  
  17.     'count the number of records using the Inc-rement method (adds +1 to the iRecords variable)
  18.     Inc iRecords
  19.   Loop
  20.  
  21.   'Get the Field cound of the csv file
  22.   aFields = myCSVfile.Fields
  23.  
  24.   'Get the Number of lines (Records)
  25.   iRecords = myCSVfile.line
  26.    
  27.   '********** DISPLAY DATA ***********
  28.    Me.Text = "Greenhouse Sensor Data: " & "Field count: " & aFields.Count & ", Record count: " & iRecords
  29.  
  30.   'Prints the number of records
  31.   txtRecordCount.Text = iRecords
  32.  
  33.   'Print a full record (In this case the last one)
  34.   ThisRecord = iRecords
  35.  
  36.   txtFullRecord.Text = colRecords[CStr(thisRecord)]["Date"] & "," & colRecords[CStr(thisRecord)]["Time"] & "," & colRecords[CStr(thisRecord)]["Temperature"]
  37.  
  38.   txtLastElement.text = colRecords[thisRecord]["Temperature"]
  39.  
Online now: No Back to the top
1 guest and 0 members have just viewed this.