Perl Guru Forums: Perl Programming Help: Beginner: Special Data Extraction

CGI/Perl Guide | Learning Center | Forums | Advertise | Login

Site Search: in


	MAIN INDEX	SEARCH POSTS	WHO'S ONLINE	LOG IN

Home: Perl Programming Help: Beginner:
Special Data Extraction

vinic0com
Novice

Feb 16, 2007, 9:49 PM

Post #1 of 16 (1179 views)

Special Data Extraction	Can't Post

I've looked at this from several viewpoints but have yet to find a way, easy or not, to accomplish this task.

Here's a sample of the data text file created from polling some temperature sensors:

Feb 16 19:33:15 Sensor 0 C: -0.81 F: 30.54
Feb 16 19:33:16 Sensor 1 C: -0.50 F: 31.10
Feb 16 19:33:17 Sensor 2 C: -1.19 F: 29.86
Feb 16 19:43:44 Sensor 0 C: -0.81 F: 30.54
Feb 16 19:43:46 Sensor 1 C: -0.44 F: 31.21
Feb 16 19:43:47 Sensor 2 C: -1.19 F: 29.86
Feb 16 19:54:14 Sensor 0 C: -0.75 F: 30.65
Feb 16 19:54:16 Sensor 1 C: -0.44 F: 31.21
Feb 16 19:54:17 Sensor 2 C: -1.19 F: 29.86

I'm wanting to pull the most recent F data for sensors 1 and 2, and compare their values against each other.

any ideas?

vinic

KevinR
Veteran

Feb 16, 2007, 10:43 PM

Post #2 of 16 (1174 views)

Re: [vinic0com] Special Data Extraction [In reply to]	Can't Post

what have you tried so far?
-------------------------------------------------

vinic0com
Novice

Feb 16, 2007, 11:13 PM

Post #3 of 16 (1171 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

The only thing I could come up with got messy quick.

I can pull out all of the data with this:

open (IN, 'tempLog.txt');
while (<IN>) {

chomp;

$month = substr $_, 0, 3;
$day = substr $_, 4, 2;
$time = substr $_, 7, 8;
$sensor = substr $_, 23, 1;
$temp = substr $_, 37, 2;

}

close (IN);

-------

and then tried to go through a series of conditions to find the most recent entry from the two important sensors, by comparing them with the last entry used, with the intent of using those for the conditional. Had trouble keeping the variables straight. I just don't know enough yet about nesting if statements.

I'm thinking there's surely a better way. Been searching to see if theres a function that can select specific lines as parameters like substr can select positions on each line. Ran across this forum in my search, but couldn't find anything even remotely similar here.

KevinR
Veteran

Feb 16, 2007, 11:16 PM

Post #4 of 16 (1170 views)

Re: [vinic0com] Special Data Extraction [In reply to]	Can't Post

is the most recent data always the last two lines of the file?
-------------------------------------------------

vinic0com
Novice

Feb 16, 2007, 11:24 PM

Post #5 of 16 (1168 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

There are three entries, and for this particular function I'm only wanting to use two of the sensors.

One option I just notice that I had (I'm using digitemp to collect the data), is to put all the sensor readings on one line like this:

0 23.22 33.12 30.44
10 23.22 33.22 30.69

KevinR
Veteran

Feb 16, 2007, 11:25 PM

Post #6 of 16 (1166 views)

Re: [vinic0com] Special Data Extraction [In reply to]	Can't Post

OK, so you no longer need help?
-------------------------------------------------

vinic0com
Novice

Feb 16, 2007, 11:29 PM

Post #7 of 16 (1166 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

But to answer your question Wink

, yes, the new data shows up on the bottom line.

The first method would produce three lines of data, with the wealth of information.

The second method has only a sequence time, then each sensor listed in a specific order.

Given a start time and date, the time of each reading can be calculated since they are taken at a set interval (mainly for my other project of graphing the data).

vinic0com
Novice

Feb 16, 2007, 11:30 PM

Post #8 of 16 (1164 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

vinic0com
Novice

Feb 16, 2007, 11:32 PM

Post #9 of 16 (1163 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

well, possibly not. let me work on this a bit and see...

thanx for being a sound board!!

sometimes just helps to talk it out

KevinR
Veteran

Feb 16, 2007, 11:35 PM

Post #10 of 16 (1163 views)

Re: [vinic0com] Special Data Extraction [In reply to]	Can't Post

yer welcome ... Cool

-------------------------------------------------

vinic0com
Novice

Feb 16, 2007, 11:44 PM

Post #11 of 16 (1161 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

If you're still around KevinR, did you have a way to extract the last line? I've seen a way that the pulls the first line out.

PGScooter
stranger

Feb 17, 2007, 12:32 AM

Post #12 of 16 (1157 views)

Re: [vinic0com] Special Data Extraction [In reply to]	Can't Post

Sorry to but-in on the two way conversation :)

vinic, you want to get the last line from a file?

I'm not sure I understood correctly, but if that is it, maybe this would work:

I've never tried it, but I think you could do a "do while" loop. That is,

Code

do { 
$lastline=$_; 
} 
while (<INFILE>);

Like I've said.. never tried it.. and I don't currently have perl so can't test it out, but I think it should work cause it should get to the end, get the last line, and then return a false...

I appologize if I misunderstood your question. Either way, good luck!
The more you teach me, the more I learn. The more I learn, the more I teach.

vinic0com
Novice

Feb 17, 2007, 1:05 AM

Post #13 of 16 (1154 views)

Re: [PGScooter] Special Data Extraction [In reply to]	Can't Post

It appears that you understood me scooter. These records are being collected in 10 minute intervals, so I figure if I just pull the data from the last entry (bottom line) every 10 minutes I'll be good to go.

Your code is giving me some problems though.

Here's how I set it up

Code

open (IN, 'tempLog.txt'); 
do { $lastline=$_;  
 }  
while (<IN>) {  
 
    	chomp;  
 
    	$month = substr $_, 0, 3;   
    	$day = substr $_, 4, 2; 
	$time = substr $_, 7, 8; 
	$sensor = substr $_, 23, 1;    
    	$temp = substr $_, 37, 5;        
 
 
   	 print "$month $day $time $sensor $temp\n";  
  
}  
 
close (IN);

but get this

Quote

C:\DIGI\WIN>perl extract.pl
syntax error at extract.pl line 4, near ") {"
syntax error at extract.pl line 18, near "}"
Execution of extract.pl aborted due to compilation errors.

I tried this

Code

open (IN, 'tempLog.txt'); 
do { $lastline=$_ 
 }  ; 
while (<IN>) { Quote  
but it reacted like the code does without the extra script.... just returns (prints) all lines of the data.

KevinR
Veteran

Feb 17, 2007, 2:18 AM

Post #14 of 16 (1153 views)

Re: [vinic0com] Special Data Extraction [In reply to]	Can't Post

if the file is not too big, a few thousand lines or less, read it into an array and pop off the last line (or lines):

Code

open (IN, 'tempLog.txt');  
my @data = <IN>; 
close(IN); 
 
my $last_line = pop @data; 
my $next_to_last_line = pop @data;

if the file is pretty big (many megabytes) using Tie::File can be used to the same affect.
-------------------------------------------------

davorg
Thaumaturge / Moderator

Feb 22, 2007, 4:52 AM

Post #15 of 16 (1108 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

No need to store it all in memory.

Code

open (IN, 'tempLog.txt') or die $!; 
my ($last_line, $next_to_last_line); 
while (<IN>) { 
  $next_to_last_line = $last_line if defined $last_line; 
  $last_line = $_; 
}

I'd probably use Tie::File. But there's also File::ReadBackwards.

--
Dave Cross, Perl Hacker, Trainer and Writer
http://www.dave.org.uk/
Get more help at Perl Monks

vinic0com
Novice

Feb 22, 2007, 7:31 AM

Post #16 of 16 (1108 views)

Re: [KevinR] Special Data Extraction [In reply to]	Can't Post

Finally got back to working on this after a whirlwind trip to Nashville.

Here's an update on what seems to work well at this point:

if (done_now $ibutton_read) {
open (IN, 'c:/misterhouse/bin/tempLog.txt');
my @data = <IN>;
close(IN);
my $last_line = pop @data;
print "$last_line";
$last_line =~ /Sensor (\d+) C: (\S+) F: (\S+)/;
print "ibutton sensor=$1 temp=$3\n";
$tanktemp = $3;
my $next_to_last_line = pop @data;
$next_to_last_line =~ /Sensor (\d+) C: (\S+) F: (\S+)/;
print "ibutton sensor=$1 temp=$3\n";
$outsidetemp = $3;
print "tank temp is $tanktemp and out side temp is $outsidetemp\n";
if ( $outsidetemp > $tanktemp ) {
print "Fan turned off\n";
set $fan 'off';
}
if ( $outsidetemp < $tanktemp ) {
set $fan 'on';
print "Fan turned on\n";
}
}

I am in interested in the other function you listed Kevin and will investigate that next. I suspect that I can keep the file size down one way or the other. Only need to collect a month's worth of data at a time.

Thanx for all the help from everyone. It's good to have found a great, friendly and informed source of perl support as I struggle thru the learning process!

clyde

Search for (options)

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives