CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
Special Data Extraction

 



vinic0com
Novice

Feb 16, 2007, 9:49 PM

Post #1 of 16 (2900 views)
Special Data Extraction Can't Post

I've looked at this from several viewpoints but have yet to find a way, easy or not, to accomplish this task.

Here's a sample of the data text file created from polling some temperature sensors:

Feb 16 19:33:15 Sensor 0 C: -0.81 F: 30.54
Feb 16 19:33:16 Sensor 1 C: -0.50 F: 31.10
Feb 16 19:33:17 Sensor 2 C: -1.19 F: 29.86
Feb 16 19:43:44 Sensor 0 C: -0.81 F: 30.54
Feb 16 19:43:46 Sensor 1 C: -0.44 F: 31.21
Feb 16 19:43:47 Sensor 2 C: -1.19 F: 29.86
Feb 16 19:54:14 Sensor 0 C: -0.75 F: 30.65
Feb 16 19:54:16 Sensor 1 C: -0.44 F: 31.21
Feb 16 19:54:17 Sensor 2 C: -1.19 F: 29.86



I'm wanting to pull the most recent F data for sensors 1 and 2, and compare their values against each other.

any ideas?

vinic


KevinR
Veteran


Feb 16, 2007, 10:43 PM

Post #2 of 16 (2895 views)
Re: [vinic0com] Special Data Extraction [In reply to] Can't Post

what have you tried so far?
-------------------------------------------------


vinic0com
Novice

Feb 16, 2007, 11:13 PM

Post #3 of 16 (2892 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

The only thing I could come up with got messy quick.

I can pull out all of the data with this:



open (IN, 'tempLog.txt');
while (<IN>) {

chomp;

$month = substr $_, 0, 3;
$day = substr $_, 4, 2;
$time = substr $_, 7, 8;
$sensor = substr $_, 23, 1;
$temp = substr $_, 37, 2;

}

close (IN);

-------



and then tried to go through a series of conditions to find the most recent entry from the two important sensors, by comparing them with the last entry used, with the intent of using those for the conditional. Had trouble keeping the variables straight. I just don't know enough yet about nesting if statements.

I'm thinking there's surely a better way. Been searching to see if theres a function that can select specific lines as parameters like substr can select positions on each line. Ran across this forum in my search, but couldn't find anything even remotely similar here.


KevinR
Veteran


Feb 16, 2007, 11:16 PM

Post #4 of 16 (2891 views)
Re: [vinic0com] Special Data Extraction [In reply to] Can't Post

is the most recent data always the last two lines of the file?
-------------------------------------------------


vinic0com
Novice

Feb 16, 2007, 11:24 PM

Post #5 of 16 (2889 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

There are three entries, and for this particular function I'm only wanting to use two of the sensors.

One option I just notice that I had (I'm using digitemp to collect the data), is to put all the sensor readings on one line like this:

0 23.22 33.12 30.44
10 23.22 33.22 30.69


KevinR
Veteran


Feb 16, 2007, 11:25 PM

Post #6 of 16 (2887 views)
Re: [vinic0com] Special Data Extraction [In reply to] Can't Post

OK, so you no longer need help?
-------------------------------------------------


vinic0com
Novice

Feb 16, 2007, 11:29 PM

Post #7 of 16 (2887 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

But to answer your questionWink, yes, the new data shows up on the bottom line.

The first method would produce three lines of data, with the wealth of information.

The second method has only a sequence time, then each sensor listed in a specific order.

Given a start time and date, the time of each reading can be calculated since they are taken at a set interval (mainly for my other project of graphing the data).


vinic0com
Novice

Feb 16, 2007, 11:30 PM

Post #8 of 16 (2885 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

 


vinic0com
Novice

Feb 16, 2007, 11:32 PM

Post #9 of 16 (2884 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

well, possibly not. let me work on this a bit and see...

thanx for being a sound board!!

sometimes just helps to talk it out


KevinR
Veteran


Feb 16, 2007, 11:35 PM

Post #10 of 16 (2884 views)
Re: [vinic0com] Special Data Extraction [In reply to] Can't Post

yer welcome ... Cool
-------------------------------------------------


vinic0com
Novice

Feb 16, 2007, 11:44 PM

Post #11 of 16 (2882 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

If you're still around KevinR, did you have a way to extract the last line? I've seen a way that the pulls the first line out.


PGScooter
stranger

Feb 17, 2007, 12:32 AM

Post #12 of 16 (2878 views)
Re: [vinic0com] Special Data Extraction [In reply to] Can't Post

Sorry to but-in on the two way conversation :)

vinic, you want to get the last line from a file?

I'm not sure I understood correctly, but if that is it, maybe this would work:

I've never tried it, but I think you could do a "do while" loop. That is,


Code
do { 
$lastline=$_;
}
while (<INFILE>);


Like I've said.. never tried it.. and I don't currently have perl so can't test it out, but I think it should work cause it should get to the end, get the last line, and then return a false...

I appologize if I misunderstood your question. Either way, good luck!
The more you teach me, the more I learn. The more I learn, the more I teach.


vinic0com
Novice

Feb 17, 2007, 1:05 AM

Post #13 of 16 (2875 views)
Re: [PGScooter] Special Data Extraction [In reply to] Can't Post

It appears that you understood me scooter. These records are being collected in 10 minute intervals, so I figure if I just pull the data from the last entry (bottom line) every 10 minutes I'll be good to go.

Your code is giving me some problems though.

Here's how I set it up


Code
open (IN, 'tempLog.txt'); 
do { $lastline=$_;
}
while (<IN>) {

chomp;

$month = substr $_, 0, 3;
$day = substr $_, 4, 2;
$time = substr $_, 7, 8;
$sensor = substr $_, 23, 1;
$temp = substr $_, 37, 5;


print "$month $day $time $sensor $temp\n";

}

close (IN);


but get this


Quote
C:\DIGI\WIN>perl extract.pl
syntax error at extract.pl line 4, near ") {"
syntax error at extract.pl line 18, near "}"
Execution of extract.pl aborted due to compilation errors.



I tried this


Code
open (IN, 'tempLog.txt'); 
do { $lastline=$_
} ;
while (<IN>) {

Quote

but it reacted like the code does without the extra script.... just returns (prints) all lines of the data.


KevinR
Veteran


Feb 17, 2007, 2:18 AM

Post #14 of 16 (2874 views)
Re: [vinic0com] Special Data Extraction [In reply to] Can't Post

if the file is not too big, a few thousand lines or less, read it into an array and pop off the last line (or lines):


Code
open (IN, 'tempLog.txt');  
my @data = <IN>;
close(IN);

my $last_line = pop @data;
my $next_to_last_line = pop @data;


if the file is pretty big (many megabytes) using Tie::File can be used to the same affect.
-------------------------------------------------


davorg
Thaumaturge / Moderator

Feb 22, 2007, 4:52 AM

Post #15 of 16 (2829 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

No need to store it all in memory.


Code
open (IN, 'tempLog.txt') or die $!; 
my ($last_line, $next_to_last_line);
while (<IN>) {
$next_to_last_line = $last_line if defined $last_line;
$last_line = $_;
}


I'd probably use Tie::File. But there's also File::ReadBackwards.

--
Dave Cross, Perl Hacker, Trainer and Writer
http://www.dave.org.uk/
Get more help at Perl Monks


vinic0com
Novice

Feb 22, 2007, 7:31 AM

Post #16 of 16 (2829 views)
Re: [KevinR] Special Data Extraction [In reply to] Can't Post

Finally got back to working on this after a whirlwind trip to Nashville.

Here's an update on what seems to work well at this point:

if (done_now $ibutton_read) {
open (IN, 'c:/misterhouse/bin/tempLog.txt');
my @data = <IN>;
close(IN);
my $last_line = pop @data;
print "$last_line";
$last_line =~ /Sensor (\d+) C: (\S+) F: (\S+)/;
print "ibutton sensor=$1 temp=$3\n";
$tanktemp = $3;
my $next_to_last_line = pop @data;
$next_to_last_line =~ /Sensor (\d+) C: (\S+) F: (\S+)/;
print "ibutton sensor=$1 temp=$3\n";
$outsidetemp = $3;
print "tank temp is $tanktemp and out side temp is $outsidetemp\n";
if ( $outsidetemp > $tanktemp ) {
print "Fan turned off\n";
set $fan 'off';
}
if ( $outsidetemp < $tanktemp ) {
set $fan 'on';
print "Fan turned on\n";
}
}

I am in interested in the other function you listed Kevin and will investigate that next. I suspect that I can keep the file size down one way or the other. Only need to collect a month's worth of data at a time.

Thanx for all the help from everyone. It's good to have found a great, friendly and informed source of perl support as I struggle thru the learning process!

clyde

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives