CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Need a Custom or Prewritten Perl Program?: I need a program that...: Re: [marky9074] Extract from text file, basic trig functions, then graph: Edit Log



Chris Charley
User

Oct 31, 2017, 11:49 AM


Views: 32219
Re: [marky9074] Extract from text file, basic trig functions, then graph

If you only need to traverse the file once, then Bill's posted solution is probably what you should use. If you need to traverse the file several times, then the best method would be to put all the data into a database.

This would be necessary with a large (1.5 GB) file as all the items in a data structure would most likely exceed your RAM memory.

Loading the database for a file this size would probably take as much time as reading through it once. But then lookups should be relatively speedy.

Below are two scripts. One loads the data into the database and the second retrieves the records. I used the SQLite database as it suffices for this problem. You would need to install DBI and DBD::SQLite. The SQLite database is included when you download DBD::SQLite.

Code
#!/usr/bin/perl 
use strict;
use warnings;
use DBI;

# see guru_fetch.pl

my $dbh = DBI->connect("dbi:SQLite:dbname=guru.lite","","",
{RaiseError => 1, PrintError => 0, AutoCommit => 0}) or die "Can't connect";

2 == @ARGV or die "Usage: perl $0 tablename filename\n";

my $table = shift;

#$dbh->do("DROP TABLE IF EXISTS $table");

$dbh->do(qq{CREATE TABLE $table
(id INTEGER,
east TEXT,
north TEXT)
});


my $sth = $dbh->prepare("INSERT INTO $table VALUES(?,?,?)");

my $format = "x a4 x a8 a9 x4 a4 x a8 a9 x4 a4 x a8 a9";
# ID E N ID E N ID E N

while (<>) {
next unless /^R\s*\d+\s/;
my @temp = unpack $format;

for my $i (0 .. $#temp) {
if ($i % 3 == 0) {
$sth->execute(@temp[$i .. $i+2]);
}
}
}
$dbh->commit;
$dbh->disconnect;



Code
#!/usr/bin/perl 
use strict;
use warnings;
use DBI;
use Math::Trig qw/asin pi/; # import arcsin & pi
use Getopt::Std;

# see guru_create.pl

my %opts;
getopts ('t:i:',\%opts);
2 == keys %opts or die "Usage: perl $0 -t tablename -i 1,564,565 ...\n";

my $table = $opts{t};
my @ids = split /,/, $opts{i};

my $dbh = DBI->connect("dbi:SQLite:dbname=guru.lite","","",
{RaiseError => 1, PrintError => 0}) or die "Can't connect";

my $sth = $dbh->prepare("SELECT north, east FROM $table where id = ?");

for my $id (@ids) {
print "\t$id\n";

$sth->execute($id);
my $aref = $sth->fetchall_arrayref;

for my $i (0 .. $#$aref-1) {
my $n = $aref->[$i+1][0] - $aref->[$i][0];
my $e = $aref->[$i+1][1] - $aref->[$i][1];
print "EAST: $e\n";
print "NORTH: $n\n";

my $dAlong = sqrt($n**2 + $e**2);
print "dAlong: $dAlong\n";

my $area = .5 * abs($n * $e);

# Area = 1/2 * b * h
# h = 2 * Area / b
my $dAcross = 2 * $area / $dAlong;
print "dAcross: $dAcross\n";

# arcsin in degrees
my $arcsin = asin($n/$dAlong) * 180 / pi;

my $azimuth = 90 - $arcsin;
print "azimuth: $azimuth\n";
print $/;
}
}
$dbh->disconnect;


Output from this is:

Code
 
C:\Old_Data\perlp>perl guru_fetch.pl -t Sample3 -i 3,7800
3
EAST: 6.5
NORTH: -4.40000000037253
dAlong: 7.84920378148499
dAcross: 3.64368167760968
azimuth: 124.094977957017

EAST: 7.20000000001164
NORTH: -4.5
dAlong: 8.49058301886082
dAcross: 3.81599236802463
azimuth: 122.005383208042

7800
EAST: 5.79999999998836
NORTH: -4.39999999944121
dAlong: 7.28010988893352
dAcross: 3.50544159168541
azimuth: 127.184706449785

EAST: 6.40000000002328
NORTH: -4.40000000037253
dAlong: 7.76659513580927
dAcross: 3.6257844666899
azimuth: 124.508522989836



I didn't understand the part you said about subtracting the eastings from the northings.

I hope this is close to what you want to do.

Update: Modified the fetch script and I'm pretty sure my calculations may not be what you're seeking, but perhaps its a start.

I named the table in the create script and the fetch script 'Sample3' so to know that data came from 'Sample3.txt'. (What naming convention you want is up to you.)

PS Your Untitled.jpg is worth a thousand words! The visualization makes the problem much easier to see.

Made a change to the create table script - faster to use a statement handle, $sth (with execute), than the original method I used, $dbh->do(... )


(This post was edited by Chris Charley on Nov 16, 2017, 11:44 AM)


Edit Log:
Post edited by Chris Charley (User) on Nov 3, 2017, 12:20 PM
Post edited by Chris Charley (User) on Nov 3, 2017, 12:23 PM
Post edited by Chris Charley (User) on Nov 3, 2017, 12:27 PM
Post edited by Chris Charley (User) on Nov 5, 2017, 6:42 PM
Post edited by Chris Charley (User) on Nov 7, 2017, 7:16 PM
Post edited by Chris Charley (User) on Nov 7, 2017, 8:45 PM
Post edited by Chris Charley (User) on Nov 16, 2017, 11:44 AM


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives