CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate: Re: [stuckinarut] HASH-O-RAMA Data Processing Problem: Edit Log



Zhris
Enthusiast

Feb 25, 2015, 9:03 AM


Views: 16675
Re: [stuckinarut] HASH-O-RAMA Data Processing Problem

Hi,

I have thrown together a rough script which constructs an appropriate data structure then extends it while cross checking.

Please note the following:
- It is not a complete solution, but potentially the base of one.
- I have assumed that the logcall, logname and logmult are always valid since they are the operators own details, in actual fact it is the first seen entries logname and logmult that is checked against throughout. This assumption makes it alot easier when checking for invalid callsigns, callnames and callmults.
- Upon reading your comments, I wasn't entirely clear on each error, particularly the NIL ones, therefore they may not be correct.
- For now / for simplicity, only one possible error is marked against each entry.
- For now, in order to run the script standalone, I have put your input data in the DATA block at the end of the script and dumped the resultant data structure to stdout instead of writing to the relevant output CSVs.
- I have included a potential non submitters hash. The idea is everytime an invalid callsign is discovered, it increments it in the hash by 1. Those with the highest count at the end are more likely to be non submitters as oppose to invalid.

Apologies if it doesn't work quite as expected, I was unable to put much time in it for now, but I'm sure you will be able to describe any issues.


Code
use strict; 
use warnings;
use Data::Dumper;

my $configuration =
{
casesensitive => 0,
input_path => 'listQ.txt',
output_scores_path => 'logscores.csv',
output_errors_path => 'logerrors.csv',
};

my $band_lookup =
{
3 => '80M',
7 => '40M',
};

my $contestants =
{

};

my $potential_non_submitters =
{

};

while ( my $line = <DATA> )
{
$line =~ s/\s+$//;

my ( $freq, $logcall, $logname, $logmult, $callsign, $callname, $callmult ) =
map { uc $_ unless $configuration->{casesensitive} }
(split( ' ', $line ))[1,5..10];

# lookup band via frequency.
my $band = $band_lookup->{substr( $freq, 0, 1 )}; # todo // error.

# add a new contestant to contestants if we haven't seen them before.
unless ( defined $contestants->{$logcall} )
{
$contestants->{$logcall} =
{
logname => $logname,
logmult => $logmult,
};
}

# add this entry to the contestants entries.
push @{$contestants->{$logcall}->{bands}->{$band}},
{
callsign => $callsign,
callname => $callname,
callmult => $callmult,
};
}

while ( my ( $logcall, $contestant ) = each ( %$contestants ) )
{
while ( my ( $band, $entries ) = each ( %{$contestant->{bands}} ) )
{
for my $entry ( @$entries )
{
# mark as seen. Used when checking for duplicate entries.
$entry->{seen} = 1;

# validate entry.
do { $entry->{errors}->{'NIL (BANDQSO)'} = 1; $potential_non_submitters->{$entry->{callsign}}++; next } unless ( defined $contestants->{$entry->{callsign}} ); # invalid callsign.
do { $entry->{errors}->{'DUPE (BANDQSO)'} = 1; next } if ( ( grep { $_->{seen} and $_->{callsign} eq $entry->{callsign} } @$entries ) > 1 ); # duplicate entry.
do { $entry->{errors}->{'INVALID (NAME)'} = 1; next } unless ( $entry->{callname} eq $contestants->{$entry->{callsign}}->{logname} ); # invalid callname.
do { $entry->{errors}->{'INVALID (MULT)'} = 1; next } unless ( $entry->{callmult} eq $contestants->{$entry->{callsign}}->{logmult} ); # invalid callmult.
do { $entry->{errors}->{'NIL (CALLSIGN)'} = 1; next } unless ( grep { $_->{callsign} eq $logcall } @{$contestants->{$entry->{callsign}}->{bands}->{$band}} ); # no return entry.
}
}
}

print Dumper $contestants, $potential_non_submitters;

__DATA__
QSO: 7040 CW 2015-01-22 0200 W7WHY Tom OR N6ZFO BILL CA
QSO: 7040 CW 2015-01-22 0200 W7WHY Tom OR N6ZFO BILL CA
QSO: 7040 CW 2015-01-22 0200 W7WHY Tom OR N6ZFO BILL CA
QSO: 7040 CW 2015-01-22 0201 W7WHY Tom OR W9RE MIKE IN
QSO: 3542 CW 2015-01-22 0231 W7WHY Tom OR N6ZF BILL CA
QSO: 3540 CW 2015-01-22 0232 W7WHY Tom OR W6NV OLI CA
QSO: 3542 CW 2015-01-22 0246 W7WHY Tom OR W9RE MIKE IN
QSO: 7000 CW 2015-01-22 0201 W9RE MIKE IN W7WHY TOM Or
QSO: 7000 CW 2015-01-22 0221 W9RE MIKE IN N6ZFO BILL Ca
QSO: 3500 CW 2015-01-22 0231 W9RE MIKE IN N6ZFO BIL Ca
QSO: 3500 CW 2015-01-22 0246 W9RE MIKE IN W7WHY TOM Or
QSO: 3500 CW 2015-01-22 0249 W9RE MIKE IN W6NV OLI Ca
QSO: 7040 CW 2015-01-22 0201 N6ZFO BILL CA W7WHY TOM OR
QSO: 7040 CW 2015-01-22 0221 N6ZFO BILL CA W9RR MIKE IN
QSO: 7042 CW 2015-01-22 0222 N6ZFO BILL CA N2NL DAVE FL
QSO: 3543 CW 2015-01-22 0231 N6ZFO BILL CA W9RE MIKE IN
QSO: 3542 CW 2015-01-22 0231 N6ZFO BILL CA W7WHY TOM OR
QSO: 3544 CW 2015-01-22 0235 N6ZFO BILL CA W6NV OLI CA


Output:

Code
$VAR1 = { 
'N6ZFO' => {
'logname' => 'BILL',
'logmult' => 'CA',
'bands' => {
'80M' => [
{
'seen' => 1,
'callmult' => 'IN',
'callsign' => 'W9RE',
'callname' => 'MIKE'
},
{
'seen' => 1,
'callmult' => 'OR',
'errors' => {
'NIL (CALLSIGN)' => 1
},
'callsign' => 'W7WHY',
'callname' => 'TOM'
},
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'NIL (BANDQSO)' => 1
},
'callsign' => 'W6NV',
'callname' => 'OLI'
}
],
'40M' => [
{
'seen' => 1,
'callmult' => 'OR',
'callsign' => 'W7WHY',
'callname' => 'TOM'
},
{
'seen' => 1,
'callmult' => 'IN',
'errors' => {
'NIL (BANDQSO)' => 1
},
'callsign' => 'W9RR',
'callname' => 'MIKE'
},
{
'seen' => 1,
'callmult' => 'FL',
'errors' => {
'NIL (BANDQSO)' => 1
},
'callsign' => 'N2NL',
'callname' => 'DAVE'
}
]
}
},
'W7WHY' => {
'logname' => 'TOM',
'logmult' => 'OR',
'bands' => {
'80M' => [
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'NIL (BANDQSO)' => 1
},
'callsign' => 'N6ZF',
'callname' => 'BILL'
},
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'NIL (BANDQSO)' => 1
},
'callsign' => 'W6NV',
'callname' => 'OLI'
},
{
'seen' => 1,
'callmult' => 'IN',
'callsign' => 'W9RE',
'callname' => 'MIKE'
}
],
'40M' => [
{
'seen' => 1,
'callmult' => 'CA',
'callsign' => 'N6ZFO',
'callname' => 'BILL'
},
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'DUPE (BANDQSO)' => 1
},
'callsign' => 'N6ZFO',
'callname' => 'BILL'
},
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'DUPE (BANDQSO)' => 1
},
'callsign' => 'N6ZFO',
'callname' => 'BILL'
},
{
'seen' => 1,
'callmult' => 'IN',
'callsign' => 'W9RE',
'callname' => 'MIKE'
}
]
}
},
'W9RE' => {
'logname' => 'MIKE',
'logmult' => 'IN',
'bands' => {
'80M' => [
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'INVALID (NAME)' => 1
},
'callsign' => 'N6ZFO',
'callname' => 'BIL'
},
{
'seen' => 1,
'callmult' => 'OR',
'callsign' => 'W7WHY',
'callname' => 'TOM'
},
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'NIL (BANDQSO)' => 1
},
'callsign' => 'W6NV',
'callname' => 'OLI'
}
],
'40M' => [
{
'seen' => 1,
'callmult' => 'OR',
'callsign' => 'W7WHY',
'callname' => 'TOM'
},
{
'seen' => 1,
'callmult' => 'CA',
'errors' => {
'NIL (CALLSIGN)' => 1
},
'callsign' => 'N6ZFO',
'callname' => 'BILL'
}
]
}
}
};
$VAR2 = {
'N6ZF' => 1,
'W9RR' => 1,
'N2NL' => 1,
'W6NV' => 3
};


Regards,

Chris


(This post was edited by Zhris on Feb 25, 2015, 9:24 AM)


Edit Log:
Post edited by Zhris (Enthusiast) on Feb 25, 2015, 9:03 AM
Post edited by Zhris (Enthusiast) on Feb 25, 2015, 9:09 AM
Post edited by Zhris (Enthusiast) on Feb 25, 2015, 9:10 AM
Post edited by Zhris (Enthusiast) on Feb 25, 2015, 9:24 AM


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives