CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
Program to filter and remove duplicates

 



Raju_P
Novice

Feb 2, 2013, 8:24 AM

Post #1 of 3 (406 views)
Program to filter and remove duplicates Can't Post

How to write a perl program to filter/sort and remove duplicates. Below is the example.

I want to sort out only cities and to remove the duplicate city names.

Input file:-

Bengaluru : City
Maharastra : State
Tamil Nadu : State
Mumbai : City
Chennai : City
Karnataka : State
Bengaluru : City
Hyderabad : City
Bengaluru : City

The out put file should only contain the city names and the repeated city names must be removed from the list(like below).

Bengaluru : City
Mumbai : City
Chennai : City
Hyderabad : City


Laurent_R
Veteran / Moderator

Feb 2, 2013, 11:04 AM

Post #2 of 3 (405 views)
Re: [Raju_P] Program to filter and remove duplicates [In reply to] Can't Post

Did you start to write something?

Assuming you know how to open and read a file line by line, you could use something like this to keep only the cities:


Code
while (my $line = <$input_file>) { 
chomp $line;
next unless $line =~ /City\s*$/; # discards lines which are not cities
my @temp_array = split /\s*:/; # split the line into an array
my $city = shift @temp-array; # get the city name
# now try to find duplicates
}


For removing duplicates, the idiomatic way is to use a hash. You check if the city name exists in the hash; if it does exist, it is a duplicate; if not, add it to to the hash and print the line out.


(This post was edited by Laurent_R on Feb 3, 2013, 2:21 AM)


BillKSmith
Veteran

Feb 2, 2013, 12:08 PM

Post #3 of 3 (402 views)
Re: [Raju_P] Program to filter and remove duplicates [In reply to] Can't Post

A truly idiomatic implementation of Laurent's algorithem.

Code
use strict; 
use warnings;
my %seen;
while (<DATA>) {
my ($name, $type) = split /\s*:\s*/;
$type =~ s/\s*$//;
next if $type ne 'City' or $seen{$name}++;
print;
}
__DATA__
Bengaluru : City
Maharastra : State
Tamil Nadu : State
Mumbai : City
Chennai : City
Karnataka : State
Bengaluru : City
Hyderabad : City
Bengaluru : City


Output:

Code
Bengaluru : City  
Mumbai : City
Chennai : City
Hyderabad : City

Good Luck,
Bill

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives