Home: Perl Programming Help: Intermediate:
Remove lines



zapzap
User

Oct 20, 2013, 12:09 PM


Views: 9900
Remove lines

Greetings,

Is there a more elegant method of removing blank lines from a file:

File:
##blank first line, this comment ain't here
Line 2
Line 3

Line 5


Line 6

####end of file

I want to modify the file so there are no blank lines so file would be:

File:
Line 2
Line 3
Line 5
Line 6
#### end of file, this ain't here

So far I have

Code
s/\n{2,}/\n/g;    ##This removes 2 or more new lines but  
## doesn't remove the leading new line

s/^\n([^\s]+\n)/$1/g; ##Removes leading new line

But I don't like it, there should be a more elegant method?

TIA,
zap


(This post was edited by zapzap on Oct 20, 2013, 12:52 PM)


Laurent_R
Veteran / Moderator

Oct 21, 2013, 1:19 PM


Views: 9892
Re: [zapzap] Remove lines

Something like this:


Code
while (<$IN>) {      # reading the file line by line 
next if /^\s*$/; #skipping blank lines (empty or only white spaces)
print $OUT $_; # print other lines to output file
}



(This post was edited by Laurent_R on Oct 21, 2013, 1:19 PM)


BillKSmith
Veteran

Oct 21, 2013, 1:30 PM


Views: 9889
Re: [Laurent_R] Remove lines

Use a "one-liner":

perl -ne"print if !/^\s*$/" inifle.txt >outfile.txt

Code
 
Use single quotes on *nix.
Good Luck,
Bill


zapzap
User

Oct 21, 2013, 5:12 PM


Views: 9885
Re: [BillKSmith] Remove lines

No, this is not what I am asking for. Is there a method to basically delete a blank line from a file using perl using the substitution 's/ / /' perl function.

Maybe using the m modifier?


FishMonger
Veteran / Moderator

Oct 21, 2013, 6:19 PM


Views: 9878
Re: [zapzap] Remove lines


In Reply To
No, this is not what I am asking for. Is there a method to basically delete a blank line from a file using perl using the substitution 's/ / /' perl function.

Maybe using the m modifier?


Nope, not possible with that regex.

Bill's solution is the cleanest most elegant approach, but if you wish you could use a modified version of that as part of a larger script.

You should provide more details on what you're needing. Since you shot down Bill's solution, I can only assume that you need the solution to fit within your current script. Since you haven't shown us that script, we can't say with any confidence which one of the various approaches would suit your needs.

Please post your script.


Zhris
Enthusiast

Oct 21, 2013, 6:27 PM


Views: 9876
Re: [zapzap] Remove lines

Hi,

It can be done in a single regexp if you prefer. This also ensures that lines containing just \s+ will also be removed as I consider them blank...


Code
s/(^|\n)[\n\s]*/$1/g;


With regards to elegance, I personally have a package of "common" functions which I can export whenever I need, keeping my main code cleaner. I would add a _remove_blank_lines function to it.

Chris


(This post was edited by Zhris on Oct 21, 2013, 6:30 PM)


Zhris
Enthusiast

Oct 21, 2013, 6:29 PM


Views: 9875
Re: [FishMonger] Remove lines

I think the OP meant "string" as oppose to "file".

Chris


Kenosis
User

Oct 21, 2013, 6:45 PM


Views: 9869
Re: [Zhris] Remove lines

Nice work, Chris!

You could just:

Code
s/(^|\n)\s*/$1/g



Zhris
Enthusiast

Oct 21, 2013, 7:11 PM


Views: 9866
Re: [Kenosis] Remove lines

EDIT: both our versions strip the whitespace ;(. I thought when I tested mine it retained, my mistake. Nice spot though! Probably best to stick to multiple regexps...


Code
$str =~ s/^\s*\n//; 
$str =~ s/\n\s*\n/\n/g;
$str =~ s/\n\s*$//;




Hi Kenosis,

Unfortunately, your modification would strip the whitespace from the beginning of non blank lines, although this might be desirable...


Code
my $str = ' 

this

is

a

test

';


$str =~ s/(^|\n)\s*/$1/g;

print $str;


output:

Code
this  
is
a
test


Chris


(This post was edited by Zhris on Oct 21, 2013, 8:05 PM)


Laurent_R
Veteran / Moderator

Oct 22, 2013, 12:32 AM


Views: 9841
Re: [Zhris] Remove lines


In Reply To
I think the OP meant "string" as oppose to "file".


The OP said file in two different posts. Removing blank lines from a string is pretty easy. Doing it from a file is slightly more complicated because it more or less requires to read the file and write a new one (and doing the necessary house cleaning afterwards). There are a couple of way to do it transparently (for example with the -i command line option), but only because the necessary operations are done behind the scene.


Zhris
Enthusiast

Oct 22, 2013, 3:40 AM


Views: 9836
Re: [Laurent_R] Remove lines


Quote
The OP said file in two different posts


Thats right, theres definitely some confusion. My assumption of the OP meaning string is based on the them wanting to use a regexp substitution, which would only be useful for this particular task once the file has been slurped into a string.

Even so, you have covered both, since your code would work with strings...


Code
my $IN_str = ' 

this

is

a

test

';
my $OUT_str = '';

open my $IN, '<', \$IN_str;
open my $OUT, '>', \$OUT_str;

while (<$IN>) { # reading the file line by line
next if /^\s*$/; # skipping blank lines (empty or only white spaces)
print $OUT $_; # print other lines to output file
}

print $OUT_str;


But this is probably a little over the top.

Chris


(This post was edited by Zhris on Oct 22, 2013, 3:45 AM)


BillKSmith
Veteran

Oct 22, 2013, 9:30 AM


Views: 9818
Re: [zapzap] Remove lines

I did show how to remove blank lines from a file. You can use this to remove them from a string.

Code
use strict; 
use warnings;

my $IN_str =
"\n"
."This\n"
."\n \n"
."\n"
."is a \n"
."\n\n"
."test.\n"
;

print $IN_str;
print '*****'x5, "\n";
$IN_str =~ s/^\s*\n+//mg;
print $IN_str;


OUTPUT:

Code
 
This



is a


test.
*************************
This
is a
test.

Note: modifier /m changes the meaning of the metacharacter (^) from "begining of string" to "beginning of line". The /g modifier says to make the change on all lines that match.
Good Luck,
Bill


Kenosis
User

Oct 22, 2013, 10:11 AM


Views: 9815
Re: [BillKSmith] Remove lines

Well done, BillKSmith!


Laurent_R
Veteran / Moderator

Oct 22, 2013, 2:25 PM


Views: 9810
Re: [BillKSmith] Remove lines

Hi Bill,
I do not really know what the OP really wants, but if, as I understand it, the OP wants to remove blank lines from a file, you haven't really done that. Your Perl one-liner is creating a new file with empty lines dropped, just as the (incomplete) code snippet I posted was doing. Now the OP says that she or he wants something else. I presume that she or he probably wants to modify the original file, not just create a new file. If such is the case, it is only a small modification (adding the -i option) to the one-liner you suggested, or possibly using another a one-liner based on my proposal:


Code
perl -i.bak -ne 'print unless /^\s*$/;' filename.txt

(Replacing single quote with double quotes under Windows or VMS.)

Now, since we don't understand the OP's real intents, we might be wasting our time and I do not intend to intervene anymore on this thread so long as these intents have not been clarified by the OP.


Zhris
Enthusiast

Oct 22, 2013, 2:39 PM


Views: 9806
Re: [BillKSmith] Remove lines

Very nice Bill, I wouldn't have thought of using the m modifier.

Chris


zapzap
User

Oct 23, 2013, 12:37 AM


Views: 9798
Re: [BillKSmith] Remove lines

Excellent Job Bill! And btw Zhris I mentioned the m flag earlier. I apologize to everyone about the ambiguity of the post. It is truly important to specify either 'file' or 'string'. Thank you everyone for your suggestions.

Peace