CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Beginner:
How to get perl inline option working on empty files?

 



matteoguglielmi
Novice

Aug 15, 2014, 6:38 AM

Post #1 of 18 (790 views)
How to get perl inline option working on empty files? Can't Post


Code
touch filetest 

perl -i -ple 'BEGIN {$tmp = shift(@ARGV)} if (eof) {print; $_ = $tmp}' 'MyNewLineAtTheEndOfFiletest' ./filetest

cat filetest
#no output! still empty!


Filetest is still an empty file despite the -i option, how to get this fixed?


(This post was edited by matteoguglielmi on Aug 15, 2014, 8:45 AM)


FishMonger
Veteran / Moderator

Aug 15, 2014, 8:18 AM

Post #2 of 18 (784 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post

What were you expecting to be written to filetest and what is the real problem you're wanting to solve that prompted you to use this as the solution?


matteoguglielmi
Novice

Aug 15, 2014, 8:43 AM

Post #3 of 18 (780 views)
Re: [FishMonger] How to get perl inline option working on empty files? [In reply to] Can't Post

The above command just appends a new line to (the end of) a given file. If the file is not empty, the above command just works perfectly (a new line is appended). If the file does not exist, perl complains about that... which is OK (the file is not created automatically and the new line is not appended to it). The problem I find is that if the file exists and it is empty, the line is not appended to it either (the loop (-p option) is never started when the file is empty... indeed there are no lines it!). But, since the file exists though, I'd actually like perl to just append the new line anyway.

How could that one-liner be modified in order to achieve this?

I've actually found a partial solution to the problem:


Code
perl -i -ple 'BEGIN {$tmp=shift(@ARGV); if (-z $ARGV[0]) {open(my $fh, ">$ARGV[0]"); print $fh "$tmp"; close $fh; exit 0}} if (eof) {print; $_ = $tmp}' \ 
'MyNewLineAtTheEndOfFiletest' ./filetest


But doing so, the -i option becomes irrelevant since I'm always writing directly to the filehandle (without -i, I'd like to get the output printed to the standard output, as usual... in my code, it will always be written to the filehandle instead and the file will always be modified).


(This post was edited by matteoguglielmi on Aug 15, 2014, 9:00 AM)


FishMonger
Veteran / Moderator

Aug 15, 2014, 9:04 AM

Post #4 of 18 (769 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post

The easiest and best solution to what you say you're wanting to achieve would be this:

Code
echo 'MyNewLineAtTheEndOfFiletest' >> filetest

filetest does not need to exist prior to that commad.


(This post was edited by FishMonger on Aug 15, 2014, 9:15 AM)


FishMonger
Veteran / Moderator

Aug 15, 2014, 9:10 AM

Post #5 of 18 (765 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post


Code
perl -MTie::File -e '$tmp=shift; tie @array, "Tie::File",shift; push @array, $tmp; untie @array;' 'MyNewLineAtTheEndOfFiletest' ./filetest



Code
perl -MTie::File -e 'tie @array,"Tie::File",$ARGV[1]; push @array,$ARGV[0]; untie @array;' 'MyNewLineAtTheEndOfFiletest' ./filetest



Code
perl -e 'system "echo $ARGV[0] >> $ARGV[1]"' 'MyNewLineAtTheEndOfFiletest' ./filetest



(This post was edited by FishMonger on Aug 15, 2014, 9:19 AM)


matteoguglielmi
Novice

Aug 15, 2014, 9:19 AM

Post #6 of 18 (753 views)
Re: [FishMonger] How to get perl inline option working on empty files? [In reply to] Can't Post

Your one-liners are really equivalent to:


Code
echo 'NewLine' >> ./filetest


but I need to get the -i option working as well...


(This post was edited by matteoguglielmi on Aug 15, 2014, 9:21 AM)


FishMonger
Veteran / Moderator

Aug 15, 2014, 9:21 AM

Post #7 of 18 (747 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post

Why do you "need" to use the -i option?

Sounds like you have an XY problem.


(This post was edited by FishMonger on Aug 15, 2014, 9:22 AM)


matteoguglielmi
Novice

Aug 15, 2014, 9:23 AM

Post #8 of 18 (744 views)
Re: [FishMonger] How to get perl inline option working on empty files? [In reply to] Can't Post

... because sometimes I need to leave the file unmodified but get the actual result (whole file + new line appended) printed on the standard output (so that I can redirect it to a new file, for instance).

This is my code:


Code
printToStdOut() { 
[ "$1" ] || {
printToStdOut "USAGE: $FUNCNAME MESSAGE"
return 0
}

printf "%s\n" "$*"
}

printToStdErr() {
[ "$1" ] || {
printToStdOut "USAGE: $FUNCNAME MESSAGE"
return 0
}

printf "%s\n" "$*" >&2
}

assertVarNameIsValid() {
[ "$1" ] || {
printToStdOut "USAGE: $FUNCNAME VARNAME"
return 0
}

case $1 in
[!a-zA-Z_]*|*[!a-zA-z0-9_]*) return 1 ;;
esac
}

assertVarIsNotSet() {
[ "$1" ] || {
printToStdOut "USAGE: $FUNCNAME VARNAME"
return 0
}

assertVarNameIsValid "$1" && [ "${!1+X}" != X ]
}

assertFileExists() {
[ "$1" ] || {
printToStdOut "USAGE: $FUNCNAME FILE"
return 0
}

[ -e "$1" ] || return 1
}

assertFileIsNotEmpty() {
[ "$1" ] || {
printToStdOut "USAGE: $FUNCNAME FILE"
return 0
}

assertFileExists "$1" && [ -s "$1" ] || return 1
}


... and the appendLine function:


Code
appendLine() { 
case $# in
0|1)
printToStdOut "USAGE: [iNLINE=] $FUNCNAME LINE FILE1 [FILE2 ... FILEN]"
;;
*)
local lINE="$1" && shift
local oPTS fILE fAULT=0

assertVarIsNotSet iNLINE || oPTS='-i'

for fILE in "$@"; do
if assertFileExists "$fILE"; then
if assertFileIsNotEmpty "$fILE"; then
perl $oPTS -ple 'BEGIN {$tmp=shift(@ARGV)} if (eof) {print; $_=$tmp}' "$lINE" "$fILE"
else
assertVarIsNotSet iNLINE && echo "$lINE" || echo "$lINE" > "$fILE"
fi
else
assertVarIsNotSet iNLINE &&
{ printToStdErr "ERROR: $FUNCNAME: File \"$fILE\" does not exist."; fAULT=1; } ||
echo "$lINE" > "$fILE"
fi
done

return $fAULT
;;
esac
}



(This post was edited by matteoguglielmi on Aug 15, 2014, 9:40 AM)


FishMonger
Veteran / Moderator

Aug 15, 2014, 9:39 AM

Post #9 of 18 (735 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post

When using -i (along with the -p option) the file contents will not be sent to stdout, so I don't see how using -i fits into your need to redirect stdout to a new file.

I still see this as an XY problem.


matteoguglielmi
Novice

Aug 15, 2014, 9:44 AM

Post #10 of 18 (733 views)
Re: [FishMonger] How to get perl inline option working on empty files? [In reply to] Can't Post


Code
[software@sbsrv14:~/tmp]$ iNLINE= appendLine 'NewLine' filetest 
[software@sbsrv14:~/tmp]$ iNLINE= appendLine 'NewLine' filetest
[software@sbsrv14:~/tmp]$ iNLINE= appendLine 'NewLine' filetest
[software@sbsrv14:~/tmp]$ cat filetest
NewLine
NewLine
NewLine
[software@sbsrv14:~/tmp]$ appendLine 'NewLine' filetest
NewLine
NewLine
NewLine
NewLine
[software@sbsrv14:~/tmp]$ cat filetest
NewLine
NewLine
NewLine
[software@sbsrv14:~/tmp]$ appendLine 'NewLine' filetest > newfile
[software@sbsrv14:~/tmp]$ cat newfile
NewLine
NewLine
NewLine
NewLine



matteoguglielmi
Novice

Aug 15, 2014, 9:53 AM

Post #11 of 18 (726 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post

And to complete the picture:


Code
appendIfNoSuchLine() { 
case $# in
0|1)
printToStdOut "USAGE: [iNLINE=] [pERLPATTERN=regexp] $FUNCNAME LINE FILE1 [FILE2 ... FILEN]"
;;
*)
local lINE="$1" && shift
local oPTS fILE fAULT=0

assertVarIsNotSet iNLINE || oPTS='-i'
assertVarIsSet pERLPATTERN ||
pERLPATTERN=$(perl -e '$tmp=shift(@ARGV); $tmp="\Q$tmp\E"; print "$tmp"' "$lINE")

for fILE in "$@"; do
if assertFileExists "$fILE"; then
if assertFileIsNotEmpty "$fILE"; then
if pERLPATTERN="$pERLPATTERN" perl -ne 'if (/$ENV{pERLPATTERN}/o) {exit 1}' "$fILE"; then
perl $oPTS -ple 'BEGIN {$tmp=shift(@ARGV)} if (eof) {print; $_=$tmp}' "$lINE" "$fILE"
else
printToStdOut "$FUNCNAME: Pattern found. Nothing to append to file \"$fILE\"."
fi
else
assertVarIsNotSet iNLINE && echo "$lINE" || echo "$lINE" > "$fILE"
fi
else
assertVarIsNotSet iNLINE &&
{ printToStdErr "ERROR: $FUNCNAME: File \"$fILE\" does not exist."; fAULT=1; } ||
echo "$lINE" > "$fILE"
fi
done

return $fAULT
;;
esac
}



matteoguglielmi
Novice

Aug 15, 2014, 10:00 AM

Post #12 of 18 (722 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post


Code
[software@sbsrv14:~/tmp]$ cat filetest 
NewLine
NewLine
NewLine
[software@sbsrv14:~/tmp]$ appendIfNoSuchLine 'NewLine' filetest
appendIfNoSuchLine: Pattern found. Nothing to append to file "filetest".
[software@sbsrv14:~/tmp]$ pERLPATTERN='b$' appendIfNoSuchLine 'NewLine' filetest
NewLine
NewLine
NewLine
NewLine
[software@sbsrv14:~/tmp]$ cat filetest
NewLine
NewLine
NewLine
[software@sbsrv14:~/tmp]$ iNLINE= pERLPATTERN='b$' appendIfNoSuchLine 'NewLine' filetest
[software@sbsrv14:~/tmp]$ cat filetest
NewLine
NewLine
NewLine
NewLine
[software@sbsrv14:~/tmp]$ appendLine 'b' filetest
NewLine
NewLine
NewLine
NewLine
b
[software@sbsrv14:~/tmp]$ iNLINE= appendLine 'b' filetest
[software@sbsrv14:~/tmp]$ iNLINE= pERLPATTERN='b$' appendIfNoSuchLine 'NewLine' filetest
appendIfNoSuchLine: Pattern found. Nothing to append to file "filetest".



FishMonger
Veteran / Moderator

Aug 15, 2014, 10:56 AM

Post #13 of 18 (717 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post

To understand why -i is not appending when the file was empty, we'd need to run the code through one of perl's low level debugging modules that can show what's going on "under the hood". However, when I do that I get back code that I don't understand.


Code
[root@099-91-RKB-2 rkb]# perl -MO=Concise -i -pe 'BEGIN {$tmp = shift(@ARGV)} print $tmp if eof' 'MyNewLineAtTheEndOfFiletest' ./filetest 
x <@> leave[1 ref] vKP/REFC ->(end)
1 <0> enter ->2
2 <;> nextstate(LINE: main 5 -e:0) v:{ ->3
w <2> leaveloop vKP/2 ->x
3 <{> enterloop(next->c last->w redo->4) v ->r
- <1> null vK/1 ->w
v <|> and(other->4) vK/1 ->w
u <1> defined sK/1 ->v
- <1> null sK/2 ->u
- <1> ex-rv2sv sKRM*/1 ->s
r <$> gvsv(*_) s ->s
t <1> readline[t1] sKS/1 ->u
s <$> gv(*ARGV) s ->t
- <@> lineseq vK ->-
b <@> leave vKP ->c
4 <0> enter v ->5
5 <;> nextstate(main 2 -e:1) v:{ ->6
- <1> null vK/1 ->b
7 <|> and(other->8) vK/1 ->b
6 <0> eof s ->7
a <@> print vK ->b
8 <0> pushmark s ->9
- <1> ex-rv2sv sK/1 ->a
9 <$> gvsv(*tmp) s ->a
p <@> leave vKP ->q
c <0> enter v ->d
d <;> nextstate(main 3 -e:0) v:{ ->e
- <1> null vK/1 ->p
h <|> or(other->i) vK/1 ->p
g <@> print sK ->h
e <0> pushmark s ->f
- <1> ex-rv2sv sK/1 ->g
f <$> gvsv(*_) s ->g
o <@> die[t5] vK/1 ->p
i <0> pushmark s ->j
- <1> ex-stringify sK/1 ->o
- <0> ex-pushmark s ->j
n <2> concat[t3] sKS/2 ->o
l <2> concat[t2] sK/2 ->m
j <$> const(PV "-p destination: ") s ->k
- <1> ex-rv2sv sK/1 ->l
k <$> gvsv(*!) s ->l
m <$> const(PV "\n") s ->n
q <0> unstack v ->r
-e syntax OK


My best guess is that as soon as the file is opened it detects eof and closes the filehandle since there is nothing for it to print/output.

Looks like you'll need to adjust your approach so you can accommodate the possibility of an empty file. I've shown several different approaches and can come up with others if needed.


Laurent_R
Veteran / Moderator

Aug 16, 2014, 1:54 PM

Post #14 of 18 (668 views)
Re: [FishMonger] How to get perl inline option working on empty files? [In reply to] Can't Post

Perhaps not so low level deparsing will help showing what if going on:


Code
$ perl -MO=Deparse,-p -i -ne 'print $_; print "foobar" if eof' key_source.pl 
BEGIN { $^I = ".bak"; }
LINE: while (defined(($_ = <ARGV>))) {
print($_);
(eof and print('foobar'));
}
-e syntax OK


In other words, you need to have at least one line in the input file to get into the loop where the last line is appended on detecting eof.

With a non empty file,

Code
$ perl -i -ne 'print $_; print "foobar" if eof' key_source.pl

does indeed add "foobar" at the end of the key_source.pl file. With an empty file, it does not.

It appears that even with an END block, it won't work:

Code
$ perl -MO=Deparse,-p -i -ne 'print $_; END { print "foobar\n";}' key_source.pl 
BEGIN { $^I = ".bak"; }
LINE: while (defined(($_ = <ARGV>))) {
print($_);
sub END {
print("foobar\n");
}
;
}
-e syntax OK


Just tried this:

Code
$ touch foo.txt 

$ perl -i -ne 'print $_; END { print "foobar\n";}' foo.txt
foobar

$ cat foo.txt

So foobar gets printed, but only to the screen, not to the empty foo.txt file.


FishMonger
Veteran / Moderator

Aug 16, 2014, 2:34 PM

Post #15 of 18 (663 views)
Re: [Laurent_R] How to get perl inline option working on empty files? [In reply to] Can't Post

-MO=Deparse is what I first tried, but it doesn't show what I was really wanting to find out which was "why wasn't it appending when the file was initially empty".

To answer that, we need to know the details of how and when the opening/reading/writing of the filehandle takes place.

After posting my comment I thought about (and tried) using strace to get those details, but it didn't answer the question to my satisfaction.

I believe the low level output from -MO=Concise answers the question, but we just don't know how to decipher its output.

I tried putting the print $tmp if eof in both the BEGIN and END blocks, but neither of them worked. Probably because the file gets opened after the BEGIN block and closed prior to the END block.

I believe that my assumption, while not completely or technically correct, is pretty close to what is going on. Specifically, the file is opened and the first readline is preformed prior to executing any of our runtime statements. At that point it already hit eof, so the if eof test we run returns false.


Laurent_R
Veteran / Moderator

Aug 16, 2014, 3:32 PM

Post #16 of 18 (661 views)
Re: [FishMonger] How to get perl inline option working on empty files? [In reply to] Can't Post

I agree with about everything you say. But...


In Reply To
-MO=Deparse is what I first tried, but it doesn't show what I was really wanting to find out which was "why wasn't it appending when the file was initially empty".


I still think that it shows fairly clearly why it does not work. (Which does not mean that this grants us a solution.) And why even the END block does not help.


In Reply To
I believe the low level output from -MO=Concise answers the question, but we just don't know how to decipher its output.

I fully agree that this is likely to be the solution, but I am just totally unable to decypher those opcodes.


matteoguglielmi
Novice

Aug 18, 2014, 8:03 AM

Post #17 of 18 (594 views)
Re: [Laurent_R] How to get perl inline option working on empty files? [In reply to] Can't Post

Thanks guys for your effort.

I think I'll keep using the simplest (bash) way of echoing the new line to the empty file:


Code
assertVarIsNotSet iNLINE && echo "$lINE" || echo "$lINE" > "$fILE"



Laurent_R
Veteran / Moderator

Aug 18, 2014, 10:04 AM

Post #18 of 18 (585 views)
Re: [matteoguglielmi] How to get perl inline option working on empty files? [In reply to] Can't Post

I think that you don't need double quotes for your shell variables. And beware that your bash code line above will clobber the content of existing files. You probably rather want this:


Code
echo $LINE >> $fILE


 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives