CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Shell command with embedded pipe: Illegal seek on open()

 



rpaskudniak
User


Oct 13, 2015, 8:39 AM

Post #1 of 11 (3002 views)
Shell command with embedded pipe: Illegal seek on open() Can't Post

Greeting, Folks.

Here a command line whose output I want to process:

Code
$ cdr list replicate brief | tail +4

That is, I know that the cdr command spits out 3 heading lines and I wan to skip them. So here is the Perl code I use for it:

Code
my $cdr_cmd = "cdr list replicate brief | tail +4 |"; 
my $cdr; # File handle for above cdr command
open($cdr, $cdr_cmd)
or die "Error <$!> opening command <$cdr_cmd>";
while ($cdr_line = <$cdr>)
{
...
}

I was getting errors about an unitialized file handle so I went into the debugger and stopped before that while loop, entering this manually:


Code
  DB<2> $x = <$cdr> 
Use of uninitialized value in <HANDLE> at (eval 18)[/usr/local/lib/perl5/5.8. /perl5db.pl:628] line 2.
at (eval 18)[/usr/local/lib/perl5/5.8.7/perl5db.pl:628] line 2
eval '($@, $!, $^E, $,, $/, $\\, $^W) = @saved;package main; $^D = $^D | $DB::db_stop;
$x = <$cdr>;

;' called at /usr/local/lib/perl5/5.8.7/perl5db.pl line 628
DB::eval called at /usr/local/lib/perl5/5.8.7/perl5db.pl line 3410
DB::DB called at ./gen-sync-repls.pl line 9
readline() on unopened filehandle at (eval 18)[/usr/local/lib/perl5/5.8.7/perl5db.pl:628] line 2.
at (eval 18)[/usr/local/lib/perl5/5.8.7/perl5db.pl:628] line 2
eval '($@, $!, $^E, $,, $/, $\\, $^W) = @saved;package main; $^D = $^D | $DB::db_stop;
$x = <$cdr>;

;' called at /usr/local/lib/perl5/5.8.7/perl5db.pl line 628
DB::eval called at /usr/local/lib/perl5/5.8.7/perl5db.pl line 3410
DB::DB called at ./gen-sync-repls.pl line 9

This frustrated, I decided look at $! before and after the open.
Before: Empty.
After: "Illegal seek"

Apparently, the open() function does not like a pipe embedded in the command (before the terminating pipe). On this guess, I took out the embedded pipe and explicitly counted lines, adding:

Code
next if (++line_count <= 3)

That solved the problem.

But it is a klutzy way to handle this, IMO. What would I have to do if my command were a long chain of piped commands. Y'all surely see that my workaround is a lousy solution! Unsure

So what is the proper way to handle this?

Another question (Sheesh! Dis guy can't shut up! Wink ):
If my open() command was getting that "Illegal seek" error, why didn't my program just barf right at the open()?

Thanks mucho!
--------------------
-- Rasputin Paskudniak (In perpetual pursuit of undomesticated, semi-aquatic avians)


FishMonger
Veteran / Moderator

Oct 13, 2015, 9:19 AM

Post #2 of 11 (3000 views)
Re: [rpaskudniak] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

Use the 3 arg form of open. That will solve the problem of the multiple pipes interfering with the open call.


(This post was edited by FishMonger on Oct 13, 2015, 9:20 AM)


FishMonger
Veteran / Moderator

Oct 13, 2015, 9:36 AM

Post #3 of 11 (2996 views)
Re: [rpaskudniak] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post


Quote
That is, I know that the cdr command spits out 3 heading lines and I wan to skip them.

To do that, read the filehandle in scalar context 3 times (in void context);

Code
<$cdr> for 1..3;



(This post was edited by FishMonger on Oct 13, 2015, 9:39 AM)


rpaskudniak
User


Oct 13, 2015, 11:46 AM

Post #4 of 11 (2991 views)
Re: [FishMonger] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

Fishmonger said:

Quote
Use the 3 arg form of open. That will solve the problem of the multiple pipes interfering with the open call.

I've just read through the perldoc page at http://perldoc.perl.org/functions/open.html and I may have missed the part where it explains the three-parameter open() for running commands. Please give me a specific syntax to follow. Thanks.

Fishmonger also said:
To do that, read the filehandle in scalar context 3 times (in void context);
<$cdr> for 1..3;
Rasputin slaps himself on the forehead "Why didn't *I* think of that?!" (Don't answer that..Crazy) I just revised the code to use this and it's MUCH cleaner!
Thanks!

Fishmonger didn't say ..
How the program didn't barf if it had the "Illegal seek" error on the open().

Gratefully yours,
--------------------
-- Rasputin Paskudniak (In perpetual pursuit of undomesticated, semi-aquatic avians)


FishMonger
Veteran / Moderator

Oct 13, 2015, 12:09 PM

Post #5 of 11 (2987 views)
Re: [rpaskudniak] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

The very first example in the doc shows the 3 arg form. 1st arg is the filehandle, 2nd arg is the mode, and 3rd is the expression (what is being opened).

Code
my $cdr_cmd = "cdr list replicate brief"; 

open(my $cdr, '-|', $cdr_cmd)
or die "Error <$!> opening command <$cdr_cmd>";



Quote
How the program didn't barf if it had the "Illegal seek" error on the open().

The open call to the forked command did not fail which is why perl did not "barf" on that statement. It was your piped command that had a runtime error when you tried to read its output.

If you want better control when forking that process, then you should use IPC::Run or IPC::Open3.
http://search.cpan.org/~toddr/IPC-Run-0.94/lib/IPC/Run.pm
http://search.cpan.org/~shay/perl-5.20.3/ext/IPC-Open3/lib/IPC/Open3.pm


rpaskudniak
User


Oct 13, 2015, 2:39 PM

Post #6 of 11 (2981 views)
Re: [FishMonger] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

Fishmonger,
You must have the patience of ..
Anyway, with your previous responses I was able to write my utility and my immediate problem was solved. Now we, the disciples, with to really learn how to open() a pipeline that has a whole chain of pipes before the last one. Now it's more a vendetta than a need to solve a problem. Mad

According to your responses, this code should open() a chain of piped commands (i.e. with embedded pipe). Let's put back that pipe and see:


Code
my $cdr_cmd = "cdr list replicate brief|tail +4";      # Skip column headings 
my $cdr; # File handle for above cdr command
open($cdr, '-|', $cdr_cmd)
or die "Error <$!> opening command <$cdr_cmd>";

In the debugger, I again get:
DB<3> p $!
Illegal seek

Well, that didn't get us anywhere.

Actually, the first example on that perldoc page is:

Code
        open(my $fh, "<", "input.txt")  
or die "cannot open < input.txt: $!";

which is simply reading from a file, not from a command. And, as shown above, did not work in this case. Frown

I'm open to ideas. I am amazed that I have not found a similar question thus far. Page 427 of the Camel Book (dead-tree edition) discusses multi-stage output pipes with the '|-" notation. I have yet to find an example of reading from a chained i.e. multi-staged pipeline.

I'm still open to ideas. C'mon, let's see someone else repond; don't leave Fishmonger to tackle me alone! Wink

Thanks to all!
--------------------
-- Rasputin Paskudniak (In perpetual pursuit of undomesticated, semi-aquatic avians)


FishMonger
Veteran / Moderator

Oct 13, 2015, 5:25 PM

Post #7 of 11 (2972 views)
Re: [rpaskudniak] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post


Quote
You must have the patience of ..

Was that meant as a complement or an insult?

The example in the doc I directed you to was to show the 3 arg form. The exact values of those args were not the main focus.

Have you tested your piped command directly on the command line? If so, does it work as expected or does it fail in some way?

Based on your debug output, it's clear that the your code is doing more than what you've shown and it's possible if not likely it's that portion of the code that is where the problem is located.

Please post a short but complete script that demonstrates the problem so that we can better test/troubleshoot. Obviously you'll need to replace your cdr command with something more generic so that we can test the script.


rpaskudniak
User


Oct 15, 2015, 2:33 PM

Post #8 of 11 (2913 views)
Re: [FishMonger] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

It was meant as a complement but I stopped in response to an interruption. Angelic

First, here's the command line and output:

Code
$ ls -E /usr/local | tail +2| head|cut -b78-90 
AZaxxon
bin
bin-orig
BUAgent
doc
etc
etc.save
etc.tar
gcc4 -> /usr/
GDS

OK, here's a simple version with that multi-stage pipeline, including line numbers for clarity:

Code
  1 #!/usr/bin/perl -d -w 
2 # read-pipe.pl - Test I/O on read form pipeline
3 #
4 use strict;
5 use Data::Dumper;
6
7 my $ls_cmd = "ls -E /usr/local | tail +2| head|cut -b78-90";
8 my $ls; # File handle
9 #
10 open ($ls, '-|', $ls_cmd)
11 or die "Error <$!> running command <$ls_cmd>";
12
13 while (my $ls_line = <$ls>)
14 {
15 chomp($ls_line);
16 printf("Input_line: <%s>\n", $ls_line);
17 }

And finally, my debugging session, in full glory:

Code
 $ ./read-pipe.pl 

Loading DB routines from perl5db.pl version 1.28
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

main::(./read-pipe.pl:7): my $ls_cmd = "ls -E /usr/local | tail +2| head|cut -b78-90";
DB<1> b 10
DB<2> r
main::(./read-pipe.pl:10): open ($ls, '-|', $ls_cmd)
main::(./read-pipe.pl:11): or die "Error <$!> running command <$ls_cmd>";
DB<2> p $!
(nothing)
DB<3> n
main::(./read-pipe.pl:13): while (my $ls_line = <$ls>)
main::(./read-pipe.pl:14): {
DB<3> p $!
Illegal seek
DB<4> c
Input_line: <AZaxxon>
Input_line: <bin>
Input_line: <bin-orig>
Input_line: <doc>
Input_line: <etc>
Input_line: <etc.save>
Input_line: <etc.tar>
Input_line: <gcc4 -> /usr/>
Input_line: <include>
Input_line: <info>
Debugged program terminated. Use q to quit or R to restart,
use O inhibit_exit to avoid stopping after program termination,
h q, h R or h O to get additional info.
DB<4> q

What??? It ran to completion???!!! How can that be?

Now from the "Illegal seek" that I was getting before, I stopped my previous debug session, since I had previous errors on the "unopened file handle". Letting it continue this time was a slip of the finger with marvelous results! It explains why my code didn't die at the open-or-die call (mentioned in my initial post). I will want to go back to my original code, of course, and try various permutations of this.

But for now, Fishmonger, you have truly helped me over this hump. Though this ain't over for me until I have it working in my utility. I'll let y'all know.

The scholarly question remains: WHY? Why am I getting an "Illegal seek" error message when no error has occurred? And why is $_ undefined after the open() call? (Well, that latter may be vanilla ignorance on my part.)
--------------------
-- Rasputin Paskudniak (In perpetual pursuit of undomesticated, semi-aquatic avians)


Laurent_R
Veteran / Moderator

Oct 15, 2015, 3:08 PM

Post #9 of 11 (2909 views)
Re: [rpaskudniak] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

Hm, I don't understand why you get that "illegal seek" error here, but I would suggest that, when debugging, you use the x command instead of the p command for displaying variables, it usually gives you better information about the content (for example, you can see the difference between empty var ('') and undef, or see if there is any invisible trailing characters, etc.).


FishMonger
Veteran / Moderator

Oct 15, 2015, 6:12 PM

Post #10 of 11 (2906 views)
Re: [rpaskudniak] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

At the moment I'm not on a system where I can run any tests so I can't say why you received the "illegal seek" error. However, I believe the answer can be found by running the script under strace to see the details of each of the underlying system calls.


rpaskudniak
User


Oct 15, 2015, 8:05 PM

Post #11 of 11 (2901 views)
Re: [Laurent_R] Shell command with embedded pipe: Illegal seek on open() [In reply to] Can't Post

Laurent,

Thanks for piping in here. I tried the x command instead of p. Some interesting results:
Before the open - My "control"

Code
DB<2> x $! 
0 ''

OK, 0 value, no message.
Now, after the open:

Code
 DB<3> x $! 
0 'Illegal seek'

Curious, as before. Now, after the first read, which is successful:

Code
DB<5> x $! 
0 'Bad file number'

Curiouser! Crazy
And thereafter, it remains with that last value - the 0 with the alleged 'Bad file number' after every read.

I would not pursue this too deeply, owing to the ancient Perl version I am forced to work with: revision 5 version 8 subversion 7. If someone has access to the latest (5.20, I think) I suspect different results. But the program runs.

By the way, experimenter that I am, I went back to the 2-parameter call with the terminating pipe:

Code
"ls -E /usr/local | tail +2| head|cut -b78-90|" 
my $ls;
open ($ls, $ls_cmd)...

and it's working just as well as the three-parameter version without the terminating pipe. Why it failed in the first place, prompting the debugging and this thread, I dunno. (Hey, the spell-checker accepted that "dunno"! Tongue) But as they say in Texas: Oil's well that ends well. Wink
--------------------
-- Rasputin Paskudniak (In perpetual pursuit of undomesticated, semi-aquatic avians)

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives