CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Another CRON PERL issue

 



rdunaway62
New User

Mar 1, 2015, 6:06 PM

Post #1 of 8 (3979 views)
Another CRON PERL issue Can't Post

Hi All,
New to the forum but stuck on this issue.

I have a script I have run for several years. I moved my OS to VM Ware. This script has had no problems up until I changed over to Arch. Again, on VM Ware.

The script runs perfectly user to user or at the cli. I can even run it with AT and it will run perfect. When I run it in cron, the script is initiated. Parts of the script run fine, I see this in a log that it writes to, when I get to a particular part of the script the data collected and manipulated is not placed in the files needed.

Again, I can run it at the cli or through at without issue. Other users can run this .pl and it works fine.

I have listed the code below that has the issue along with some text to explain. Also I have listed the cron format to show how it is initiated by through cron.


# Description and Notes:
# Greps the first line of data on a file based by the time: 12:00AM.
# Places the data in an array, creates a new $ input with the captured
# data needed.
# Opens and places data in a file then closes file.
# Next open a log file, makes an entry to the log stating the .pl script
# has been executed by CRON, closed log the log file. End of Script.
################################################
#!/bin/perl
use strict;
use warnings;

my $daily = `grep "12:00am" /home/username/Documents/webfiles/html/file.txt`;
my @input = split (' ',$daily);
my $data = $input[25];
open (TESTDUMP, "> /home/username/bin/scripts.pl/tempdump");
print TESTDUMP $data;
close (TESTDUMP);

## Log file entry ##
my $log1 = "Scp - "; # Starts the sting "log1" for log file entry.
my $date1 = `date "+%m/%d/%Y %l:%M:%S %p - Yesterdays PDA Script"`; # Starts the string "date1" to add the date.
$date1 =~ s/^0+//; #removes the excess 0's.
$date1 =~ s/ / /; #removes the double spacing between date and time.

open (PDA, ">> /home/username/Documents/webfiles/logs/PDA.log");
print PDA "$log1$date1"; #prints and concantanates $log1 and $date1 to the log file.
close (PDA);



### crontab input ####
# This was setup to run every 5 minutes as a test. This normally run @
# midnight.
#############################

*/5 * * * * /usr/bin/perl /home/weather/bin/scripts.pl/ystrdy_wndrn.pl > /dev/null 2>&1

Further note:
The script is executed by cron. The log entry portion is noted in the log file. The data portion is not posted
to the tempdump file. This is a portion of the .pl script. The script repeats the same data portion over
several times with the same results as what is shown.

Again, this script will run from the cli, at and different users without issue. It has run on other systems for
several years, no problem.

Thank you for any help you can provide.


Laurent_R
Veteran / Moderator

Mar 2, 2015, 12:13 AM

Post #2 of 8 (3972 views)
Re: [rdunaway62] Another CRON PERL issue [In reply to] Can't Post

I would think that this is not a Perl problem, but a problem with the cron "user". I have seen that many times that programs run under cron don't work because some of the environment variables are not set properly (or not as expected). But not having access to your system, it is pretty difficult, well, actually almost impossible, to figure out what exactly is missing.

Have a look at your shell configuration scripts, and see in you can source them in your cron entry or in the script launching your Perl program (if any).


rdunaway62
New User

Mar 2, 2015, 10:46 AM

Post #3 of 8 (3965 views)
Re: [Laurent_R] Another CRON PERL issue [In reply to] Can't Post

Hi Laurent_R, Thank you for your reply.

I keep thinking the same thing. Unfortunately I don't have the system I had been running these scripts from before. But I will say that the CRON format was taken from the previous system and is basically the same with the exception of the path to PERL.

The thing that gets me scratching my head is... The Logging portion of the script is working. I've been working on this off and on for a few months now and decided that I would try to get down to the bottom of this.

Other perl scripts work perfectly and are a lot more complicated. When I compare the syntax across a few of them, they are basically the same. This is what is confusing.

Sometime back, I did ran across an issue with the permissions on other files. Again this being a VM Ware session hosted on a windows platform, the script directories are shared/mounted from host to guest. This was my first thought, running into this situation in the past.

I will try to run this script from a different user in cron to see what the out come is.

As you said, It looks as if this may be a system env. issue rather then a perl/cron issue. I'll keep at it.

Again, Thank you for your help.


FishMonger
Veteran / Moderator

Mar 3, 2015, 6:59 AM

Post #4 of 8 (3936 views)
Re: [rdunaway62] Another CRON PERL issue [In reply to] Can't Post

If a script works correctly when manually ran from the command line but fails when ran as a cron job, it's ALWAYS due to the difference in the environment.

You can start by using full paths when executing system commands.

Personally, I'd get rid of most if not all shell commands. Perl has functions that do the same thing without being tied down to the system, which will make the script platform independent.


Code
$date1 =~ s/^0+//; #removes the excess 0's. 
$date1 =~ s/ / /; #removes the double spacing between date and time.

Those regex's aren't doing exactly what the comments states. The first one only strips the leading 0 from the month and that's all. Why strip that leading 0 but not any of the others?

The second one doesn't do anything useful. It removes the first space it finds and replaces it with a space. That's a non-op and useless.

BTW, your date string still retains the \n char which you don't want and could cause problems when using it in an open call.

Here's a better more perl'ish method to get your datastamp.

Code
use POSIX qw(strftime); 

my $date2 = strftime("%m/%d/%Y %H:%M:%S %p - Yesterdays PDA Script", localtime);


Get rid of the bareword filehandles and instead use lexical vars for the filehandles. Also, use the 3 arg form of open and most importantly you should ALWAYS check the return code on your open calls and take proper action if they fail.

Code
my $PDA_file = '/home/username/Documents/webfiles/logs/PDA.log'; 
open(my $log_fh, '>>', $PDA_file) or die "Failed to open '$PDA_file' <$!>";


A much more robust method for logging would be to use Log::Log4perl.
http://search.cpan.org/~mschilli/Log-Log4perl-1.46/lib/Log/Log4perl.pm


rdunaway62
New User

Mar 5, 2015, 7:05 PM

Post #5 of 8 (3887 views)
Re: [FishMonger] Another CRON PERL issue [In reply to] Can't Post

Hi FishMonger,

Thank you for your reply. Sorry for the long wait in response to.

Okay, these scripts, as said earlier, have run for a very long, long time with out issues. I wrote these when I was taking a PERL class in school, some long time ago. Believe it or not, I was also told by the Instructor to correct the "date entry" similar to what you had suggested. I just didn't. I didn't make the corrections on these scripts due to the "RULE" of "If It's Isn't broken, Don't Fix It!" attitude. Other scripts I have written since have the date entries in a more PERLish syntax. Now that I am trying to get it working again, I'll make the changes to be more PERLish. I did and do a lot of mixing and matching of script languages. I find it easy to do this in PERL. That's what I consider to be the strength of PERL. So pardon the perl/sh scripting here. Again, I'll make the corrections.

The striping of the leading 0 is correct on the variable. I'll make the notation to match the syntax function of this line. This is done to match other entries to the log file made from other systems that run Windows software. I have little or no control of how the date is formatted with these software's. So I formatted the entries to match the entries of the other platforms.

You are correct on the spacing in the variable $date1. It was supposed to be like this:
$date1 =~ s/ / /; #removes the double spacing between date and time.
My previous forum entry was hand typed, Sorry for the mistake. I should have reviewed the syntax a little better. Thank you for pointing this out.

Below is the script edited.

*****************Test.pl*****************************
#################################
#!/bin/perl
use strict;
use warnings;
use POSIX qw(strftime);

my $log1 = "Scp - "; # Starts the sting "log1" for log file entry.
my $datestring = strftime "%m/%e/%Y %l:%M:%S %p - Test Script from User \n", localtime;
$datestring =~ s/^0+//; #removes the excess 0's.
$datestring =~ s/\/ /\//; #removes the spacing in the day of the date.
$datestring =~ s/ / /; #removes the double spacing between the year and the hour.

#### Captures the data at the first line below the header by the time 12:00am.
my $daily = `grep "12:00am" /home/user/Documents/webfiles/daily.txt`;
my @input = split (' ',$daily);
my $col24 = $input[24];

#### Modified as recommended. Dump contents of variable when running by cron or manual.
my $PDA_file = '/home/user/bin/scripts.pl/tempdump';
open(my $log_fh, '>>', $PDA_file) or die "Failed to open '$PDA_file' <$!>";
print $log_fh "$col24\n";
print $log_fh "$log1$datestring";


#### Static Placement of html and data
my $line_a = ' <td align="left" bgcolor="#FFFFFF" width="170"><font size="-2" font face="Arial"><ystr_wr>';
my $line_b = '</ystr_wr> ^uni125^</font></td>';

my $head = `head -46 /home/user/Documents/webfiles/LogFile.htx`;
my $tail = `tail -304 /home/user/Documents/webfiles/LogFile.htx`;


#### Output to mail to review the running of script by cron.
my $to = 'user@UbeLinux';
my $from = 'user@UbeLinux';
my $subject = "$log1$datestring";
my $message = "$line_a$col24$line_b";

open(MAIL, "|/bin/sendmail -t");

# Email Header
print MAIL "To: $to\n";
print MAIL "From: $from\n";
#print MAIL "Bcc: $bcc\n";
print MAIL "Subject: $subject\n\n";
# Email Body
print MAIL $message;

**********************************************

So with the changes made I still get the error from CRON not annotating the $col24 to the dump file. It was suggested that my CRON env might be the issue. I've spent a bit of time to reading about setting up a CRON env. This still didn't seem to help or even made things worse.

I did send the output of data to be mailed to me so that I can verify it's output. I also redirected messages from CRON to and output file to capture the errors if noted. This is what I get when running from CRON:

Use of uninitialized value $col24 in concatenation (.) or string at /home/User/bin/scripts.pl/test.pl line 21.
Use of uninitialized value $col24 in concatenation (.) or string at /home/User/bin/scripts.pl/test.pl line 37.
**** EoF ****

Again, if I run this manually as any user the script run fine, as expected. I setup another VM Session with a different version of Linux that I had used to run some time back. I checked out the CRON env by outputting to a file and the ENV was exactly the same as my current. I ran the script setting it up in that CRON with the same error results as above. Again, I can run it as the user without issue. So something has changed in this script.

*************** CRON Entry ***************
SHELL=/bin/sh
USER=User
HOME=/home/User
MAILTO="User@UbeLinux"
PATH=/home/User/bin:/home/User/bin/scripts.pl:/home/User/bin/scripts.sh::/home/User/bin/scripts.py:/home/User/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local:/usr/local/bin:/usr/bin/vendor_perl:.

# ---------- --------------- Format --------------- ---------- #
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 7) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,
# | | | | |
# * * * * * command to be executed
#
#
# */5 * * * * command to be executed every 5 minutes.
# 12 10-16/2 * * root backup.sh is the same as: * 12 10,12,14,16 * * root backup.sh
################################################################################
#
# ---------- ---------- Default is Empty ---------- ---------- #

# ---------- ---------- Every 06 Minutes ---------- ---------- #

# ---------- ---------- Every 05 Minutes ---------- ---------- #
#*/5 * * * * /bin/perl /home/User/bin/scripts.pl/test.pl >> /home/User/bin/scripts.pl/tempdata 2>&1

# ---------- ---------- Every 30 Minutes ---------- ---------- #



Again, sorry for the late reply and the long post.

tThank you for your help.
--
rdunaway62


Laurent_R
Veteran / Moderator

Mar 6, 2015, 1:51 AM

Post #6 of 8 (3876 views)
Re: [rdunaway62] Another CRON PERL issue [In reply to] Can't Post

I don't have time to look at your code right now, but please note that it is not Perlish to write PERL. The proper name of the language is Perl, with an initial capital P and the other letters in lower case.

And the name of the interpreter and compiler is perl, all lower case.


FishMonger
Veteran / Moderator

Mar 6, 2015, 6:24 AM

Post #7 of 8 (3874 views)
Re: [rdunaway62] Another CRON PERL issue [In reply to] Can't Post

Since $col24 is uninitialized in the print statements, then that means that @input (more specifically $input[24]) doesn't contain what you expect. It also means that the value of $daily isn't what you expect.

Use the Data::Dumper module to dump $daily and @input to verify what they hold and if it's not sensitive data, please post that output.

Also, when posting code or its output, it's important that you post it within code tags so that the formatting/indentation is retained. Doing so makes it much easier for use to read.


rdunaway62
New User

Mar 10, 2015, 11:19 AM

Post #8 of 8 (3813 views)
Re: [FishMonger] Another CRON PERL issue [In reply to] Can't Post

Thank you FishMonger and Laurent_R,

Here's an update,

That is correct. I used the Data::Dumper as suggested. I also used the "use diagnostics". Both let me see a bit more of what is happening with this script while also showing me corrections that were much needed. I had to open the books again.

Through cron, I still get the dropping of file content from the daily.txt file but not at the user cli. As Laurent_R suggested, there is an issue with the cron environment.

I have verified the simple things in my cron env. vs user env., it looks okay. I don't see anything unusual with either env. Still the script is not grabbing the data from the daily.txt file.

There is data in the $daily. It's just not the data expected??? The element I am asking for is far beyond the elements being loaded at the split. I believe (as suggested) this to be why I get the "uninitialized" error message I receive. I haven't tried to select/request one of those elements being loaded yet or have it sent to the Data::Dumper. I suspect if I did, it would probably work and this error would go away. But that's not the data I'm trying to capture.

So I'm looking into why the daily.txt file is not being accessed through cron properly. I tried to dump the file content to another file, through cron, and it didn't work. It passed no data. Even with hard paths set in the cron statement, it still didn't work or pass the content to the new file. This is a mounted location off of another system using samba. This so far is the only issue I found with this mount. So I don't expect it to be an issue with the mount. Besides, other scripts seem to work perfectly (sh,pl and py) with other files in the same directory in this mount.

So the problems are/were 2 issues. Pour scripting, (taken care of hopefully), which we knew and what I believe is a cron env. issue as suggested before. So I need to get the cron issue taken care of first. I think that may clear up the script issue if I get this taken care of.

I started looking back at my notes for when I setup this VM session. I assumed all the scripts were working fine since they worked at the cli. Doh! Never assume.

Btw, this has lead to a re-write and cleanup of this script (another version). And I didn't realize just how much I missed doing this.

So until I get the issue with accessing this file through cron figured out....

Again, thank you both for your help.
--
rdunaway62

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives