CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
INDEX
Search Posts SEARCH
POSTS
Who's Online WHO'S
ONLINE
Log in LOG
IN

Home: Perl Programming Help: Intermediate:
Perl script will not sort files from August to Dec 2007 for some reason

 



jonathan184
New User

Jan 26, 2008, 8:55 PM

Post #1 of 7 (693 views)
Perl script will not sort files from August to Dec 2007 for some reason Can't Post

Hi I have a perl script, basically what it is suppose to do is check a folder with files. Now the files are checked using a timestamp with the command ls -l so the timestamp in this format is checked. Now what the script does is it checks the time stamp and creates a year folder if it does not exist and then creates a month folder if it does not exist and puts the respective files in the month folders. If the files are created this month then it leaves it in the folder it checks and moves others that are older to year, month folders.

Now I have tested the script and it works great . The script is doing what it suppose to but for some reason it is not processing files with timestamps from Aug 2007 to Dec 2007
It creates from Jan to July 2007 and all the other years and months but for 2007, for some reason these files disappear. Now I created an exception if it does nto go in any of the folders b put in exception and this does not happen.

Could somebody help me out in troublleshooting this or is this a bug. I am not sure where to check.


This also uses a sort_config.txt file
the file has info like this and you could make your own file if you want

So basically in the text file but info in a line like this
sourcepath destinationpath pathtologfile
e.g
/home/unix/sourcewherefilesare /home/destinationmonthsandyears /home/log20.txt

Make sure and separate the paths by a space and if you want to add comments
put # at the start of the line.

Here is the code
Code: ( text )
  1. #!/usr/bin/perl
  2. #Config File: sort_config.txt
  3. #Purpose: To sort files by create date and to archive them in respecitive
  4. #folders by years and months.
  5. #Date: 01-17-2008
  6. use File::Copy;
  7. ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isd st) = localtime();
  8. $logdateTime2 = sprintf("%02d-%02d-%04d", $mon + 1, $mday, $year + 1900, $hour, $min, $sec);
  9. # Get Current Date Time
  10. @datepcs = split(/\s+/, scalar localtime);
  11. $dateTime = "@datepcs[1]";
  12. #open config file
  13. open(HAN, "sort_config.txt") || die " Unable to run date command $!";
  14. while ($configline = <HAN>)
  15. {
  16. $_ = $configline;
  17. # This next if statement is to remove lines from the array that start
  18. # with #
  19. chomp($configline);
  20. # s/#.*//; # ignore comments by erasing them
  21. # next if /^(\s)*$/; # skip blank lines
  22. next if ($_ =~ /^(#|$)/);
  23. # Putting config file elements in an array
  24. push @array1, [ split ];
  25. }
  26. close HAN;
  27. $i = 0;
  28. foreach (@array1)
  29. {
  30. # Processing configuration file
  31. # Record Start Time for the Log
  32. ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isd st) = localtime();
  33. $logdateTime = sprintf("%02d-%02d-%04d %02d:%02d:%02d", $mon + 1, $mday, $year + 1900, $hour, $min, $sec);
  34. $sourcedir1 = $array1[$i][0];
  35. $destdir1 = $array1[$i][1];
  36. $log = $array1[$i][2];
  37. chomp($sourcedir1);
  38. chomp($destdir1);
  39. chomp($log);
  40. $line1= 'Start Date/Time | Files Processed | Duration';
  41. #IF log file does not exist create the log with the title header.
  42. if (-e "$log") {}
  43. else
  44. {
  45. open(NEWLOG, "> $log") || die ("Could not NEWLOG open file $!");
  46. print NEWLOG "$line1\n";
  47. close (NEWLOG);
  48. };
  49. ###print Header on LOG
  50. open(INFILE2,"< $log") || die("Could not open INFILE2 File!");
  51. $header=<INFILE2>;
  52. chomp($header);
  53. if($header eq "$line1") {}
  54. else
  55. {
  56. open(OUTFILE2, "> $log") || die ("Could not OUTFILE2 open file $!");
  57. print OUTFILE2 "$line1\n";
  58. print OUTFILE2 "$header\n";
  59. while (<INFILE2>)
  60. {
  61. print OUTFILE2 "$_";
  62. }
  63. close OUTFILE2;
  64. }
  65. close INFILE2;
  66. ### Adding log info
  67. open (LOG2, ">>$log") || die "error recording log $!";
  68. &sort("$sourcedir1", "$destdir1");
  69. #Converting time and date formats from the subroutine below
  70. my ($y, $m, $d, $hh, $mm, $ss) = (localtime)[5,4,3,2,1,0]; $y += 1900; $m++;
  71. my $iso_now = sprintf("%d-%02d-%02d %02d:%02d:%02d", $y, $m, $d, $hh, $mm, $ss);
  72. #Calculating time difference
  73. my $timeDiffStr = &timeDiff( date1 => $logdateTime, date2 => $iso_now );
  74. #Printing date time and Time difference and files processed to the log
  75. print LOG2 "$logdateTime | $rc | $timeDiffStr ", "\n";
  76. close (LOG2);
  77. ++$i;
  78. } # Close array1 loop
  79. # Start Subroutine to create folders and files in them.
  80. sub sort
  81. {
  82. #Run the date command to parse to get the year from the system.
  83. $yearcmd = "date |";
  84. open (ZING, "$yearcmd") || die " Unable to run date command $!";
  85. $_ = <ZING>;
  86. $_ = split;
  87. $system_year = $_[5]; # Year showing up on the UNIX system
  88. chomp($system_year);
  89. $system_month = $_[1]; # Month showing up on the UNIX system
  90. chomp($system_month);
  91. chdir("$sourcedir1");
  92. $test = "ls -l |";
  93. open (INFO, "$test") || die " Unable to run list command $!";
  94. @rows = <INFO>;
  95. $rc = -1; # So result will not include the first line in the ls command
  96. foreach (@rows)
  97. {
  98. $_ = split; # Split the line output in elements
  99. $year = "$_[7]"; # element to show the year
  100. chomp($year);
  101. $month = "$_[5]"; # Shows element of month
  102. chomp($month);
  103. #print "This is the time of the file $year \n and this is the year on the
  104. #system **** $system_year ** \n";
  105. if ($year =~ m/\:/)
  106. {
  107. #Checking for files for this month
  108. if ($year =~ m/\:/ && ($month eq $dateTime))
  109. {}
  110. elsif (-d "$destdir1/$system_year") #Check for files for this year only
  111. {
  112. if (-d "$destdir1/$system_year/$system_month")
  113. {
  114. move("$_[8]", "$destdir1/$system_year/$system_month");
  115. }
  116. else
  117. {
  118. # Create new dir if it does not exist
  119. mkdir("$destdir1/$system_year/$system_month") or die "Can't open $srcdir: $!";
  120. move("$_[8]", "$destdir1/$system_year/$system_month");
  121. }
  122. }
  123. else {
  124. #Create new dir if it does not exist
  125. mkdir("$destdir1/$system_year") or die "Can't open $srcdir: $!";
  126. mkdir("$destdir1/$system_year/$system_month") or die "Can't open $srcdir: $!";
  127. move("$_[8]", "$destdir1/$system_year/$system_month");
  128. }
  129. }
  130. else #else 1
  131. {
  132. if (-d "$destdir1/$year")
  133. {
  134. if (-d "$destdir1/$year/$month")
  135. {
  136. move("$_[8]", "$destdir1/$year/$month");
  137. }
  138. else
  139. {
  140. mkdir("$destdir1/$year/$month") or die "Can't create sort dir $!";
  141. move("$_[8]", "$destdir1/$year/$month");
  142. }
  143. }
  144. else
  145. {
  146. # Create new dir if it does not exist
  147. mkdir("$destdir1/$year") or die "Can't create sort dir $!";
  148. mkdir("$destdir1/$year/$month") or die "Can't create sort dir $!";
  149. # Check if directory was created now
  150. if (-d "$destdir1/$year/$month")
  151. {
  152. move("$_[8]", "$destdir1/$year/$month");
  153. }
  154. else
  155. {
  156. mkdir("$destdir1/exception") or die "Can't create sort dir $!";
  157. move("$_[8]", "$destdir1/exception");
  158. }
  159. }
  160. } #close else 1
  161. ++$rc;
  162. } # close loop
  163. close(ZING);
  164. close(INFO);
  165. } # close subroutine
  166. # To calculate the time difference
  167. sub timeDiff (%)
  168. {
  169. my %args = @_;
  170. my @offset_days = qw(0 31 59 90 120 151 181 212 243 273 304 334);
  171. my $year1 = substr($args{'date1'}, 0, 4);
  172. my $month1 = substr($args{'date1'}, 5, 2);
  173. my $day1 = substr($args{'date1'}, 8, 2);
  174. my $hh1 = substr($args{'date1'},11, 2) || 0;
  175. my $mm1 = substr($args{'date1'},14, 2) || 0;
  176. my $ss1 = substr($args{'date1'},17, 2) if (length($args{'date1'}) > 16);
  177. $ss1 ||= 0;
  178. my $year2 = substr($args{'date2'}, 0, 4);
  179. my $month2 = substr($args{'date2'}, 5, 2);
  180. my $day2 = substr($args{'date2'}, 8, 2);
  181. my $hh2 = substr($args{'date2'},11, 2) || 0;
  182. my $mm2 = substr($args{'date2'},14, 2) || 0;
  183. my $ss2 = substr($args{'date2'},17, 2) if (length($args{'date2'}) > 16);
  184. $ss2 ||= 0;
  185. my $total_days1 = $offset_days[$month1 - 1] + $day1 + 365 * $year1;
  186. my $total_days2 = $offset_days[$month2 - 1] + $day2 + 365 * $year2;
  187. my $days_diff = $total_days2 - $total_days1;
  188. my $seconds1 = $total_days1 * 86400 + $hh1 * 3600 + $mm1 * 60 + $ss1;
  189. my $seconds2 = $total_days2 * 86400 + $hh2 * 3600 + $mm2 * 60 + $ss2;
  190. my $ssDiff = $seconds2 - $seconds1;
  191. my $dd = int($ssDiff / 86400);
  192. my $hh = int($ssDiff / 3600) - $dd * 24;
  193. my $mm = int($ssDiff / 60) - $dd * 1440 - $hh * 60;
  194. my $ss = int($ssDiff / 1) - $dd * 86400 - $hh * 3600 - $mm * 60;
  195. #"$dd Tage $hh Std. $mm Min.";
  196. "$hh Hours $mm Mins $ss secs";
  197. }




KevinR
Veteran


Jan 26, 2008, 10:34 PM

Post #2 of 7 (691 views)
Re: [jonathan184] Perl script will not sort files from August to Dec 2007 for some reason [In reply to] Can't Post

Whats the data in "sort_config.txt" look like? Post some real lines from the file and explain them if need be.
-------------------------------------------------


jonathan184
New User

Jan 27, 2008, 7:00 AM

Post #3 of 7 (688 views)
Re: [KevinR] Perl script will not sort files from August to Dec 2007 for some reason [In reply to] Can't Post

Hi the config file has the line there but here it is



This also uses a sort_config.txt file
the file has info like this and you could make your own file if you want

So basically in the text file but info in a line like this
sourcepath destinationpath pathtologfile
e.g
/home/unix/sourcewherefilesare /home/destinationmonthsandyears /home/log20.txt

Make sure and separate the paths by a space and if you want to add comments
put # at the start of the line.


So basically the source has the files i want to sort which will have different years and months in the timestamps the destination dir is where when we process the files the years dirs and month dirs will be created and the files processed will be put in there. the 3rd section of the line is where a log file is created if it does not exist and will tell us how many files were processed and how long.

the script was designed to process on unix, so if you have any more questions or need for m to explain something else let me know. I appreciate your help.


Oh now lets say some files have this months timestamp it will stay in the archive folder and everything else that does not have this month timestamp are processed and moved to the destination dir


Thank you


(This post was edited by jonathan184 on Jan 27, 2008, 7:01 AM)


KevinR
Veteran


Jan 27, 2008, 11:46 AM

Post #4 of 7 (684 views)
Re: [jonathan184] Perl script will not sort files from August to Dec 2007 for some reason [In reply to] Can't Post

Please post some of the real lines from the file, twenty lines if there is that many would be a good number to start with. I know you posted what the format of the lines are, but I would prefer to see some of the real lines.
-------------------------------------------------


jonathan184
New User

Jan 29, 2008, 12:38 PM

Post #5 of 7 (671 views)
Re: [jonathan184] Perl script will not sort files from August to Dec 2007 for some reason [In reply to] Can't Post

Hi Kevin here are the real lines of the config file

# Testing comment dfsdfgdshgfhkjgkhjhj

/home/tibco/tibco/Susan/source3 /home/tibco/tibco/Susan/archive /home/tibco/tibco/newlog19.txt
#this is a test
/home/tibco/tibco/Susan/source4 /home/tibco/tibco/Susan/archive2 /home/tibco/tibco/newlog20.txt
#testline 2


KevinR
Veteran


Jan 31, 2008, 9:22 AM

Post #6 of 7 (659 views)
Re: [jonathan184] Perl script will not sort files from August to Dec 2007 for some reason [In reply to] Can't Post

jonathan184,

I am not able to determine what the problem is.
-------------------------------------------------


jonathan184
New User

Feb 1, 2008, 9:45 PM

Post #7 of 7 (654 views)
Re: [jonathan184] Perl script will not sort files from August to Dec 2007 for some reason [In reply to] Can't Post

Hi Kevin , nevermind i figured out the problem, the perl script is fine its the unix that was the problem basically if there was time showing where the year was suppose to be, i considered it to be the same month we are in, but i was wrong the unix shows the time stamp and day month time for the last 6 months and before those six months its going to be dd month year

i was assuming it would of show up with the yeat from jul to aug 2007 but was wrong.

 
 


Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives