CGI/Perl Guide | Learning Center | Forums | Advertise | Login
Site Search: in

  Main Index MAIN
Search Posts SEARCH
Who's Online WHO'S
Log in LOG

Home: Need a Custom or Prewritten Perl Program?: I Need a Programmer for Freelance Work:
Reply to: Re: [mixxed] crawl & analyze generic web pages.
Guest Username:
E-mail address:
Post Style:      Get Markup Help


Replying to: Re: [mixxed] crawl & analyze generic web pages. by lifesaver

In Reply To
hi, i'm new to this forum so bare with me a bit :)

i'm looking for a script/app that would perform the following actions in the first stage of the project:

Start from: a database with links and priorities.

1. read the list and determine the highest link to crawl
2. crawl the page and store locally.
3. create a database of such links
4. analyze the page to determine:
- identify primary content areas (exclude header/footer, banners, features)
- type of page (blog, forum,news,etc..),
- page last modified, commented, etc.
- various page-related or site-related parameters
- other various scores (based on formulas I provide)
5. analyze the primary content
- similarity to previously known contents (trained bayesian or smth similar)
- recency of the content
- retrieve the main keywords
6. store scores&values from #4 and #5 in a database.

I'm expecting this to be a 1-2 months project.
Is there anyone here that could get this done and provide a quote ?


We can do it,if interested contact our designer live or put your requirement in our messegebord @

Search for (options) Powered by Gossamer Forum v.1.2.0

Web Applications & Managed Hosting Powered by Gossamer Threads
Visit our Mailing List Archives