I am a Perl programmer with some good scripts that receive HTTP pages (from a text file URL in the URL ) Curl and save them to a folder.
However, in order to get the number of pages in lakhs, sometimes the script fails to number 170,000 and I have to start the script again manually. It reads the URL automatically and sees whether a page has been downloaded and dropped. But, with a few million people, it still takes a few hours, as far as it was left. Obviously, this is not going to end in the end.
I have been told that instead of saving a text file, which is hard to find and modify, I have to use the database. I do not know much about the database, just a year ago The school is abusive with MySQL on the server. I need the ability to add millions of rows and some static columns, search / modify a quickly , and do it locally on a lane (or the same computer if it is difficult) . And of course, I need to use Pearl for this database.
Where should I start? What do I need to download to start a server on Windows? Which Pearl module should I use? (I am using an ActiveState distro)
Since you need to search only on one column Normally, you can think of these key / value databases as a Pearl hash which is operated from memory rather than disk, as if the key / value is to store the store database. The exact key look up is very fast, but everyone else needs to scan the entire dataset.
Comments
Post a Comment