perl - SQLite Optimization for Millions of Entries? -

I'm trying to deal with a problem using a SQLite database and the Perl module. Finally, millions of entries will be required to log in. I have a text string for the only unique identifier URL for each item. I'm thinking of doing it in two ways:

Way # 1: A good table, bad table, unstructured table (I need to check HTML and decide that I want it.) We have a total of 1 billion pages, each table has 333 million URLs. I have a new URL to add to, and I have to check and see if it is in a table, and add it to unsorted, if it is unique Additionally, I would like to extend several rows with this option. Way # 2: I have 2 tables, master and good. There are all 1 billion page URLs in the master, and in 333 million well I want to. The new URL needs to be done the same thing, except for this time, I ask only one table, and I will never remove a line from the master, just add data to the best.

So basically, give me a strong SQLite database to ask that the ~ ~ 20-letter text string is unique, know best setup, then add It is not.

EDIT: I am now trying to get Berkeley DB to use the Perl module, but there is no dice. Here's what I have:

  Usage Berkeleybi; $ DbFolder = 'C: \ somedirectory'; My $ env = berkeleydi :: envi-> new (-home = & gt; $ dbFolder); My $ db = Berkeley DB :: hash-> new (-filename = & gt; "fred.db", -Env = & gt; $ env); My $ position = $ db- & gt; Db_put ("apple", "red");  

And when I run it, I get the following:

Do not call "db_put" method on an undefined value at C: \ Directory can do. Perlfile.pl Line 42, & lt; STDIN & gt; Line 1.

If $ db is undefined, open Database is failing, and you $! Why should and $ BerkeleyDB :: error be inspected.

Have you already created the database? If not, you'll see the -flags = & gt; DB_CREATE .

Job example:

  Use strict; Use warnings; Use Berkeley DB; My $ dbFolder = '/ home / ysth / bdbtmp /'; My $ db = Berkeley DB :: hash- & gt; New (-Filename = & gt; "$ dbFolder / fred.db", -Flags = & gt; DB_CREATE,) or die "could not be made: $ !, $ BerkeleyDB :: Error \ n". My $ position = $ db- & gt; Db_put ("apple", "red");  

I am unable to do anything useful to Berkeley DB :: ANV, though; Whatever I tried, the constructor returned the undef.