0

I have some files which I import from CSV to SQLite everyday. There are about 30,000 - 60,000 rows per day which are added to my table which makes this table pretty large.

There is some other data which I normally keep separated and vlookup, but it would be way more efficient to just import them into my database and use Joins. This table would be smaller (maybe 500 - 1000 records per day).

Does it impact performance to have a ton of tables within a single database file? Is the size of the table more important that the total database size? I could technically add a few other files too that could be linked together but I never did this before because I was using Access and Excel previously which just couldn't handle large files.

trench
  • 5,075
  • 12
  • 50
  • 80
  • This type of open ended question is more suited on http://programmers.stackexchange.com/ . The community here focuses more on questions that have a specific answer. – kurast Aug 28 '14 at 15:47
  • I think the question is specific enough. Lance is asking if the number of tables has a bigger effect on performance than table size. This is a legitimate question in my opinion. – ApplePie Aug 28 '14 at 16:23
  • Databases can happily handle millions/billions or rows and provided they are properly structured and indexed will continue to perform very efficiently. This answer will give you an idea of just how big sqllite tables could get theoretically: http://stackoverflow.com/questions/1546947/maximum-number-of-rows-in-a-sqlite-table. It is impossible to give a clearer answer, as you question is a bit vague as it stands. – John Powell Aug 28 '14 at 16:26
  • Thanks a bunch. So I should not be afraid to add my other data to the same database (and then I can query them together) in different tables rather than maintaining separate databases. – trench Aug 28 '14 at 17:53

1 Answers1

1

Databases are designed to contain multiple tables. Having all tables in the same database makes joins much easier than doing lookups manually.

Performance is affected most by the amount of data that must be accessed, regardless of where it is stored.

SQLite's database size limit is about 140 terabytes.

CL.
  • 173,858
  • 17
  • 217
  • 259