0

I always use MySQL for storing and operating on data.

But this time I have around 4 GB csv dataset.

I have imported this into MySQL.

It was importing for about 2-3 hours.

It's a one table with about 7.500.000 rows and few columns.

Importing time was long.

Operating with MySQL queries on this dataset happens long too.

Do I really do good thing to use this with MySQL database?

Maybe I should use something like nosql database? Or serverless database?

I don't know if I am doing it proper way.

What should I do with it? How should I operate on this dataset?

Krystian Walicki
  • 51
  • 2
  • 9
  • MySQL is just fine for such task. The questions you _should_ ask are: 1. why do you actually expect a much faster processing? and 2. can you speed it up? The answer to the second question is interesting, but we cannot help with that unless you add more details to the question. – arkascha Mar 17 '19 at 11:14
  • @arkascha isn't nosql or file database better for it? than serverful sql database like MySQL? – Krystian Walicki Mar 17 '19 at 11:36
  • Well the answer obviously depends on the use case: what you want to do with the data. For _typical_ use cases file based solutions (you probably refer to things like SQLite or BerkelyDB) are _much_ slower and nosql databases are very limited in how to retrieve the data. But there are single cases where you might want to try such an approach, sure. Let's try like this: why do _you_ think that a no sql based solution is superior for your task? If you can answer that you are a step further, or are you just guessing? I suggest you enhance the question and add some information about your use case. – arkascha Mar 17 '19 at 11:58
  • And it is interesting that you chose not to answer to any of the questions in the first comment but just chose to repeat what you already wrote in the question. Why is that so? Why don't you answer? The questions were meant to help you to decide... – arkascha Mar 17 '19 at 12:00
  • @arkascha Operations made on this table will be just select. I will be just selecting. And there is one table. No relations. Only one table, few columns and many rows. I don't expect faster processing, I don't expect anything. I just want good solution for my problem and I am wondering if mysql is it. 2.I don't know if I can speed it up xD. And I don't know superior solution for this task Columns; id, url, title, duration, keywords (table that aggregate information about some videos) – Krystian Walicki Mar 17 '19 at 12:35
  • Sure, all fine, so basically you say you have no idea about anything but you feel that MySQL might not be the right tool. That comes very close to a religious statement, doesn't it? Don't get me wrong, not trying to mock you, just stating... I learned not to argue with such views. I provided what insight I could give. That definitely is a subjective view. If you think you know better then that is perfectly fine! Go and make a few tests! – arkascha Mar 17 '19 at 13:34
  • 7 million rows is [absolutely nothing for MySQL](https://stackoverflow.com/q/50897375) (and also for NoSQL-databases). [The decision](https://stackoverflow.com/q/3713313) here will mostly depend [on what you like best](https://stackoverflow.com/a/2440141). But if you need help optimizing indexes and your query, you'll need to provide details about your indexes and query. If you need help optimizing your process (e.g. you may need to import your csv daily, then speeding up your 2 hour import might be a good idea), we would need details about your process. – Solarflare Mar 17 '19 at 14:15

0 Answers0