1

I have a JSON file that I am trying to bulk upload to MySql. The file is around 50gb. Is there a simple method to get all of the data into MySql? I tried watching videos on youtube on how to do this, but all of the tutorials were for super simple json data that don't have nested data like this. Any help would be amazing. Here is a sample so you can see the structure of it: 

{
  "PatentData": [


{
  "patentCaseMetadata": {
    "applicationNumberText": {
      "value": "16315092",
      "electronicText": "16315092"
    },
    "filingDate": "2019-07-03",
    "applicationTypeCategory": "Utility",
    "partyBag": {
      "applicantBagOrInventorBagOrOwnerBag": [
        {
          "applicant": [
            {
              "contactOrPublicationContact": [
                {
                  "name": { "personNameOrOrganizationNameOrEntityName": [ { "organizationStandardName": { "content": [ "SEB S.A." ] } } ] },
                  "cityName": "ECULLY",
                  "geographicRegionName": {
                    "value": "",
                    "geographicRegionCategory": "STATE"
                  },
                  "countryCode": "FR"
                }
              ]
            }
          ]
        },
        {
          "inventorOrDeceasedInventor": [
            {
              "contactOrPublicationContact": [
                {
                  "name": {
                    "personNameOrOrganizationNameOrEntityName": [
                      {
                        "personStructuredName": {
                          "firstName": "Johan",
                          "middleName": "",
                          "lastName": "SABATTIER"
                        }
                      }
                    ]
                  },
                  "cityName": "Mornant",
                  "geographicRegionName": {
                    "value": "",
                    "geographicRegionCategory": "STATE"

The end goal is to have the JSON file in a MySQL database in the following format:

Name | Address | State | Country ... | Abstract
Tim  - 23 North-  TX   - US      ... | The tissue...
Tom  - 33 North-  TX   - US      ... | The engineer...
Kim  - 78 North-  TX   - US      ... | The lung...
Bob  - 123 North-  TX   - US      ... | The tissue...
Rob  - 93 North-  TX   - US      ... | The scope...
EatSleepCode
  • 452
  • 7
  • 21
  • What do you want to do with the data? Stuffing it all into one [JSON column](https://dev.mysql.com/doc/refman/8.0/en/json.html) is easy. Anything else you'll need to explain how you intend to break the JSON up and what your schema is. – Schwern Jul 30 '19 at 20:58
  • I just updated my post with that information added – EatSleepCode Jul 30 '19 at 21:07
  • You'll have to write a program to parse the JSON, iterate through the `PatentData` array, and insert the appropriate pieces into the database. Since trying to parse a 50 gig JSON file all at once will probably blow out your memory, you'll need a [streaming parser](https://stackoverflow.com/questions/444380/is-there-a-streaming-api-for-json) that will load only part of the document at a time. – Schwern Jul 30 '19 at 21:15
  • Have you tried the JSON bulk loader in the new MySQL Shell? https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-utilities-json.html – Dave Stokes Jul 31 '19 at 01:00

0 Answers0