0

This is a followup to this question:

Ruby create JSON from SQL Server

I was able to create nested arrays in JSON. But I'm struggling with looping through records and appending a file with each record. Also how would I add a root element just at the top of the json and not on each record. "aaSequences" needs to be at the top just once... I also need a comma between each record.

here is my code so far

require 'pp'
require 'tiny_tds'
require 'awesome_print'
require 'json'

class Document
    def initialize strategy
        @document = strategy

    #load helper functions
    load "helpers_ruby.rb"

    #set environment 'dev', 'qa', or 'production'
    load "envconfig_ruby.rb"
    
    end

    def StartUP
        @document.StartUP
    end

    def getseqrecord
        @document.getseqrecord
    end


end

class GetSqlaaSequence

  def StartUP
    ##system "clear"    ##linux
    system "cls"        ##Windows   

        # create connection to db

    $connReportingDB = createReportingxxSqlConn($ms_sql_host, $ms_sql_user, $ms_sql_password, $ms_sql_dbname)

    ##$currentDateTime = DateTime.now
    ##pp 'def StartUP ran at: '+$currentDateTime.to_s

  end


    def getseqrecord

  
        # get the aaaaSequences data
        @result = $connReportingDB.execute("SELECT 
        [jsonFile]
      ,[id]
      ,[title]
      ,[authorIds]
      ,[name]
      ,[aminoAcids]
      ,[schemaId]
      ,[registryId]
      ,[namingStrategy] 
      FROM tablename      
      ")
    
        $aaSequences = Array.new
        @i = 0

        @result.each do |aaSequence|

    jsonFile = aaSequence['jsonFile']
    id = aaSequence['id']
    title = aaSequence['title']
    authorIds = aaSequence['authorIds']
    name = aaSequence['name']
    aminoAcids = aaSequence['aminoAcids']
    schemaId = aaSequence['schemaId']
    registryId = aaSequence['registryId']
    namingStrategy = aaSequence['namingStrategy']
      
            ##end

            @hash = Hash[
                "jsonFile", jsonFile,
                "id", id,
                "title", title,
                "authorIds", authorIds,
                "name", name,
                "aminoAcids", aminoAcids,
                "schemaId", schemaId,
                "registryId", registryId,
                "namingStrategy", namingStrategy
                    ]
                    
            @filename = jsonFile


jsonFileOutput0 = {:"#{title}" => [{:authorIds => ["#{authorIds}"],:aminoAcids => "#{aminoAcids}",:name => "#{name}",:schemaId => "#{schemaId}",:registryId => "#{registryId}",:namingStrategy => "#{namingStrategy}"}]}

                        
jsonFileOutput = JSON.pretty_generate(jsonFileOutput0)     


File.open(jsonFile,"a") do |f|
  f.write(jsonFileOutput)

####ad the comma between records...Not sure if this is the best way to do it...
# File.open(jsonFile,"a") do |f|
  # f.write(',')  
 # end
 
end


            
            $aaSequences[@i] = @hash                    
            @i = @i + 1
            
            
        ##@createReportingSqlConn.close 
            end
        
        end     
    end

Document.new(GetSqlaaSequence.new).StartUP
  
#get aaSequences and create json files
Document.new(GetSqlaaSequence.new).getseqrecord

here is a sample of the json it creates so far...

{
  "aaSequences": [
    {
      "authorIds": [
        "fff_fdfdfdfd"
      ],
      "aminoAcids": "aminoAcids_data",
      "name": "fdfdfddf-555_1",
      "schemaId": "5555fdfd5",
      "registryId": "5fdfdfdf",
      "namingStrategy": "NEW_IDS"
    }
  ]
}{
  "aaSequences": [
    {
      "authorIds": [
        "fff_fdfdfdfd"
      ],
      "aminoAcids": "aminoAcids_data",
      "name": "fdfdfddf-555_2",
      "schemaId": "5555fdfd5",
      "registryId": "5fdfdfdf",
      "namingStrategy": "NEW_IDS"
    }
  ]
}

and here is an example of what I need it to look like

{
  "aaSequences": [
    {
     "authorIds": [
        "authorIds_data"
      ],
      "aminoAcids": "aminoAcids_data",
      "name": "name_data",
      "schemaId": "schemaId_data",
      "registryId": "registryId_data",
      "namingStrategy": "namingStrategy_data"
    },
    {
     "authorIds": [
        "authorIds_data"
      ],
      "aminoAcids": "aminoAcids_data",
      "name": "name_data",
      "schemaId": "schemaId_data",
      "registryId": "registryId_data",
      "namingStrategy": "namingStrategy_data"
    }
  ]
} 
hollecar
  • 17
  • 4
  • 1
    Have you considered using SQL Server for this? You could do this pretty simply with `FOR JSON PATH, ROOT('aaSequences')` at the end of your `SELECT` query. Sample data would help – Charlieface Mar 29 '22 at 21:07
  • I will try FOR JSON in sql again. But I couldn’t figure out how to build the format with the nested arrays. – hollecar Mar 30 '22 at 10:38
  • I feel like I’m very close with ruby. I just cant figure out how you add a root element to the data then the rest of the data. This is part of a larger integration and I wrote a lot of it in ruby – hollecar Mar 30 '22 at 10:44
  • Oh btw I provided the sample data json in the question text. – hollecar Mar 30 '22 at 10:45
  • < https://stackoverflow.com/questions/47814217/create-nested-json-arrays-using-for-json-path> I found this. I will try this – hollecar Mar 30 '22 at 10:50
  • We need sample *input* data, preferably `CREATE TABLE` and `INSERT` statements. Arrays are a bit annoying in SQL Server, because there is no `JSON_AGG` function: the usual answer is to use `STRING_AGG` instead. You probably need something like `SELECT authorIds, aminoAcids, name, schemaId, registryId, namingStrategy FROM tablename FOR JSON PATH, ROOT('aaSequences')` but it's unclear how `authorIds` fits in, can't say without seeing your input data – Charlieface Mar 30 '22 at 11:04
  • I'm using a view off a table for the query. Insert statements?? Why would you need those? I'm not inserting anything back to sql? here is the view: CREATE VIEW vw_aaSequences AS SELECT 'createBulkAASeq_' + seq_id + '.json' AS jsonFile, seq_id AS [id], 'aaSequences' AS title, 'ent_LaFp3bgw' AS authorIds, 'AAORF00' + CAST(id AS varchar) + SUBSTRING(seq_id, 6, 4) AS [name], [seq] AS aminoAcids, 'ts_3Hvv84dU' AS schemaId, 'src_abswnJB0' AS registryId, 'NEW_IDS' AS namingStrategy FROM [GeneralDB17].[dbo].[FosmidFAA_Raw0] – hollecar Mar 30 '22 at 11:59
  • This works so far... I just need to nest the authorids array... SELECT [authorIds], [aminoAcids], [name], [schemaId], [registryId], [namingStrategy] FROM [dbo].[vw_aaSequences] FOR JSON PATH, ROOT('aaSequences') – hollecar Mar 30 '22 at 12:05
  • I'm not sure why this is an array [authorIds]...this is data I am posting to a 3rd party cloud app and that is their required format. My guess is the [authorIds] is a list vs a reg field. – hollecar Mar 30 '22 at 12:07
  • We need `INSERT` statements so we can construct a https://dbfiddle.uk to replicate your scenario. Apart from the method I noted above for `authorIds` another method if you only have a single value to putin the array is `JSON_MODIFY('[]', 'append $', authorIds) AS authorIds` which will return `["someAuthorid"]` as part of your JSON. See eg https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=31edd74ad132d4f42011aee9a35f1d6f – Charlieface Mar 30 '22 at 12:11
  • here you go ................ INSERT INTO [dbo].[aaSequences]( [authorIds], [aminoAcids], [name], [schemaId], [registryId], [namingStrategy] ) VALUES( 'authorIds_data', 'aminoAcids_data', 'name_data', 'schemaId_data', 'registryId_data', 'namingStrategy_data' ); – hollecar Mar 30 '22 at 12:28
  • hey this works the json looks correct.......... – hollecar Mar 30 '22 at 12:33
  • OK so what's wrong with what I said? https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=558473790840760ef121c262e89183e1 – Charlieface Mar 30 '22 at 12:33
  • SELECT JSON_MODIFY('[]', 'append $', authorIds) AS authorIds, 'aminoAcids_data'[aminoAcids], [name], [schemaId], [registryId], [namingStrategy] FROM [dbo].[vw_aaSequences] FOR JSON PATH, ROOT('aaSequences') – hollecar Mar 30 '22 at 12:34
  • nothing is wrong – hollecar Mar 30 '22 at 12:34
  • this works SELECT JSON_MODIFY('[]', 'append $', authorIds) AS authorIds, 'aminoAcids_data'[aminoAcids], [name], [schemaId], [registryId], [namingStrategy] FROM [dbo].[vw_aaSequences] FOR JSON PATH, ROOT('aaSequences') – hollecar Mar 30 '22 at 12:35
  • this is awesome thanks – hollecar Mar 30 '22 at 12:35
  • instead of adding a comment reply as answer and I will accept as answer...the sql method is creates the json exactly as needed...Thank you soo much!! – hollecar Mar 30 '22 at 12:36

1 Answers1

0

You can just do the whole thing in SQL using FOR JSON.

Unfortunately, arrays are not possible using this method. There are anumber of hacks, but the easiest one in your situation is to just append to [] using JSON_MODIFY

SELECT
  authorIds = JSON_MODIFY('[]', 'append $', a.authorIds),
  [aminoAcids],
  [name],
  [schemaId],
  [registryId],
  [namingStrategy]
FROM aaSequences a
FOR JSON PATH, ROOT('aaSequences');

db<>fiddle

Charlieface
  • 52,284
  • 6
  • 19
  • 43
  • Any chance you would you know how I create json like this with the for JSON? { "assayResults": [ { "fields": { "entity": "dfdfdfd", "fosmid": "gfhfghfghf", "gc_content": "444", "gene_end": "444", "gene_id": "7", "gene_start": "4", "gene_type": "fsdfsf", "length": "4343", "strand": "-" }, "schemaId": "fdfdfdfdf" } ] } – hollecar Apr 01 '22 at 19:00
  • this is what I tried... SELECT top 1 JSON_MODIFY('{'+[entity]+[fosmid]+[gc_content]+[gene_end]+[gene_id]+ [gene_start]+[gene_type]+[length]+[strand]+'}', 'append $', [fields]) as [fields], [schemaId] FROM tablename FOR JSON PATH, ROOT('assayResults') – hollecar Apr 01 '22 at 19:00
  • should I add as a new question? – hollecar Apr 01 '22 at 19:01
  • oh wait i figured it out...image that...using the docs.Microsoft websites works sometimes, lol SELECT TOP 1 entity AS 'fields.entity', fosmid AS 'fields.fosmid', gc_content AS 'fields.gc_content', gene_end AS 'fields.gene_end', gene_id AS 'fields.gene_id', gene_start AS 'fields.gene_start', gene_type AS 'fields.gene_type', [length] AS 'fields.length', strand AS 'fields.strand', schemaId FROM tablename FOR JSON PATH, ROOT('assayResults'); – hollecar Apr 01 '22 at 19:21
  • You could also nest `fields` in a subquery also with `FOR JSON` – Charlieface Apr 02 '22 at 20:18