How can I separate generated code package and user code but have them accessible from one place in code

Question

I am newer to golang, so I have some courses that I bought from udemy to help break me into the language. One of them I found very helpful for a general understanding as I took on a project in the language.

In the class that I took, all of the sql related functions were in the sqlc folder with the structure less broken out:

    sqlc
        generatedcode
        store

One of those files is a querier that is generated by sqlc that contains an interface with all of the methods that were generated. Here is the general idea of what it currently looks like: https://github.com/techschool/simplebank/tree/master/db/sqlc

package db

import (
    "context"

    "github.com/google/uuid"
)

type Querier interface {
    AddAccountBalance(ctx context.Context, arg AddAccountBalanceParams) (Account, error)
    CreateAccount(ctx context.Context, arg CreateAccountParams) (Account, error)
    ...
}

var _ Querier = (*Queries)(nil)

Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier? I'm also trying to have it so that the sqlc generated code is in its own folder. The structure I am aiming for is:

    sql
        sqlc
            generatedcode
        store - (wraps it all together)
        dynamicsqlfiles

This should clear up what a I mean by store: https://github.com/techschool/simplebank/blob/master/db/sqlc/store.go

package db

import (
    "context"
    "database/sql"
    "fmt"
)

// Store defines all functions to execute db queries and transactions
type Store interface {
    Querier
    TransferTx(ctx context.Context, arg TransferTxParams) (TransferTxResult, error)
}

// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
    db *sql.DB
    *Queries
}

// NewStore creates a new store
func NewStore(db *sql.DB) Store {
    return &SQLStore{
        db:      db,
        Queries: New(db),
    }
}

I'm trying to run everything through that store (both generated and my functions), so I can make a call similar to the CreateUser function in this file (server.store.): https://github.com/techschool/simplebank/blob/master/api/user.go

    arg := db.CreateUserParams{
        Username:       req.Username,
        HashedPassword: hashedPassword,
        FullName:       req.FullName,
        Email:          req.Email,
    }

    user, err := server.store.CreateUser(ctx, arg)
    if err != nil {
        if pqErr, ok := err.(*pq.Error); ok {
            switch pqErr.Code.Name() {
            case "unique_violation":
                ctx.JSON(http.StatusForbidden, errorResponse(err))
                return
            }
        }
        ctx.JSON(http.StatusInternalServerError, errorResponse(err))
        return
    }

I've tried creating something that houses another querier interface that embeds the generated one, then creating my own db.go that uses the generated DBTX interface but has its own Queries struct, and New function. It always gives me an error that the Queries struct I created aren't implementing the functions I made, despite having it implemented in one of the custom methods I made.

I deleted that branch, and have been clicking through the simplebank project linked above to see if I can find another way this could be done, or if I missed something. If it can't be done, that's okay. I'm just using this as a good opportunity to learn a little more about the language, and keep some code separated if possible.

---

UPDATE: There were only a few pieces I had to change, but I modified the store.go to look more like:

// sdb is imported, but points to the generated Querier
// Store provides all functions to execute db queries and transactions
type Store interface {
    sdb.Querier
    DynamicQuerier
}

// SQLStore provides all functions to execute SQL queries and transactions
type SQLStore struct {
    db *sql.DB
    *sdb.Queries
    *dynamicQueries
}

// NewStore creates a new Store
func NewStore(db *sql.DB) Store {
    return &SQLStore{
        db:             db,
        Queries:        sdb.New(db),
        dynamicQueries: New(db),
    }
}

Then just created a new Querier and struct for the methods I would be creating. Gave them their own New function, and tied it together in the above. Before, I was trying to figure out a way to reuse as much of the generated code as possible, which I think was the issue.

Why I wanted the Interface: I wanted a structure that separated the files I would be working in more from the files that I would never touch (generated). This is the new structure:

I like how the generated code put everything in the Querier interface, then checked that anything implementing it satisfied all of the function requirements. So I wanted to replicate that for the dynamic portion which I would be creating on my own.

It might be complicating it a bit more than it would 'NEED' to be, but it also provides an additional set of error checking that is nice to have. And in this case, even while maybe not necessary, it ended up being doable.

Answer 1

Would it be possible to wrap both what sqlc generates AND any queries that a developer creates (dynamic queries) into a single querier?

If I'm understanding your question correctly I think that you are looking for something like the below (playground):

package main

import (
    "context"
    "database/sql"
)

// Sample SQL C Code
type DBTX interface {
    ExecContext(context.Context, string, ...interface{}) (sql.Result, error)
    PrepareContext(context.Context, string) (*sql.Stmt, error)
    QueryContext(context.Context, string, ...interface{}) (*sql.Rows, error)
    QueryRowContext(context.Context, string, ...interface{}) *sql.Row
}

type Queries struct {
    db DBTX
}

func (q *Queries) DeleteAccount(ctx context.Context, id int64) error {
    // _, err := q.db.ExecContext(ctx, deleteAccount, id)
    // return err
    return nil // Pretend that this always works
}

type Querier interface {
    DeleteAccount(ctx context.Context, id int64) error
}

//
// Your custom "dynamic" queries
//

type myDynamicQueries struct {
    db DBTX
}

func (m *myDynamicQueries) GetDynamicResult(ctx context.Context) error {
    // _, err := q.db.ExecContext(ctx, deleteAccount, id)
    // return err
    return nil // Pretend that this always works
}

type myDynamicQuerier interface {
    GetDynamicResult(ctx context.Context) error
}

// Combine things

type allDatabase struct {
    *Queries  // Note: You could embed this directly into myDynamicQueries instead of having a seperate struct if that is your preference
    *myDynamicQueries
}

type DatabaseFunctions interface {
    Querier
    myDynamicQuerier
}

func main() {
        // Basic example
    var db DatabaseFunctions
    db = getDatabase()
    db.DeleteAccount(context.Background(), 0)
    db.GetDynamicResult(context.Background())

}

// getDatabase - Perform whatever is needed to connect to database...
func getDatabase() allDatabase {
    sqlc := &Queries{db: nil}           // In reality you would use New() to do this!
    myDyn := &myDynamicQueries{db: nil} // Again it's often cleaner to use a function
    return allDatabase{Queries: sqlc, myDynamicQueries: myDyn}
}

The above is all in one file for simplicity but could easily pull from multiple packages e.g.

type allDatabase struct {
    *generatedcode.Queries
    *store.myDynamicQueries
}

If this does not answer your question then please show one of your failed attempts (so we can see where you are going wrong).

One general comment - do you really need the interface? A common recommendation is "Accept interfaces, return structs". While this may not always apply I suspect you may be introducing interfaces where they are not really necessary and this may add unnecessary complexity.

I thought that the Store, which was housing both Queriers, was tying it all together. Can you explain a little with the example above (in the question post) why it's not necessary? How does SQLStore get access to all of the Querier interface functions?

The struct SQLStore is what is "tying it all together". As per the Go spec:

Given a struct type S and a named type T, promoted methods are included in the method set of the struct as follows:

• If S contains an embedded field T, the method sets of S and *S both include promoted methods with receiver T. The method set of *S also includes promoted methods with receiver *T.
• If S contains an embedded field *T, the method sets of S and *S both include promoted methods with receiver T or *T.

So an object of type SQLStore:

type SQLStore struct {
    db *sql.DB
    *sdb.Queries
    *dynamicQueries
}

var foo SQLStore // Assume that we are actually providing values for all fields

Will implement all of the methods of sdb.Queries and, also, those in dynamicQueries (you can also access the sql.DB members via foo.db.XXX). This means that you can call foo.AddAccountBalance() and foo.MyGenericQuery() (assuming that is in dynamicQueries!) etc.

The spec says "In its most basic form an interface specifies a (possibly empty) list of methods". So you can think of an interface as a list of functions that must be implemented by whatever implementation (e.g. struct) you assign to the interface (the interface itself does not implement anything directly).

This example might help you understand.

Hopefully that helps a little (as I'm not sure which aspect you don't understand I'm not really sure what to focus on).