0

I use a temporary table to hold a range of ID's so I can use them in several other queries without adding a long list of ID's to every query.

I'm building this in GO and this is new for me. Creating the temporary table works, fetching the ID's succeed and also adding those IDs to the temporary table is successful. But when I use the temporary table I get this error:

pq: relation "temp_id_table" does not exist

This is my code (EDITED: added transaction):

//create context
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()

// create database connection
psqlInfo := fmt.Sprintf("host=%s port=%s user=%s "+
    "password=%s dbname=%s sslmode=disable",
    c.Database.Host, c.Database.Port, c.Database.User, c.Database.Password, c.Database.DbName)

db, err := sql.Open("postgres", psqlInfo)

err = db.PingContext(ctx)

tx, err := db.BeginTx(ctx, &sql.TxOptions{Isolation: sql.LevelSerializable})

// create temporary table to store ids
_, err = tx.ExecContext(ctx, "CREATE TEMPORARY TABLE temp_id_table (id int)")

// fetch all articles of set 
newrows, err := db.QueryContext(ctx, "SELECT id FROM article WHERE setid = $1", SetId)

var tempid int
var ids []interface{}

for newrows.Next() {
    err := newrows.Scan(&tempid)
    ids = append(ids, tempid)
}

// adding found ids to temporary table so we can use it in other queries
var buffer bytes.Buffer
buffer.WriteString("INSERT INTO temp_id_table (id) VALUES ")
for i := 0; i < len(ids); i++ {
    if i>0 {
        buffer.WriteString(",")
    }
    buffer.WriteString("($")
    buffer.WriteString(strconv.Itoa(i+1))
    buffer.WriteString(")")
}
_, err = db.QueryContext(ctx, buffer.String(), ids...)

// fething article codes
currrows, err := db.QueryContext(ctx, "SELECT code FROM article_code WHERE id IN (SELECT id FROM temp_id_table)")

(I simplified the code and removed all error handling to make the code more readable)

When I change it to a normal table everything works fine. What do I do wrong?

EDIT 05-06-2019:

I created a simple test program to test new input from the comments below:

func main() {

var codes []interface{}

codes = append(codes, 111)
codes = append(codes, 222)
codes = append(codes, 333)

config := config.GetConfig();

// initialising variables
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()

// create database connection
log.Printf("create database connection")
db, err := connection.Create(config, ctx)
defer db.Close()
if err != nil {
    log.Fatal(err)
}

// create transaction
log.Printf("create transaction")
tx, err := db.BeginTx(ctx, &sql.TxOptions{Isolation: sql.LevelReadUncommitted})
if err != nil {
    log.Fatal(err)
}

// create temporary table to store IB codes
log.Printf("create temporary table to store codes")
_, err = tx.ExecContext(ctx, "CREATE TEMPORARY TABLE tmp_codes (code int)")
if err != nil {
    log.Fatal(err)
}

// adding found IB codes to temporary table so we can fetch the current articles
log.Printf("adding codes to temporary table so we can fetch the current articles")
_, err = tx.QueryContext(ctx, "INSERT INTO tmp_codes (code) VALUES ($1),($2),($3)", codes...)
if err != nil {
    log.Fatal(err)
}

testcodes, err := tx.QueryContext(ctx, "SELECT * FROM tmp_codes")
if err != nil {
    log.Fatal(err)
}
defer testcodes.Close()

var testcount int

for testcodes.Next() {
    testcount++
}

log.Printf(fmt.Sprintf("%d items in temporary table before commit, %d ibcodes added", testcount, len(codes)))


// close transaction
log.Printf("commit transaction")
tx.Commit()

}

DeniseMeander
  • 806
  • 2
  • 9
  • 28

1 Answers1

2

The problem is the connection pool. You're not guaranteed to use the same server connection for each query. To guarantee this, you can start a transaction with Begin or BeginTx.

The returned sql.Tx object is guaranteed to use the same connection for its lifetime.


Related:

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
  • I already thought it might be something like this so I tried transactions, but only by sending queries "BEGIN" and "COMMIT", without the use of Begin() or BeginTx(). To rule this out I implemented both but the result is the same. Still after filling the IDs to my table the next query using the temporary table is gone for some reason. – DeniseMeander May 20 '19 at 15:28
  • 1
    You'll need to show your code with the transactions, if you want help debugging it. – Jonathan Hall May 20 '19 at 20:38
  • True, I'm sorry. I updated the code in my question. – DeniseMeander May 21 '19 at 10:03
  • You're not using the transaction for the queries. You must use `tx` everywhere, not `db`. – Jonathan Hall May 21 '19 at 10:04
  • This don't fix the problem because the values are only stored in the temporary table after the commit. Before the commit - of course - the table remains empty so we cannot use it. After the commit the transaction is gone and so is the temporary table and we're back at square one. – DeniseMeander Jun 05 '19 at 12:16
  • @DeniseMeander: That doesn't match the [documented behavior](https://www.postgresql.org/docs/9.3/sql-createtable.html) of Postgres temp tables. I don't know what you're doing wrong, but I don't think your description is accurate. – Jonathan Hall Jun 05 '19 at 12:18
  • I don't think it is a PostgreSQL problem. I've used temporary tables before, but not with GO. I think maybe there is a bug in the GO dbal. Everything I do works fine if I use a regular table but when I change it into a temporary one it falls apart. – DeniseMeander Jun 05 '19 at 12:23
  • Go does nothing special. It just manages a pool of connections. There's absolutely nothing in Go that would interfere with your use of temporary tables, by deleting them, or not using them properly. – Jonathan Hall Jun 05 '19 at 12:26
  • Why are you requesting `Isolation: sql.LevelSerializable`? That may be your problem. My understanding is that this effectively freezes the database at the beginning of the transaction, so all updates will become visible atomically only at the end of the transaction. This would match your described behavior. – Jonathan Hall Jun 05 '19 at 12:30
  • Isn't that the whole idea about using a transaction? Rolling back a transaction should reset things to the state of the beginning of transaction. But, in this case I only insert things in the temporary table so this is no issue, that's why I didn't use a transaction in the first place. – DeniseMeander Jun 05 '19 at 13:40
  • If that was the only reason for a transaction, there wouldn't be different isolation levels. There are many reasons for transactions. In your case, the only reason is to ensure you're talking to the same connection--so why are you requesting the highest possible isolation level? You should request the lowest level--or probably use the default unless that doesn't work for some reason. – Jonathan Hall Jun 05 '19 at 13:43
  • Please see the new test code in my question. Still - using the lowest isolation level - I get no result querying the temporary table. Also, and expected, when I create a regular table, this is empty before the commit. When I query the regular table after the commit I do get the expected results. I'm really thankful for your help but if we can only use a temporary table in a transaction it is useless. Normally the temporary table is active during the lifetime of the database connection, only when the connection closes the table disappears. I think this is a bug in the GO dbal. – DeniseMeander Jun 05 '19 at 15:33