Convert an unsigned byte stream to signed bytestream Golang

Question

What is the best way to convert an bytestream to a bytestream slice in Golang? Currently I'm trying to replicate a java program in golang and I believe I am having some issues with the fact that java reads the bytestream as a signed value, whereas golang treats it as an unsigned value.

When I print in Java note the negative values are different. Positive are the same:

Java:

8|-8|-58|-61|-56|-113|42|16|-64|2|24|-16|1

Golang:

8|248|198|195|200|143|42|16|192|2|2|240|1

Currently, my implementation right in GO looks like:

//open the file with os.open
//create new reader and unzip it

//reads from zip file and returns an array of bytes associated with it.
data, err := ioutil.ReadAll(fz)
if err != nil {
    return nil, err
}
//reflect.TypeOf shows this is a []uint8
//I want this to be a []int8 (so signed).

In Java the implementation was pretty much:

  //create buffer reader
  //create input stream
  //create datainput stream
  //use the .Read function to get the values required.

I didn't see any easy way to typecast it quickly to a signed int (maybe I'm wrong). I could try to iterate through the entire slice, converting each value to a signed int, but that approach seems rather messy. It would also require me to operate on each. Is there a cleaner way of converting the slice?

The top answers to http://stackoverflow.com/q/12357865/4740606 and http://stackoverflow.com/q/28949063/4740606 might be helpful to you — Snowman, May 22 '16 at 15:20

score 4 · Answer 1 · answered May 22 '16 at 14:59

You can use unsafe.Pointer if you know exactly what you are doing. Because as its name suggests, it is unsafe. Therefore, it will blow up if you use it unwisely.

package main

import (
    "fmt"
    "unsafe"
)

func main() {
    unsigned := make([]uint8, 0)
    unsigned = append(unsigned, 128)
    unsigned = append(unsigned, 255)

    signed := *(*[]int8)(unsafe.Pointer(&unsigned))
    fmt.Printf("%d : %d\n", signed[0], unsigned[0])
    fmt.Printf("%d : %d\n", signed[1], unsigned[1])
}
// -128 : 128
// -1 : 255

You can find this example on the Go playground

This seems to work, though the unsafe operations scares me a little bit. Will play with this to make sure I understand this completely — andor kesselman, May 22 '16 at 15:10

icza · Accepted Answer · 2016-10-15T04:34:42.793

Some things to clear: both Java and Go reads the data the same way from files.

Files are series of binary data grouped by 8 bits, which we call a byte. This byte is 8 bits, how you interpret it is entirely up to you.

Both Go and Java will read the same 8-bit groups, but Java will store it in a Java byte type which is a signed type, and Go will store it in a Go byte type which is unsigned. But both will have the same 8 bits, the read data will be equal:

var b byte = 248
var i int8 = -8
fmt.Printf("%x %x\n", b, byte(i))

Output:

f8 f8

Should you need to interpret the read 8 bits as a signed value, simply use a type conversion:

data := []byte{8, 248, 198}
for _, v := range data {
    fmt.Println(int8(v))
}

Output (same as the Java output):

8|-8|-58|

Try the examples on the Go Playground.

Should you worry about performance (because of type conversions)? Absolutely not. byte -> int8 conversion has no runtime cost, as both have the same memory layout (that is, 8 bits = 1 byte), and the conversion just tells to interpret those 8 bits differently.

score 1 · Answer 3 · answered May 22 '16 at 16:06

1

The unsafe answer above is perfectly fine, however it wouldn't work on appengine.

Here's the safe version:

signed := make([]int8, len(unsigned))
for i, v := range unsigned {
    signed[i] = int8(v)
}

playground

answered May 22 '16 at 16:06

OneOfOne

95,033
20
184
185

Convert an unsigned byte stream to signed bytestream Golang

3 Answers3