How to convert a long string to a short one, and it is reversible operation

Question

I have a long string from Base64, but it is too long, how can I covert it to a short one. And I want it is a reversible operation, as I want to get the long one from the short one. By the way, I do not want to save these two strings to database.

Answer 1

Converting arbitrary long strings to arbitrary short strings is not possible in general because of the Pigeonhole principle:

• reversibility implies that the conversion codomain (small strings) needs to be have at least as many elements as the conversion domain (large strings),
• there exist vastly more distinct long strings than distinct short strings.

As an analogy with integers, consider that it is not possible to convert a 3-digit number (900 possibilities) into a 2-digit number (90 possibilities) and still be reversible, as 90 < 900.

However, if the long string is predictable and has a lot of redundancy, you may try to apply a lossless compression algorithm like gzip, in order to shorten it in the common case.

Answer 2

Base64 encoding and decoding are indeed reversible operations. The decoded string is shorter, so it may be cheaper to transfer and store in some cases (not always, YMMV).

The base64 package in the standard library has everything you need to easily decode the "long" Base64 encoded string into the original decoded string, roughly 25% shorter:

decoded, err := base64.StdEncoding.DecodeString(encoded)
if err != nil {
    fmt.Println("error:", err)
}

You can try this in the Playground:

• decoding only
• encoding then decoding

Note that there exists several variants of the Base64 format, so you need to know exactly which one was used to encode, and use the same to decode. base64.StdEncoding is the variant RFC 4648.

Answer 3

The next code in GoLang worked for me:

package main

import (
"bytes"
"compress/gzip"
"encoding/json"
"io/ioutil"
"net/http"

"github.com/gin-gonic/gin"
)

type User struct {
Name  string `json:"name"`
Email string `json:"email"`
}

func main() {
r := gin.Default()

// Endpoint to compress JSON data using GZIP
r.POST("/compress", func(c *gin.Context) {
    var user User
    if err := c.BindJSON(&user); err != nil {
        c.AbortWithStatusJSON(http.StatusBadRequest, gin.H{"error": "invalid json"})
        return
    }

    jsonData, err := json.Marshal(user)
    if err != nil {
        c.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{"error": "json marshalling failed"})
        return
    }

    var buf bytes.Buffer
    gz := gzip.NewWriter(&buf)
    if _, err := gz.Write(jsonData); err != nil {
        c.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{"error": "gzip compression failed"})
        return
    }
    if err := gz.Close(); err != nil {
        c.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{"error": "gzip writer close failed"})
        return
    }

    c.Data(http.StatusOK, "application/octet-stream", buf.Bytes())
})

// Endpoint to decompress GZIP compressed JSON data
r.POST("/decompress", func(c *gin.Context) {
    data, err := ioutil.ReadAll(c.Request.Body)
    if err != nil {
        c.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{"error": "failed to read request body"})
        return
    }

    var buf bytes.Buffer
    buf.Write(data)

    gz, err := gzip.NewReader(&buf)
    if err != nil {
        c.AbortWithStatusJSON(http.StatusBadRequest, gin.H{"error": "invalid gzip format"})
        return
    }
    defer gz.Close()

    decompressedData, err := ioutil.ReadAll(gz)
    if err != nil {
        c.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{"error": "gzip decompression failed"})
        return
    }

    var user User
    if err := json.Unmarshal(decompressedData, &user); err != nil {
        c.AbortWithStatusJSON(http.StatusInternalServerError, gin.H{"error": "json unmarshalling failed"})
        return
    }

    c.JSON(http.StatusOK, user)
})

// Run the server
if err := r.Run(":8080"); err != nil {
    panic(err)
}
}