Stack Overflow
Stack Overflow

Questions tagged [varint]

Varints are a method that stores integers in bytes by serialization.

12 questions
20
votes
4 answers

Why is varint an efficient data representation?

I am currently studying the documentation of protocol buffers. Varints are described as: Each byte in a varint, except the last byte, has the most significant bit (msb) set – this indicates that there are further bytes to come. The lower 7 bits…
user878242
15
votes
6 answers

Optimizing variable-length encoding

I've got a case where I need to compress a lot of often small values. Thus I compress them with a variable-length byte encoding (ULEB128, to be specific): size_t compress_unsigned_int(unsigned int n, char* data) { size_t size = 0; while (n >…
Alexandre Hamez
  • 7,725
  • 2
  • 28
  • 39
6
votes
1 answer

Given an integer, how big is its varint encoding?

I have a python list of integers, and I'd like to know how much space it will take up when encoded as a sequence of Protocol Buffers variable-length integers, or varints. What's the best way to figure this out without actually encoding the…
Emmett Butler
  • 5,969
  • 2
  • 29
  • 47
5
votes
1 answer

How to write LEB128 in Go

How do you write an integer to LEB128 format in Go? I'm trying to encode an int32 to a Minecraft VarInt, so far I've tried importing the example on the wiki to Go. I get the wrong results when testing though, wiki says -1 should equal [255 255 255…
Angel
  • 81
  • 1
  • 6
4
votes
2 answers

Is there any difference in the efficiency of counting leading/trailing ones/zeros?

I'm designing a prefixed variable length integer. Rust has methods for counting leading and trailing ones and zeros: https://doc.rust-lang.org/std/primitive.u64.html#method.leading_zeros Is there any difference in the efficiency of these methods on…
fadedbee
  • 42,671
  • 44
  • 178
  • 308
3
votes
1 answer

QWORD shuffle sequential 7-bits to byte-alignment with SIMD SSE...AVX

I would like to know if the following is possible in any of the SIMD families of instructions. I have a qword input with 63 significant bits (never negative). Each sequential 7 bits starting from the LSB is shuffle-aligned to a byte, with a…
IamIC
  • 17,747
  • 20
  • 91
  • 154
2
votes
2 answers

Decode a prepended VarInt from a byte stream of unknown size in javascript/Nodejs

I am writing a small utility library for me to request server status of a given minecraft host in js on node. I am using the Server List Ping Protocol as outlined here (https://wiki.vg/Server_List_Ping) and got it mostly working as expected, albeit…
AbandonedCrypt
  • 187
  • 1
  • 12
2
votes
1 answer

Can you check a flag on a byte, AND retrieve the remaining 7-bit integer value, in one assembly operation?

What is the most optimal way (fewest/fastest operations) to take an 8-bit, 16-bit, 32-bit or 64-bit number, extract the first bit off of it, check if that bit is true, and meanwhile store the resulting number after removing the leading bit? (In…
Lance
  • 75,200
  • 93
  • 289
  • 503
1
vote
1 answer

How do varints take less space?

I'm trying to learn about varints and the best thing I found is this Google Protocol Buffers spec. In their example, they show that this number 1010 1100 0000 0010, when encoded with varints, is 300 as opposed to 44034. Normally the number 300 takes…
Maxime Dupré
  • 5,319
  • 7
  • 38
  • 72
0
votes
0 answers

I need help implementing signed VarInt's in C++

I have been working on a program that can connect to a Minecraft Server and exchange packets with it but Minecraft's server packets heavily rely on signed VarInts. On the site documenting how their communication works is explanation of VarInt and…
themakabrapl
  • 1
  • 2
0
votes
1 answer

Negative number causes VarInt's encoding function to crash

I try to implement the function below in Python but as soon as I enter a negative number the function crashes. Does anyone understand why? public static void writeVarInt(int value) { do { byte temp = (byte)(value & 0b01111111); …
Tao
  • 11
  • 3
0
votes
1 answer

How to decode a specific byte string of varints in PHP

I'm trying to decode a string in a specific format (Hearthstone deck code), using PHP, like this: AAEBAc2xAgjAAe0E7QX3DdYRh6wC8fsCoIADC8kDqwTLBPsMhRDH0wKW6AK0/ALNiQPXiQOfmwMA or AAEBAf0GBAD6DoyAA6CAAw37AZwCigbJB/gHlA+CEIUQrRDy0AL2/QKJgAPRgAMA The…
oelna
  • 2,210
  • 3
  • 22
  • 40