22

I need to count how long in bytes a textarea is when UTF8 encoded using javascript. Any idea how I would do this?

thanks!

mcintyre321
  • 12,996
  • 8
  • 66
  • 103

10 Answers10

19
encodeURIComponent(text).replace(/%[A-F\d]{2}/g, 'U').length
Tgr
  • 27,442
  • 12
  • 81
  • 118
  • 2
    This is pretty slick. The one issue is that it will throw if the string contains an invalid surrogate pattern. E.g. `encodeURIComponent('\ud800a')`. Just something to be aware of. – broofa Aug 30 '12 at 21:10
  • 1
    How can you insert into textarea a string which contains an invalid surrogate pattern? I tried to insert the text '\ud800a' to this [test page](http://mothereff.in/byte-counter) (which uses `encodeURI` -function internally to encode inserted text) but couldn't reproduce such an error situation - instead I saw: `document.getElementsByTagName("textarea")[0].value === "\\ud800a"`. – Lauri Oherd Sep 02 '12 at 08:03
  • Used for counting length of UTF-8 string. – Satya Prakash Oct 11 '13 at 07:58
  • 1
    @LauriOherd: (very!) late response here, but to answer your question, textareas will accept invalid strings. E.g. `textarea.value = '\ud800' && encodeURIComponent(textarea.value)` will throw (at least, in Chrome it will). – broofa Feb 19 '19 at 18:57
18

Combining various answers, the following method should be fast and accurate, and avoids issues with invalid surrogate pairs that can cause errors in encodeURIComponent():

function getUTF8Length(s) {
  var len = 0;
  for (var i = 0; i < s.length; i++) {
    var code = s.charCodeAt(i);
    if (code <= 0x7f) {
      len += 1;
    } else if (code <= 0x7ff) {
      len += 2;
    } else if (code >= 0xd800 && code <= 0xdfff) {
      // Surrogate pair: These take 4 bytes in UTF-8 and 2 chars in UCS-2
      // (Assume next char is the other [valid] half and just skip it)
      len += 4; i++;
    } else if (code < 0xffff) {
      len += 3;
    } else {
      len += 4;
    }
  }
  return len;
}
broofa
  • 37,461
  • 11
  • 73
  • 73
  • I had a badly design situation where i was forced to count the bytes explicitly and handle it out. On top of the above snippet, I also had to add a handling for next line characters since they are also 2 bytes. – RBz Jun 23 '20 at 02:38
  • 1
    @RBz Are you referring to the NEL (U+0085) character? That should be counted properly by this function as `0x7f < NEL < 0x7ff`. Regardless, it turns out there is now a `TextEncoder` API that most JS environments support. See my recent edit to the accepted answer, above. – broofa Jun 23 '20 at 14:26
  • I am not sure how this works, but if I press enter it was counted as 1 by this snippet. I read earlier chrome was treating it as 2 and now they have fixed it to reflect 1. However, for me, it had to be counted as 2 since by back-end database treats it as 2. – RBz Jun 24 '20 at 06:50
  • 1
    @RBz Be aware there are a perhaps surprising number of line termination characters in Unicode. Some encode as one byte, some as two bytes. So it really depends on what specific character(s) are used/expected. See https://en.wikipedia.org/wiki/Newline#Unicode . – broofa Jun 24 '20 at 15:08
16

[June 2020: The previous answer has been replaced due to it returning incorrect results].

Most modern JS environments (browsers and Node) now support the TextEncoder API, which may be used as follows to count UTF8 bytes:

const textEncoder = new TextEncoder();
textEncoder.encode('⤀⦀⨀').length; // => 9

This is not quite as fast as the getUTF8Length() function mentioned in other answers, below, but should suffice for all but the most demanding use cases. Moreover, it has the benefit of leveraging a standard API that is well-tested, well-maintained, and portable.

broofa
  • 37,461
  • 11
  • 73
  • 73
derflocki
  • 863
  • 1
  • 11
  • 19
  • 5
    I don't think this implementation is correct since it counts surrogate characters twice: once when encountering the high surrogate, then once when encountering the low one. For example, the following returns 6: getUTF8Length(String.fromCharCode(0xD800, 0xDC00)) although this represents a single character (I must admit I don't know which one, I just combined 2 surrogate char codes…). I am no expert in unicode though… – Didier L Apr 04 '12 at 07:57
  • @Didier L, yes you are right! It should be added to the case list and be accounted for – Sebastian Nov 16 '12 at 15:58
14

If you have non-bmp characters in your string, it's a little more complicated...

Because javascript does UTF-16 encode, and a "character" is a 2-byte-stack (16 bit) all multibyte characters (3 and more bytes) will not work:

    <script type="text/javascript">
        var nonBmpString = "foo€";
        console.log( nonBmpString.length );
        // will output 5
    </script>

The character "€" has a length of 3 bytes (24bit). Javascript does interpret it as 2 characters, because in JS, a character is a 16 bit block.

So to correctly get the bytesize of a mixed string, we have to code our own function fixedCharCodeAt();

    function fixedCharCodeAt(str, idx) {
        idx = idx || 0;
        var code = str.charCodeAt(idx);
        var hi, low;
        if (0xD800 <= code && code <= 0xDBFF) { // High surrogate (could change last hex to 0xDB7F to treat high private surrogates as single characters)
            hi = code;
            low = str.charCodeAt(idx + 1);
            if (isNaN(low)) {
                throw 'Kein gültiges Schriftzeichen oder Speicherfehler!';
            }
            return ((hi - 0xD800) * 0x400) + (low - 0xDC00) + 0x10000;
        }
        if (0xDC00 <= code && code <= 0xDFFF) { // Low surrogate
            // We return false to allow loops to skip this iteration since should have already handled high surrogate above in the previous iteration
            return false;
            /*hi = str.charCodeAt(idx-1);
            low = code;
            return ((hi - 0xD800) * 0x400) + (low - 0xDC00) + 0x10000;*/
        }
        return code;
    }

Now we can count the bytes...

    function countUtf8(str) {
        var result = 0;
        for (var n = 0; n < str.length; n++) {
            var charCode = fixedCharCodeAt(str, n);
            if (typeof charCode === "number") {
                if (charCode < 128) {
                    result = result + 1;
                } else if (charCode < 2048) {
                    result = result + 2;
                } else if (charCode < 65536) {
                    result = result + 3;
                } else if (charCode < 2097152) {
                    result = result + 4;
                } else if (charCode < 67108864) {
                    result = result + 5;
                } else {
                    result = result + 6;
                }
            }
        }
        return result;
    }

By the way... You should not use the encodeURI-method, because, it's a native browser function ;)

More stuff:


Cheers

frankneff.ch / @frank_neff
frank_neff
  • 952
  • 8
  • 16
  • Hi Frank, I used your method and it works correctly for multi byte char strings. i have a text area where I need to count chars / bytes as soon as user types. I tried the key press event but it does not get fired when we do copy / paste. Can you please suggest some reliable & efficient way to count bytes while user types? I need to show a count like "300 left.." Thanks & regards, Nadeem – Nadeem Ullah Oct 15 '12 at 13:29
  • There is no need for the `else if (charCode < 67108864) {}` bit and the `else` that follows it. Unicode stops at U+10FFFF and it’s impossible to represent a non-Unicode code point in JavaScript. – Mathias Bynens Aug 21 '13 at 18:17
  • This is true according to the RFC3629 specification. But the original specification allows up to six byte characters. I'm not sure which implementation should be respected but I would say this is the correct solution. –  May 14 '14 at 12:51
  • @DaanBiesterbos: JavaScript uses UTF-16[\*](https://mathiasbynens.be/notes/javascript-encoding "effectively; the answer’s the same"), though, which can’t represent codepoints (the ones that don’t exist) above U+10FFFF anyway. – Ry- Nov 02 '15 at 07:17
  • @frank_neff What's wrong with using a native browser function? – Daniel Lidström Jan 27 '17 at 13:39
2

Add Byte length counting function to the string

String.prototype.Blength = function() {
    var arr = this.match(/[^\x00-\xff]/ig);
    return  arr == null ? this.length : this.length + arr.length;
}

then you can use .Blength() to get the size

Ryan Wu
  • 5,963
  • 2
  • 36
  • 47
0
encodeURI(text).split(/%..|./).length - 1
Lauri Oherd
  • 1,383
  • 1
  • 12
  • 14
0

How about simple:

unescape(encodeURIComponent(utf8text)).length

The trick is that encodeURIComponent seems to work on characters while unescape works on bytes.

qbolec
  • 5,374
  • 2
  • 35
  • 44
  • the `unescape` function is [deprecated and obsolete as of JavaScript 1.5](https://developer.mozilla.org/en-US/docs/JavaScript/Guide/Functions#escape_and_unescape_functions(Obsoleted_above_JavaScript_1.5)) – jvatic May 15 '13 at 22:30
0

I have been asking myself the same thing. This is the best answer I have stumble upon:

http://www.inter-locale.com/demos/countBytes.html

Here is the code snippet:

<script type="text/javascript">
 function checkLength() {
    var countMe = document.getElementById("someText").value
    var escapedStr = encodeURI(countMe)
    if (escapedStr.indexOf("%") != -1) {
        var count = escapedStr.split("%").length - 1
        if (count == 0) count++  //perverse case; can't happen with real UTF-8
        var tmp = escapedStr.length - (count * 3)
        count = count + tmp
    } else {
        count = escapedStr.length
    }
    alert(escapedStr + ": size is " + count)
 }

but the link contains a live example of it to play with. "encodeURI(STRING)" is the building block here, but also look at encodeURIComponent(STRING) (as already point out on the previous answer) to see which one fits your needs.

Regards

-1

Try the following:

function b(c) {
     var n=0;
     for (i=0;i<c.length;i++) {
           p = c.charCodeAt(i);
           if (p<128) {
                 n++;
           } else if (p<2048) {
                 n+=2;
           } else {
                 n+=3;
           }
      }return n;
}
Sankumarsingh
  • 9,889
  • 11
  • 50
  • 74
-2

set meta UTF-8 just & it's OK!

<meta charset="UTF-8">
<meta http-equiv="content-type" content="text/html;charset=utf-8">

and js:

if($mytext.length > 10){
 // its okkk :)
}
akc42
  • 4,893
  • 5
  • 41
  • 60