Code Points vs Code Units: What Is the Difference?

Code points vs code units at a glance

A code point answers “Which Unicode value is this?” A code unit answers “How does this encoding represent it?”

What is a Unicode code point?

A Unicode code point is an abstract numeric value in the Unicode codespace. It is commonly written as U+ followed by hexadecimal digits. Code points are independent of UTF-8, UTF-16 and UTF-32. A code point may represent a letter, symbol, punctuation mark, control, combining mark or emoji, and the same value has different code-unit representations in different encodings. For the broader standard, read What Is Unicode?.

For a deeper definition, read What Is a Unicode Code Point?, inspect text with the Unicode Character Inspector, or search by value with Unicode Character Lookup.

What is a code unit?

A code unit is the smallest unit used by a Unicode encoding form. UTF-8 uses 8-bit code units, UTF-16 uses 16-bit code units and UTF-32 uses 32-bit code units. One code point can require one or more code units, so code-unit count depends on the encoding.

One code point in UTF-8, UTF-16 and UTF-32

The code point is unchanged, but the code-unit sequence differs. UTF-8 uses bytes as code units. UTF-16 uses a surrogate pair for this supplementary code point. UTF-32 uses one 32-bit code unit.

Read Unicode vs UTF-8, compare UTF-8 vs UTF-16, or convert bytes with the UTF-8 Encoder and Decoder.

Interactive code point and code unit analyzer

This local analyzer shows one row per code point and summarizes grapheme clusters plus UTF-8, UTF-16 and UTF-32 units.

Why UTF-8 uses multiple code units

UTF-8 is variable-length. ASCII-range code points use one byte. Higher values use two, three or four bytes. Each byte is one UTF-8 code unit. The leading byte identifies sequence length, and continuation bytes carry additional bits. Invalid byte sequences should not be interpreted as valid code points.

A → U+0041 → 41
é → U+00E9 → C3 A9
字 → U+5B57 → E5 AD 97
😀 → U+1F600 → F0 9F 98 80

Why UTF-16 uses surrogate pairs

UTF-16 uses 16-bit code units. BMP code points usually use one code unit. Supplementary code points require two code units: a high surrogate and a low surrogate. Surrogates are reserved code-point values, and isolated surrogates are not valid Unicode scalar values.

😀
Code point: U+1F600
UTF-16 code units: D83D DE00

D83D → high surrogate
DE00 → low surrogate

Use the Unicode Sequence Analyzer when surrogate pairs or code-unit counts look surprising.

Why UTF-32 uses one code unit per code point

UTF-32 code units are 32 bits, so every Unicode scalar value fits in one UTF-32 code unit. This makes code-point indexing simpler, but UTF-32 usually uses more storage and byte order matters when serialized.

A → 00000041
é → 000000E9
😀 → 0001F600

Code points vs bytes

A byte is always 8 bits. A code point is an abstract Unicode number. In UTF-8, code units are bytes. In UTF-16 and UTF-32, code units contain multiple bytes, and byte count depends on encoding and byte order.

U+00E9 is the code point. C3 A9 is its UTF-8 byte sequence.

Code points vs grapheme clusters

A grapheme cluster is a user-perceived character. One grapheme cluster may contain one or several code points. Combining marks, emoji modifiers and zero-width joiners create multi-code-point graphemes, and code-unit counts can be even larger.

Precomposed letter

é
U+00E9
1 grapheme cluster
1 code point
2 UTF-8 code units
1 UTF-16 code unit

Decomposed letter

e + ◌́
U+0065 U+0301
1 grapheme cluster
2 code points
3 UTF-8 code units
2 UTF-16 code units

Family emoji

👨‍👩‍👧‍👦 may display as one grapheme while containing several code points and many code units.

Use the Unicode Character Counter, Unicode Sequence Analyzer and Unicode Normalizer.

Why string length can be misleading

Programming languages and databases count different units, so the word “length” is incomplete unless the unit is named.

Exact behavior depends on language, runtime and API. Code-point count is not universally equivalent to visible-character count.

Code points and code units in JavaScript

JavaScript strings are sequences of UTF-16 code units. charCodeAt() returns one UTF-16 code unit, codePointAt() can return a full code point, for...of iterates code points, and Intl.Segmenter is needed for grapheme-aware segmentation. Spread syntax is not grapheme-aware.

const text = "😀";

console.log(text.length);       // 2 UTF-16 code units
console.log([...text].length);  // 1 code point

const emoji = "😀";

console.log(emoji.charCodeAt(0).toString(16)); // d83d
console.log(emoji.charCodeAt(1).toString(16)); // de00
console.log(emoji.codePointAt(0).toString(16)); // 1f600

for (const character of "A😀") {
    console.log(character, character.codePointAt(0));
}

const segmenter = new Intl.Segmenter("en", {
    granularity: "grapheme",
});

const count = [...segmenter.segment("👨‍👩‍👧‍👦")].length;

Code points and code units in Python

Python strings represent Unicode text. Iteration usually yields code points, and len() counts string elements/code points in normal Python builds. Encoding produces byte sequences. Grapheme clusters still require a segmentation library.

text = "A😀"

print(len(text))  # 2 code points

for character in text:
    print(character, f"U+{ord(character):04X}")

encoded = text.encode("utf-8")
print(list(encoded))

utf16 = "😀".encode("utf-16-le")
units = [
    int.from_bytes(utf16[index:index + 2], "little")
    for index in range(0, len(utf16), 2)
]

print([f"{unit:04X}" for unit in units])

Code points and code units in PHP

PHP strings are byte sequences. strlen() counts bytes, mb_strlen() performs encoding-aware length calculation, and IntlChar::ord() can retrieve a code point. Unicode-safe iteration requires suitable extensions such as mbstring and intl.

$text = "A😀";

echo strlen($text);
echo mb_strlen($text, "UTF-8");

foreach (mb_str_split($text, 1, "UTF-8") as $character) {
    $codepoint = IntlChar::ord($character);
    printf("%s U+%04X\n", $character, $codepoint);
}

Code points and code units in databases

Databases may expose character length, byte length, column storage limits, index limits, collation and normalization behavior. UTF-8-compatible character sets can store full Unicode, but byte limits can reject strings with few visible characters.

A 10-character field may require more than 10 bytes when stored as UTF-8.

User-facing character limits should not rely only on byte count, and normalization can affect uniqueness comparisons. Database-specific behavior varies, so check the actual database documentation.

Safe string truncation

Truncation is difficult because unsafe approaches can cut UTF-8 at an arbitrary byte offset, cut UTF-16 between surrogate code units, cut after a base character before a combining mark, split an emoji sequence or break a zero-width-joiner sequence.

Common code point and code unit mistakes

Treating JavaScript length as visible characters

It counts UTF-16 code units, so supplementary characters count as two.

Treating UTF-8 bytes as characters

Non-ASCII code points can require multiple bytes.

Splitting surrogate pairs

Broken pairs can produce invalid or replacement output.

Counting code points instead of grapheme clusters

Combining marks and emoji sequences can make several code points one visible character.

Assuming UTF-32 solves character counting

UTF-32 is fixed for code points, not graphemes.

Using byte limits as user-facing character limits

Multilingual text and emoji make byte length differ from visible length.

Confusing code units with code points in APIs

Document whether offsets and lengths are bytes, UTF-16 indexes, code-point indexes or grapheme indexes.

Mixing indexes from different representations

An index from one representation cannot safely be reused in another without conversion.

Confusing normalization with encoding

Normalization changes code-point sequences while encoding changes code points into code units. Read Unicode Normalization Explained before normalizing production identifiers.

Assuming visually identical text has identical code units

Normalization and invisible characters can create different sequences.

How to choose the right counting unit

Use bytes for file-size limits, network payload limits, binary formats and encoded storage. Use code units when an API is defined in UTF-16 indexes or when working directly with an encoding representation. Use code points for Unicode values, scalar validation and property logic. Use grapheme clusters for visible limits, cursor movement, deletion, display names and text editors.

Practical debugging workflow

1. Preserve the original text and bytes.
2. Determine the declared encoding.
3. Inspect code points.
4. Inspect code units in the relevant encoding.
5. Count grapheme clusters.
6. Check normalization.
7. Detect invisible and combining characters.
8. Check for isolated UTF-16 surrogates.
9. Compare byte and character limits.
10. Test with emoji and non-Latin scripts.

Use Unicode Sequence Analyzer, Unicode Character Inspector, Unicode Character Counter, UTF-8 Encoder and Decoder, Unicode Text Compare and Invisible Character Detector.

Try these UnicodeNow tools

These tools help inspect code points, code units, byte length, normalization and visible counts in real input.

Frequently asked questions

References

• The Unicode Standard
• Unicode glossary
• Unicode Character Database
• Unicode Standard Annex #15: Unicode Normalization Forms
• Unicode Standard Annex #29: Unicode Text Segmentation
• RFC 3629: UTF-8, a transformation format of ISO 10646
• MDN: JavaScript strings
• MDN: String.prototype.codePointAt()
• MDN: Intl.Segmenter
• Python documentation: Unicode HOWTO
• PHP manual: Multibyte String