Press Backspace once.
One visible character disappears.
But the software may have removed several Unicode code points.
A user-perceived character is often a grapheme cluster: one visible text unit assembled from one or more Unicode code points.
This is the first pillar beneath How Encoding Works. The master separates meaning, character, glyph, code point and byte. This article owns the boundary between encoded code points and the text units users actually select, delete, count and navigate.
Quick Read
Unicode code points are encoding units, not guaranteed user-visible characters. A letter plus a combining accent can be several code points but one user-perceived character. Emoji can contain base emoji, skin-tone modifiers, variation selectors and zero-width joiners while still appearing as one symbol. Unicode Standard Annex #29 defines default grapheme cluster boundaries so software can approximate these user-perceived units consistently. Good text systems therefore distinguish bytes, code units, code points, grapheme clusters and glyphs rather than calling all of them “characters.”
bytes → decoded code points → grapheme segmentation → user-perceived characters → shaping → glyphs
One Word, Five Different Counting Units
Ask a programmer:
How long is this string?
The answer is incomplete until the unit is named.
- bytes: storage or transmission units;
- code units: units used by a particular encoding/API, such as UTF-16 code units;
- code points: Unicode scalar or assigned positions;
- grapheme clusters: approximate user-perceived characters;
- glyphs: rendered shapes selected by a font/shaping system.
These counts can all differ for the same visible text.
Unicode UAX #29 Calls Grapheme Clusters “User-Perceived Characters”
The current published Unicode 17 version of Unicode Standard Annex #29: Unicode Text Segmentation describes grapheme clusters as a programmatic approximation to user-perceived characters. It explicitly notes that one basic user text unit can consist of multiple Unicode code points.
That wording is important.
Unicode is not claiming that every language has one universal psychological definition of “character.” It provides interoperable default boundaries that work well across many scripts and can be tailored where language or application rules need something different.
A Combining Mark Is the Simplest Example
Take a Latin letter and add a combining accent.
At code-point level:
base letter + combining mark
At user level, the result is normally experienced as one accented character.
If a text editor places the cursor between the base and its accent during ordinary navigation, it can behave in a way that feels broken even though the underlying Unicode sequence is technically valid.
Precomposed and Decomposed Text Can Look the Same
Some accented forms can be represented as one precomposed code point or as a base code point followed by one or more combining marks.
These sequences can be canonically equivalent while having different code-point counts.
The second pillar, Unicode Normalization, owns why equivalent text can have different binary representations and how NFC/NFD handle that difference.
Grapheme Boundaries Are Designed to Survive Canonical Equivalence
UAX #29 makes a valuable promise: default grapheme cluster boundaries are designed so canonically equivalent forms have corresponding boundaries.
That means a user interface can treat the composed and decomposed forms as the same editing unit even when the underlying sequences differ.
Emoji Make the Layering Impossible to Ignore
An emoji that appears as one symbol can involve several encoded elements.
- a base emoji;
- a variation selector;
- a skin-tone modifier;
- a zero-width joiner;
- another emoji component;
- possibly additional joined components.
The visible symbol can therefore be much richer than “one code point.”
Zero-Width Joiner Sequences Are Explicitly Composite
A zero-width joiner, often abbreviated ZWJ, can participate in emoji sequences that request a combined presentation.
The joiner itself is not a visible picture.
It changes how neighbouring encoded elements are treated as a sequence.
UAX #29 and Unicode emoji specifications therefore need segmentation rules that avoid splitting many such sequences during ordinary editing.
A Flag Can Be Two Regional Indicators
Many flag emoji are encoded as pairs of regional-indicator symbols.
The user sees one flag.
The code-point sequence contains more than one encoded element.
Again, display-level character count and code-point count diverge.
Indic and Southeast Asian Scripts Make “Character” Even More Contextual
In many writing systems, consonants, vowel signs, combining marks and shaping behaviour form orthographic units that do not map cleanly to a Western one-letter-one-code-point intuition.
UAX #29 recommends extended grapheme clusters because they generally produce better editing units across scripts, while also recognising that some languages and applications may need tailored segmentation.
The important engineering lesson is not to force every script into the text assumptions of English.
Cursor Movement Is a Segmentation Decision
Press the left-arrow key once.
Should the cursor move one byte?
One UTF-16 code unit?
One code point?
One grapheme cluster?
For ordinary text editing, users usually expect movement by a visible text unit rather than by an arbitrary encoding unit.
Backspace Is Also a Segmentation Decision
If a visible accented letter consists of a base plus combining mark, deleting only the mark can leave the base behind.
Sometimes that behaviour is desirable in specialist editors.
Sometimes users expect the whole grapheme to disappear.
There is no single UI rule for every application, but the implementation must know which layer it is editing.
Text Selection Needs the Same Awareness
Drag across one emoji sequence.
If the selection splits it in the middle, copying and pasting can produce a surprising or incomplete result.
Segmentation-aware selection treats many multi-code-point sequences as atomic user units by default.
Character Limits Are Usually Underspecified
A form says:
Maximum 20 characters.
What does that mean?
- 20 bytes?
- 20 Unicode code points?
- 20 UTF-16 code units?
- 20 grapheme clusters?
For user-facing length, grapheme clusters may be the closer conceptual match—but databases, protocols and security limits may need byte or code-point ceilings too.
Storage Limits and User Limits Should Be Separate
A database field has a byte budget.
A product specification has a visible-name limit.
Do not pretend those are automatically the same quantity.
A safe design may enforce both:
- maximum encoded bytes for storage/transport;
- maximum grapheme clusters for the user experience.
Programming Languages Expose Different Units
Some string APIs index bytes.
Some index UTF-16 code units.
Some provide Unicode-scalar iteration.
Grapheme segmentation is often a separate operation.
Therefore:
string.length is not a universal human-character count.
Regex “One Character” Can Mean the Wrong Layer
A regular-expression dot may consume a code point, code unit or implementation-defined text element rather than one grapheme cluster.
Unicode-aware regex libraries can provide grapheme-cluster matching, but the syntax and behaviour vary.
Pattern logic should state whether it is validating encoded units or user-perceived units.
Search Has Yet Another Boundary
Search systems may tokenize words, normalise canonical equivalents, fold case and handle script-specific boundaries.
Grapheme clusters are useful text elements, but they are not automatically search tokens or linguistic words.
How Indexing Works owns the later search representation stage.
Glyphs Are Downstream Again
A grapheme cluster is not necessarily one glyph.
Fonts and shaping engines can:
- combine several characters into a ligature;
- choose contextual forms;
- position marks;
- substitute script-specific glyph sequences.
Encoding gives the abstract text sequence.
Segmentation groups text elements.
Shaping decides how those elements become visible glyphs.
Fonts Do Not Define Unicode Identity
Change the font and a glyph can look different while the code points remain identical.
Conversely, visually similar glyphs can represent different code points.
Never use visual sameness alone as a text-identity test.
Security Problems Live Between These Layers
An identifier may contain visually confusable characters.
A UI may truncate after a number of glyphs while the backend evaluates code points.
Combining marks can make text visually unusual while remaining valid Unicode.
Security-sensitive systems need policies for identifiers, normalization, confusables and length that go beyond “it displays correctly.”
Do Not Normalize Merely to Make Counting Easier
Normalization can standardise canonically equivalent sequences.
It is not a general substitute for grapheme segmentation.
Even NFC text can contain multi-code-point grapheme clusters such as many emoji sequences and script combinations.
Segmentation Is Versioned Knowledge
Unicode segmentation rules evolve as scripts and emoji behaviour are refined.
As of 31 August 2026, Unicode 17.0 remains the latest published Unicode Standard; Unicode 18.0 is still pre-release for September 2026.
Implementations should know which Unicode/CLDR segmentation data they use rather than assuming every library version produces identical boundaries forever.
CLDR Tailoring Can Improve Locale Behaviour
UAX #29 explicitly permits profiles and tailoring, and recommends CLDR extended rules where additional locale or emoji behaviour is needed.
The default algorithm is a strong interoperable baseline, not a claim that all text communities experience boundaries identically.
A Better Text-Unit Model
information → Unicode code points → normalization state → grapheme segmentation → shaping/font → glyphs → user interaction
Each arrow has a different job.
Debugging improves dramatically when the failing layer is named.
A 30-Lens Grapheme Cluster Audit
- User unit: what does the person perceive as one character?
- Bytes: how many encoded bytes?
- Code units: what does the API index?
- Code points: how many Unicode points?
- Grapheme clusters: how many segmentation units?
- Glyphs: how many rendered shapes?
- Combining marks: are marks attached to a base?
- Canonical equivalence: could composed/decomposed forms differ?
- Emoji modifier: is skin tone or variation state involved?
- ZWJ: is a joined emoji sequence involved?
- Regional indicators: is a flag sequence present?
- Script: do orthographic units require tailoring?
- Unicode version: which segmentation data apply?
- CLDR profile: is locale tailoring used?
- Cursor: what unit moves one step?
- Backspace: what unit is deleted?
- Selection: can the UI split a grapheme?
- Substring: can slicing cut a combining sequence?
- Limit: bytes, code points or graphemes?
- Storage: is byte capacity separate from UX length?
- Regex: what does one pattern unit match?
- Search: are graphemes being confused with words?
- Shaping: is glyph count being confused with text count?
- Font: can rendering alter visible form?
- Normalization: is segmentation being mistaken for normalization?
- Security: can confusables or combining marks mislead?
- Testing: are multilingual/emoji cases included?
- Round trip: does editing preserve valid text?
- Accessibility: how do assistive technologies expose the unit?
- Receiver: does the application behave as users of that script expect?
Laboratory 1: Count the Same String Five Ways
Take a string containing ASCII, an accented character and an emoji sequence. Count bytes, code units, code points, grapheme clusters and visible glyphs. Explain why the answers differ.
Laboratory 2: Backspace Test
Enter a decomposed accented character and a joined emoji sequence into several editors. Observe cursor movement and deletion. Which applications behave by code point and which approximate grapheme clusters?
Laboratory 3: Twenty “Characters”
Design a username field with a 20-character user limit and a separate storage byte limit. Explain why one numeric ceiling cannot safely serve both jobs.
For Primary Readers
Show a letter with an accent made from two magnetic pieces. The computer can store two pieces while the reader sees one written character.
For Secondary Readers
Distinguish code point from grapheme cluster and explain why emoji and combining marks break the assumption that one visible symbol equals one encoded point.
For Advanced Readers
Model a grapheme cluster as a segmentation-layer equivalence unit over Unicode code-point sequences. User-interface atomicity is therefore downstream of decoding and normalization but upstream of shaping, with locale-specific tailoring allowed where default segmentation is insufficient.
Common Misconceptions
- “One character equals one Unicode code point.” Many user-perceived characters contain several code points.
- “One grapheme cluster always equals one glyph.” Shaping can map text elements to glyphs in more complex ways.
- “NFC makes every visible character one code point.” Many legitimate multi-code-point graphemes remain multi-code-point after normalization.
- “String length tells me what users see.” APIs may count bytes or code units instead.
- “Unicode has one perfect universal definition of character.” UAX #29 provides default programmable approximations and allows tailoring.
Research Corridor
- Unicode Standard Annex #29 — Unicode Text Segmentation — extended grapheme clusters and user-perceived characters.
- Unicode Standard Annex #15 — Unicode Normalization Forms — canonical equivalence and normalized forms.
- Unicode Consortium — Recent Releases — current published Unicode version.
- eduKateSG — How Encoding Works.
Frequently Asked Questions
What is a grapheme cluster?
A grapheme cluster is a Unicode text-segmentation unit intended to approximate what a user experiences as one character, and it can contain one or several code points.
Why can one emoji contain several code points?
Emoji sequences can combine base emoji, modifiers, regional indicators, variation selectors and zero-width joiners into one displayed unit.
Should form limits count grapheme clusters?
For a user-visible “character” limit, grapheme clusters are often closer to user expectation. Systems may still need separate byte or code-point limits for storage, protocol and security reasons.
Final Thought: Users Edit Text, Not Encoding Internals
The bytes matter.
The code points matter.
But the cursor belongs to the person.
Good encoding systems respect every layer: machine units underneath, human-perceived units at the interface, and an explicit contract connecting the two.
ENCODING · FOUR PILLAR LEGS
Return to How Encoding Works, or continue through Unicode Normalization, Mojibake and Malformed Input. Return to the Information & Representation Hub.