Unicode Encoder / Decoder

What Is Unicode?

Unicode is the universal character encoding standard that assigns a unique number (code point) to every character in every writing system. From basic Latin letters to emoji, CJK ideographs, Vietnamese diacritics, and mathematical symbols — Unicode covers them all. Version 15.1 includes over 149,000 characters from 161 scripts.

When working with international data, you frequently need to convert between plain text and encoded representations like \uXXXX (JavaScript), U+XXXX (standard notation), &#xHHHH; (HTML entities), or raw UTF-8 hex bytes.

Encoding Formats Explained

1. JavaScript Escape (\uXXXX)

Used in JavaScript source code and JSON strings. Each character in the Basic Multilingual Plane (BMP, code points 0–FFFF) is represented as \uXXXX with exactly 4 hex digits. Characters outside the BMP (like emoji) use the extended \u{XXXXX} syntax with 5-6 hex digits, introduced in ES6.

2. U+XXXX (Standard Notation)

This is the canonical way to reference a Unicode code point. It appears universally in technical documentation, character charts, and Unicode discussions. It is not a programming syntax but a notational convention used by the Unicode Consortium.

3. HTML Entities (&#xHHHH;)

Used in HTML and XML to embed special characters without changing the file encoding. Both hex &#xHHHH; and decimal &#DDDD; forms are valid. Especially useful for multilingual web content and ensuring characters render correctly regardless of the document charset declaration.

4. UTF-8 Hex Bytes

Shows how characters are actually stored in memory and files. UTF-8 uses 1–4 bytes per character: ASCII uses 1 byte, accented Latin and Vietnamese use 2–3 bytes, CJK uses 3 bytes, and emoji use 4 bytes. Understanding UTF-8 byte sequences is essential for debugging encoding issues like mojibake (garbled characters).

When Do You Need Unicode Conversion?

• Embedding special characters in JSON strings without breaking the parser.
• Decoding obfuscated escape sequences from API responses or log files.
• Creating HTML entities for safe rendering on web pages across all browsers.
• Debugging encoding problems by inspecting the raw UTF-8 bytes of each character.
• Looking up the code point for any character including emoji, CJK, and accented letters.
• Preparing internationalized text with proper escape sequences for multilingual applications.

Unicode Tips for Developers

• Always use [...str].length instead of str.length in JavaScript to count characters correctly (surrogate pairs count as 2 in .length).
• Use String.fromCodePoint() instead of String.fromCharCode() for characters outside the BMP.
• In Python 3, all strings are Unicode natively — use .encode() to convert to bytes when needed.
• When working with databases, ensure the collation supports full Unicode (e.g., utf8mb4 in MySQL, not just utf8).
• The HTTP header Content-Type: text/html; charset=utf-8 tells browsers how to decode the response correctly.