Every piece of text on a screen is stored as a sequence of numbers. Character encoding defines how those numbers map to characters, and getting it wrong produces garbled text, broken URLs, and security bugs.
This guide covers the encodings web developers run into most often: Base64, URL encoding, HTML entities, and Unicode, along with when each one is the right choice.
ASCII: Where It All Started
The American Standard Code for Information Interchange (ASCII) was published in 1963 and defined 128 characters: the English alphabet, digits, punctuation, and control characters, each numbered from 0 to 127 (7 bits). It worked for English and had no way to represent accented letters, Chinese, Arabic, or other writing systems. That limitation drove most of what follows.
Base64: Turning Binary into Text
Base64 is an encoding scheme that converts binary data into a text representation using 64 printable ASCII characters. It was originally designed for email (the MIME standard), where binary attachments needed to travel through text-only channels.
How Base64 Works
The algorithm processes input bytes in groups of three (24 bits). Each group is split into four 6-bit chunks, and each 6-bit value maps to one of 64 characters: A-Z, a-z, 0-9, +, and /. If the input length is not divisible by three, the output is padded with = signs to signal how many bytes were in the final group.
// JavaScript Base64 encoding and decoding
const encoded = btoa("Hello, World!");
console.log(encoded); // "SGVsbG8sIFdvcmxkIQ=="
const decoded = atob("SGVsbG8sIFdvcmxkIQ==");
console.log(decoded); // "Hello, World!"
When to Use Base64
Common uses include embedding small images as data URIs, encoding email attachments, and carrying binary data in JSON payloads or text-only fields. Base64 is not encryption: anyone can decode it. It also increases size by about 33%, because every 3 bytes become 4 characters. Try it with the Base64 Encoder/Decoder.
Two details cause most Base64 bugs. First, btoa() only accepts characters in the 0 to 255 range, so non-Latin text throws an error unless you convert it to bytes first:
btoa("caf\u00e9 \u2713"); // InvalidCharacterError
const bytes = new TextEncoder().encode("caf\u00e9 \u2713");
const encoded = btoa(String.fromCharCode(...bytes));
console.log(encoded); // "Y2Fmw6kg4pyT"
Second, standard Base64 uses + and /, which have special meaning in URLs. The URL-safe variant (Base64url, used in JWTs) replaces them with - and _ and usually drops the padding.
URL Encoding: Making URLs Safe
URLs have a restricted character set. Characters like spaces, ampersands, question marks, and non-ASCII characters cannot appear directly in a URL because they have special meaning or are simply not allowed by the specification. URL encoding (also called percent-encoding) solves this by replacing unsafe characters with a percent sign followed by their hex value.
Space -> %20
& -> %26
= -> %3D
? -> %3F
/ -> %2F
cafe latte -> cafe%20latte
café -> caf%C3%A9 (the UTF-8 bytes of é, each percent-encoded)
encodeURIComponent vs. encodeURI
JavaScript provides two functions for URL encoding, and using the wrong one is a common source of bugs:
const query = "name=Alice & Bob";
// encodeURIComponent: encodes everything except A-Z a-z 0-9 - _ . ! ~ * ' ( )
encodeURIComponent(query);
// "name%3DAlice%20%26%20Bob"
// encodeURI: preserves URL structure characters (: / ? # & = etc.)
encodeURI("https://example.com/search?q=" + query);
// "https://example.com/search?q=name=Alice%20&%20Bob" (BROKEN!)
// Correct approach for query parameters:
"https://example.com/search?q=" + encodeURIComponent(query);
// "https://example.com/search?q=name%3DAlice%20%26%20Bob"
The rule: use encodeURIComponent() for individual query parameter values and path segments, and encodeURI() only for a complete URL. For query strings, URLSearchParams handles the encoding for you, though it writes spaces as + rather than %20:
new URLSearchParams({ q: "Alice & Bob" }).toString();
// "q=Alice+%26+Bob"
Experiment with the URL Encoder/Decoder.
HTML Entities: Safe Characters in Markup
In HTML, certain characters have special meaning. The less-than sign < opens a tag. The ampersand & starts an entity reference. If you want to display these characters as literal text, you must encode them as HTML entities.
Named vs. Numeric Entities
HTML entities come in two forms. Named entities use readable labels like < for the less-than sign and & for the ampersand. Numeric entities use the character's code point in decimal (<) or hex (<). Named entities are easier to remember, while numeric entities work for any Unicode character.
XSS Prevention
If your application inserts user content into HTML without encoding it, an attacker can inject <script> tags that execute in other visitors' browsers. This is Cross-Site Scripting (XSS). The safest approach is to let your framework's templating escape output, or to set textContent rather than innerHTML. HTML entity encoding only protects HTML text and quoted attribute values; JavaScript and URL contexts need their own handling. Inspect entities with the HTML Entity Encoder/Decoder.
Unicode
Unicode aims to include every character from every writing system, plus mathematical symbols and emoji, with more than 150,000 characters assigned so far. Unicode itself is a numbered list of characters. Encodings such as UTF-8 and UTF-16 define how those numbers are stored as bytes.
Code Points
Every Unicode character is assigned a unique number called a code point, written as U+ followed by a hex value:
A U+0041
e (accented) U+00E9
Sigma U+03A3
Han character U+4E16
Smiling face U+1F600
UTF-8 Encoding
UTF-8 is the dominant encoding on the web: W3Techs surveys report it on over 98% of websites. It is a variable-width encoding that uses 1 to 4 bytes per character:
| Code Point Range | Bytes | Example |
|---|---|---|
| U+0000 to U+007F | 1 | ASCII characters (A, 5, !) |
| U+0080 to U+07FF | 2 | Latin accented, Greek, Cyrillic |
| U+0800 to U+FFFF | 3 | Chinese, Japanese, Korean, most symbols |
| U+10000 to U+10FFFF | 4 | Emoji, historic scripts, math symbols |
Any valid ASCII text is also valid UTF-8, because the first 128 code points use one byte each. That is a large part of why UTF-8 spread so easily.
Surrogate Pairs and Emoji
JavaScript strings use UTF-16 internally. Characters with code points above U+FFFF (like most emoji) are stored as surrogate pairs, which are two 16-bit code units that together represent a single character:
const emoji = "\u{1F600}"; // Grinning face
console.log(emoji.length); // 2 (two UTF-16 code units!)
console.log([...emoji].length); // 1 (one code point)
console.log(emoji.codePointAt(0)); // 128512 (decimal for U+1F600)
Spreading into an array counts code points, which is still not what users think of as characters. A family emoji is several code points joined by zero-width joiners. For user-visible characters, use Intl.Segmenter:
const family = "\u{1F468}\u200D\u{1F469}\u200D\u{1F467}";
family.length; // 8
[...family].length; // 5
[...new Intl.Segmenter().segment(family)].length; // 1
Explore code points with the Unicode Converter.
Binary and Hexadecimal Representations
At the lowest level, all data is binary: sequences of ones and zeros. Hexadecimal (base 16) provides a more compact way to represent binary data, where each hex digit maps to exactly 4 bits:
Character: A
ASCII: 65
Binary: 01000001
Hex: 41
Character: Z
ASCII: 90
Binary: 01011010
Hex: 5A
Hex appears everywhere: CSS color codes (#FF5733), memory addresses, MAC addresses, and cryptographic hashes. Convert between text, binary, and hex using our Binary to Text Converter and Hex Editor.
Morse Code: A Historical Perspective
Before digital encoding, there was Morse code. Developed by Samuel Morse and Alfred Vail in the 1830s, Morse code encodes letters as sequences of short signals (dots) and long signals (dashes). It was the first widely adopted system for encoding text for electronic transmission, and it remained in active use for maritime communication until the late 1990s.
H ....
E .
L .-..
L .-..
O ---
"HELLO" in Morse: .... . .-.. .-.. ---
Morse code is a variable-length encoding: common letters (E, T) get short codes, while rare ones (Q, Z) get longer ones. This is the same principle behind modern Huffman coding. Play with it using our Morse Code Translator.
Common Pitfalls and How to Avoid Them
Double Encoding
One of the most frequent encoding bugs is applying the same encoding twice. For URL encoding, this turns a space into %20, and then into %2520 because the percent sign itself gets encoded. The fix is simple: always encode exactly once, at the boundary where data enters a new context.
Encoding Mismatches and Mojibake
If a file is saved as UTF-8 but read as ISO-8859-1 or Windows-1252, characters outside ASCII come out garbled. For example, café displays as café, because the two UTF-8 bytes of é are read as two separate characters. This is called mojibake. Declare <meta charset="UTF-8"> and have your server send a matching Content-Type header.
Storing Encoded Values
If a user submits "Tom & Jerry" and you store it as Tom & Jerry, the database now holds encoded text. When your template engine encodes it again on output, the user sees the literal text Tom & Jerry on the page. Store raw data, and encode only at the point of output.
A Quick Reference
| Encoding | Use Case | Size Overhead |
|---|---|---|
| Base64 | Binary data in text contexts | ~33% |
| URL Encoding | Special characters in URLs | ~3x for encoded chars |
| HTML Entities | Special characters in HTML | 4 to 8 chars per entity |
| UTF-8 | Universal text encoding | 1 to 4 bytes per character |
| Hex | Byte-level data inspection | 2x |
Tools for Encoding Work
Each of these does its conversion in page JavaScript and makes no network request with your input: