Why We Replaced IndexedDB with SQLite WASM — A Game Changer for Large Datasets

At Supersorted, We build an offline-first email client designed to handle thousands of messages with real-time responsiveness. We started with a React + Vite + TypeScript stack and integrated it with the Gmail API to deliver a fast, desktop-like experience directly in the browser.

As the product grew, our biggest challenge was performance at scale — especially managing large mailboxes locally. IndexedDB soon hit its limits, leading to slow queries and noticeable UI lag. To fix this, I migrated our storage layer to SQLite (via WebAssembly) and added an in-memory B-tree caching layer for instant lookups. This architectural shift cut load times by nearly 10× and made the client feel genuinely native, even offline.

In this post, I’ll walk through that journey - from the early performance struggles to the design decisions that shaped a faster, more reliable email client.

The Struggle

Performance bottlenecks that kept us up at night:

• Paginated queries taking 2-5 seconds to return results
• No efficient way to filter across multiple fields simultaneously
• Complex joins requiring multiple round trips to IndexedDB
• Inconsistent write performance under load

We weren't just fighting slow queries—we were compromising on features. Advanced filtering? Too slow. Complex search? Not feasible. We had to choose between functionality and performance.

The Workaround (And Its Cost)

Desperate for performance, we loaded the entire mailbox into memory at app startup. Yes, you read that right—100,000+ email records, all in JavaScript objects, consuming gigabytes of RAM just to make queries feel instant.

// The painful reality
const allMessages = await loadAllMessagesFromIndexedDB(); // Takes 10+ seconds
memdb.bulkInsert(allMessages); // Our internal memory database built with BTree.

// Then we could "query" from memory
const results = memdb.where({ sender: 'john@example.com', hasAttachments: true }).toArray();

The tradeoffs were brutal:

• 700MB to 1.5GB memory usage for a single mailbox
• Slow initial load times
• Browser tab crashes when memory ran out
• Had to abandon features that required complex queries

We accepted these compromises because IndexedDB simply couldn't deliver the performance we needed.

The Discovery: SQLite WASM

SQLite compiled to WebAssembly runs entirely in the browser. You get a full SQL engine without a server.

Why It Caught My Attention

• Real SQL: Full SQL queries, joins, subqueries, everything
• True indexes: Create indexes exactly where you need them
• ACID guarantees: Transactions and data integrity built-in
• Minimal overhead: ~3MB gzipped, runs in a Web Worker
• OPFS persistence: Data survives browser restarts via Origin Private File System

The more we researched, the more we realized: why are we treating email data like a NoSQL document store when it's clearly relational?

The Migration: From Struggle to Relief

The migration wasn't trivial, but the results were immediate.

High-Level Implementation

Using the official SQLite WASM package and comprehensive documentation:

import sqlite3InitModule from '@sqlite.org/sqlite-wasm';

// Initialize SQLite in a Web Worker
const sqlite3 = await sqlite3InitModule.default();
const PoolUtil = await sqlite3.installOpfsSAHPoolVfs({
	initialCapacity: 3,
	clearOnInit: false, // Preserve data across sessions
	name: 'myapp-pool',
});

// Open database
const db = new PoolUtil.OpfsSAHPoolDb({
	filename: 'mail.db',
	flags: 'create',
	vfs: 'opfs-sahpool',
});

Setup was straightforward. Create tables, define indexes, start querying.

Schema Design for Performance

CREATE TABLE messages (
  messageId TEXT PRIMARY KEY,
  threadId TEXT NOT NULL,
  userAccount INTEGER NOT NULL,
  date INTEGER NOT NULL,
  folder TEXT NOT NULL,
  sender TEXT NOT NULL,
  -- ... other columns
);

-- Strategic indexes for fast queries
CREATE INDEX idx_messages_threadId ON messages(threadId);
CREATE INDEX idx_messages_userAccount_folder_date
  ON messages(userAccount, folder, date DESC);

The magic wasn't just SQL—it was the query planner using these indexes efficiently.

Persistence Strategy

We use the Origin Private File System (OPFS) for file-like access. Data survives browser restarts, syncs efficiently, and feels native.

// Export for backup
const file = await PoolUtil.exportFile('/mail.db');

// Import for restore
await PoolUtil.importDb('/mail.db', uint8Array);

Why SAH Pool (Synchronous Access Handler Pool)?

When choosing a VFS (Virtual File System) for SQLite WASM, we selected SAH Pool (Synchronous Access Handler Pool). Here's why it works for our use case and the tradeoffs we accepted.

What is SAH Pool?

SAH Pool provides synchronous, high-performance access to the Origin Private File System. It uses a pool of worker threads to handle database operations, giving you near-native SQLite performance while maintaining thread safety. The pool architecture ensures efficient resource utilization and prevents worker thread exhaustion.

Pros of SAH Pool:

• Maximum performance: Synchronous-like access with minimal overhead
• Thread safety: Built-in concurrency management with worker pools
• Persistence: Data survives browser restarts via OPFS
• Efficient writes: Batched operations minimize I/O overhead
• Backup/restore: Built-in export/import functionality for database management

Cons of SAH Pool:

• Single connection limitation: Only one database connection can be active at a time per origin
• No multi-tab support: Can't have the app open in multiple browser tabs simultaneously
• OPFS requirement: Requires browser support for OPFS (Chrome 102+, Safari 15.2+, Firefox 111+)

Our Multi-Tab Solution:

The single-connection limitation means users can't run the app in multiple browser tabs at once. We handle this using BroadcastChannel to detect and manage multiple instances. When a second tab opens, we show a friendly message with options to switch tabs. If the user chooses to switch, we gracefully close the SQLite connection in the other tab and redirect them—ensuring a smooth handoff. This tradeoff is worth it for the performance gains SAH Pool provides.

The Results: Game Changing Performance

I can't overstate the difference this made. The first time I ran a query on 1 million messages and saw sub-second results, I was genuinely shocked.

Real-World Benchmarks

Testing with realistic mailbox sizes:

100,000 messages (email client territory):

• Before IndexedDB: Loading paginated view = 2-5 seconds
• After SQLite WASM: Loading paginated view = 50-100ms
• Improvement: 20-100× faster

1 million messages (stress test):

• Before: Wouldn't even load (crashed browser)
• After: Paginated queries in 100-200ms

The Performance Numbers

Based on official SQLite WASM benchmarks and our testing:

The gains are dramatic, but the real win is consistency. SQLite's query planner handles complex cases predictably.

What SQLite WASM Gives You

• Full SQL support: Joins, subqueries, window functions, JSON operators
• Proper indexing: Create multi-column indexes for specific query patterns
• Transactions: Consistent data updates
• Minimal memory: No need to load everything into memory
• Scalability: From thousands to millions of records

Lessons Learned: When Each Tool Makes Sense

Use SQLite WASM When:

• Relational data with complex querying
• Large datasets (100K+ records) with frequent reads
• Pagination and filtering are primary operations
• Strong consistency and transactions
• Advanced features like joins across tables

IndexedDB Is Still Fine For:

• Simple key-value caching
• Large binary objects (images, videos)
• Minimal query requirements
• Quick prototyping

Browser Support Reality Check

• Modern Chromium (Chrome 109+, Edge 109+)
• Modern Safari (16.4+)
• Firefox (69+)
• Mobile Safari and Chrome

This covers more than 95% of users. For an email client targeting modern browsers, it’s not a blocker.

The Impact: Features We Could Finally Build

With IndexedDB, we had to choose features based on performance. With SQLite WASM, possibilities opened up:

1. Advanced search: Full-text across multiple fields
2. Complex filters: Union/intersection of multiple criteria
3. Thread grouping: Fast queries for related messages
4. Analytics: Aggregations over large datasets
5. Real-time sync: Fast writes without blocking UI

The most important change: we stopped compromising.

The Truth About Bundle Size

@sqlite.org/sqlite-wasm adds ~3MB to your bundle. In an email client, this is a small cost for faster queries and richer features. For apps where every KB matters, lazy-load the SQLite worker after the initial app load.

Conclusion

If you're working with large datasets client-side, SQLite WASM is a strong choice. It shifted our email client from sluggish at scale to smooth at over 1 million messages.

The migration involved real work, but the results made it worth it. I can’t imagine building this without it.

The takeaway: When IndexedDB is the bottleneck, SQLite WASM is the path forward.

---