Architecture

Overview

Wormholes combines several open source components as building blocks.

• PostgreSQL is a relational database that stores links and users data.
• TimescaleDB is a time-series database that is used to store historical click events for links.
• Redis is an in-memory cache that we use as MRU Cache when storing or retrieving links from PostgreSQL.

Generator

It is responsible for generating collision-free IDs. It contains following components —

1. Bloom - It's a thread-safe wrapper around bloom. The bloom-filter is populated with all existing IDs on startup.
2. Bucket - It's a thread-safe wrapper around a slice of given capacity. It is responsible for storing generated IDs. A bucket can have two states — empty and full.
3. Store - It's an in-memory store containing a slice of buckets and a channel. When a bucket it popped, the channel receives index of that bucket.
4. Factory - It's responsible for generating IDs. when store channel receives an index for empty Bucket, the factory runs a goroutine to populate that bucket.

In distributed mode, Generator runs with a gRPC server and exposes a GetBucket method to retrieve one bucket at a time.

Creator

Creator is a REST server using Fiber that is responsible for creating, updating, deleting and reading links data. Upon startup and before all of the IDs in a Bucket are used, the creator instance retrieves a Bucket full of IDs from Generator.

Instead of adding generated links immediately to database, It uses a queue to batch-process several links at a time.

Redirector

Redirector is a REST server that powers link redirection. It also collects all the necessary metrics for each click and sends those to TimescaleDB. This is done with a Pipe containing multiple streams.

Analyzer

Analyzer is a REST server that provides all the stats related to links by running analytical queries against TimescaleDB.