In our Distributed Systems course (ECE419) at UofT, two peers and I collaborated on developing a comprehensive key-value storage system. The system comprises various components, including an External Coordination Service (ECS), Servers (KVServers), Clients (KVClients), and an Authentication Server (AuthServer).

The ECS manages the entire system and deploys a ring of KVServers, as depicted in the diagram above. The ECS and KVServers communicate with each other using ZooKeeper, where metadata and other information are stored. Data is distributed across the servers using consistent hashing, with each server responsible for an assigned range of the hash ring. Additionally, each server holds two replicas of other servers in the ring. Data is replicated eagerly upon addition to the system. When servers are added or removed, the ring redistributes data efficiently. The ECS also employs failure detection to detect server failures, triggering the relaunch of a new server whenever a failure occurs.

Prior to connecting to the server ring, KVClients must authenticate with the AuthServer. The client uses credentials to log in or create an account, and the AuthServer provides the client with a token if the credentials are valid. Once the client has a token, it can connect to the server ring to retrieve and store data. Servers ensure that connecting clients have valid tokens before performing any operations. Data is stored on the servers with the owner's token, ensuring that clients can only access and manipulate their own data. If clients connect to a server not responsible for their data, the server automatically redirects the client to the correct server.

The system is robust as it replicates all data to prevent data loss and corruption, handles and detects server failures, and employs authentication to protect against malicious users.