github.com/addi-11/system-design-excercises

Sytem Design Excercises

To run any excercise, go the folder and run the following, Additional details added in the excercise folder:

go mod init example.com/main
go mod tidy
go run .

Week-1

• Implement a simple connection pool using Bounded Blocking Queue
• Implement Database Sharding and Routing (from API server)
• Setup Read-replica from a MySQL locally
• Implement fair multi-threaded program
• Implement server-sent events
• Implement server-sent events using Message Broker
• Implement server-sent events on React Components on a web-page.
• Setup RabbitMQ and Kafka locally.Write producer and consumer for them.
  • Setup RabbitMQ
  • Setup Kafka
• Implement real-time chat using socket IO: Slack-Realtime Text Chat Reference
• Mock EC2 creation & implement Short Polling and Long Polling

Week-2

• Implement Airline Check-in System
• Deadlock simulation
• Implement a toy KV store on top of MySQL
• Implement simple sharding with a hash or range based routing strategy in above KV store
• Implement flag driven consistent reads.
• Implement Distributed Transactions using 2PC.
• Ingest data in Neo4j and try paginating it.
• Ingest data in MongDB and write aggregation pipeline.
• Implement Message Broadcast across servers using Star Topology leveraging Redis PubSub.

Week-3

• Implement a load-balancer
• Implement a simple blogging application where you shard by user id; and try to provide a unique ID to each blog. The idea is to understand the need to ID generation when database is sharded.
• Build a simple atomically incrementing integer ID
• Implement the "Amazon's Way" of central ID generation service
• Implement ths sturcutre of MongoDB Object ID
• Benchmark the impact of UUID on relational database as Primary Key
• Benchmark MySQL's UPSERT using ON DUPLICATE KEY UPDATE and REPLACE INTO
• Implement Flickr's Odd-Even based ID generation
• Implement Snowflake on
  • API, and
  • Database as stored procedure
• Benchmark Pagination approaches.
• Implement Zomato Ordering Service using Distributed Transactions using 2PC

Week-4

• Implement a Toy CDN
• Mimick CDN Failover - on Toy CDN
• Implement pre-signed URL based upload on S3
• Configure CDN to serve Popular Searches JSON response
• Implement JWT based auhthentication
• Build GitHub like OG image and server it via CDN
  • Key learning: generating images in backend server and putting it behind a CDN
• Measure the impact of denormalization
  • Define a user collection in MongoDB with blogs as its attribute
  • Store blogs object in the user document demonting all blogs that a person wrote.
  • Store the entire object intead of reference.
  • Now benchmark and find out how slow the response times gets as we increase the number of elements in the blogs array
• Implement Lazy Loading of images on frontend
• Implement 5 approaches to count post per hashtag & STOP THE WORLD
• Populate on_msg_event while using websocket.
  • Try to identify when the connection breaks and use that opportunity to write event to Kafka
• Configure Redis in cluster mode and figure out how data is distrubuted
• Implement newly unread message indicator on database
  • Compute on the fly
  • Creates messages table with 1 million rows
  • Add one indexes for each column part of the where clause that is queried and measure the time taken
  • Compute with mentioned composite indexes, and measure the performance
  • Re-arrange the columns and mesure the performance impact

Week-5

• Implement Consistent Hashing
• Implement consistent hashing as a load balancer algorithm
• Solve skewness problem in consistent hashing with Virtual Nodes
• Implement a simple in-memory single-node cache like Redis as discussed in the session
• Implement the word dictionary on local machine
  • using CSV format
  • using Bitcask format
• Partial data write problem by writing 100mb file and killing the process in between
• Implement Checksum based WAL or bitcast
• Implement Checksum with DB recovery, as discussed in session.
• Implement Bitcask
  • Basic KV operations
  • Merge and compaction
• Benchmark sequential IO vs random IO

Week-6

• Implement B+ Trees
• LSM Tree Based Key-Value Store.
• Implement Bloom Filters and measure: FPR vs Size Vs Num of Hash Func
• Implement Deletable Bloom Filters
• Setup HLS Streaming following Akamai’s Documentation
• Video HLS Streaming Server in Go
• Implement a TCP server that accepts 1GB file
• Transfer the file via one POST request
• Stream the file from client to server from scratch
• Implement GFS

Week-7

• Implement recent search as discussed during the session
• Capture search logs and make them queryable
  • From an HTTP request, extract all possible meta info
  • Ingest them in ES
  • Plot different graphs, segmenations, and gain insights using Kibana
• Implement Full Text Search on your phone contacts
  • implement fuzzy searching
  • implement spell correction
  • implement synonymic query expansion
  • add support for phonetic search
• Cache API responses on Akamai for very short duration
  • Option 1: Set TTL on Akamai Console
  • Option 2: Drive TTL using response headers from the origin
• Stream some dummy logs from local machine to S3
  • Query them using Athena
• Implement Task Scheduler as discussed in the session
  • Fixed Time Execution and Cron Schedule
  • Implement Job Puller
  • Make Jobs Puller Fault Tolerant
  • For your machine, compute Unit Tech Economics for Job Puller
  • Define a format that allows user to specify any task
  • Build capability to run it - Docker Images a simple solution but overskill for simple tasks
  • Induce failures in your scheduler and set up alerts if you breach SLA
• Implement Team Relabance feature in Task scheduler
  • Do it for Fault Tolerance
  • Do it if you want to auto scale
• Implement Brokers in all 3 flavours
  • SQS like broker using MySQL as backend
  • Kafka like broker using MySQL as backend
  • SQS like broker using Bitcask as backend
• Create an account on Razorpay and build simple payment system using their API
  • use their “Test Mode”
  • use Webhooks to receive Payment Notifications

Week-8

• Implement GeoHash
• Implement Zoom-in and Zoom-out using Trie based approach
• Evaluate the difference between EVAL and EVAL_RO command
  • Fire EVAL command on Replica and observe the output
  • Fire EVAL command on Replica using Redis SDK and observe
• Write Lua script and mimic matching algorithm
  • use factors like - vehicle type, rating, etc.
• Implement cursor based pagination on MySQL
• Benchmark ingestion throughput when using
  • Auto-inc as Primary key
  • UUID as primary key

Excercises that can be extended:

• Zomato Delivery System
• Airline Checkin System
• Load Balancer
• TODO: React loading using Server Sent Events
• GFS