(Under development, not ready for public consumption yet. please move on..)
GoSiege
A cluster based distributed load generator based on siege stress testing tool. Creates a
siege cluster that can distribute it's load and target a web server for stress. Can scale
linearly based on the number of nodes in the cluster. Is fault-tolerant and can adjust
load dynamically based on complete node failures or nodes with reduced capacity.
Provides UI to administer the load and dynamically scale up or down the stress limits.
Can also run on a single node.
Design
Terminology
GoSiege Cluster - A set of nodes configured to generate load against one or more targets.
Siege Node - An individual machine(or container) with gosiege installed.
Session - A single load testing session with a set of nodes and targets.
GoSiege state - Current state of the GoSiege cluster arranged as topics. Contains
current node and session information.
Session State - Set of information pertaining to a session.
Components
- GoSiege service daemon
Main daemon that spins up the siege tool. Is made of a set of goroutines(threads) that
do a set of operations. Also listens to local http port for incoming commands for
administration. - Admin Web UI
Administration UI running on NodeJS that provides interface to Add, Remove or Update
Siege sessions. - gosiege command line tool
A command line tool that can do all that the Admin UI can do.
Design Decisions
Initial versions will be provided as Docker containers. Each docker container will
act as a siege node.
Architecture Pattern
- No component talks to each other directly. Uses channels to communicate.
- GoSiegeState is the king here. Every component reacts to and updates state.
- State is stored in a distributed key-value store(e.g. etcd)
- Components don't talk to each other, but change the state.
- Components react to changes in the state
- Each component subscribes to topics in the state and listen for state change
over a channel notifications.
- Each component is a go routine that subscribes to messages to the state engine and listen on a channel
TODO: Using etcd has issues with having to get a token from internet for each new
cluster instantiation. This may not fly for a non-internet connected machines.
i.e. isolated networks etc.,
Administrative Operations
Admin operations can be done using either the Admin UI or gosiege command line tool.
- GoSiege Cluster
- GoSiege Session
- Create a new session
- Stop a session
- Update a session with
- New requests
- New requests per second
- New target information
Data Structures
- Configuration
- Session State Key Value Pairs
- Commands issued from the web UI
Runtime Routine Design
gosiege, the main program, spins up a 'siege' command for each session configured for the session.
Main spins up these go routines that does the following:
- GoSiegeState Handler
- Provides interfaces for reading or changing GoSiegeState
- Monitors GoSiegeState for updates and notifies components that subscribed to topics
- Accepts subscriptions for topics
- Session handler
Load distribution
When there are more than one node in the GoSiege cluster, the load needs to be distributed across all equally. Below are couple options:
- A leader elected within the group periodically decides how much each node should generate.
- Each node gets around the network table, every 5 seconds, and decide how much each
should generate to achieve the target.
Option two is better since there is no need for electing a leader maintaining the same.
Whoever talks first is the leader in each iteration.
Load consensus protocol
Each node gets around the network table, every 5 seconds, and agrees upon how much each
should generate to achieve the target. This is how the dialogue goes, if you evesdrop on them.
Lets say there are 3 nodes NodeA, NodeB and NodeC in a GoSiege Cluster and target requests/sec
is 5000. The GoSiege Session is configured by the manager and is just starting.
They enter a room with a white board and one marker, figure out we are evesdropping
so they don't talk but write on the board to communicate. Each gets one chance with
the marker.
(Node A gets marker first and writes)
Node A : OK, target is 5000. Am big and strong, I can do 4000 of it, writes
Node A: 4000, Can do 4000
Total: 4000
(Node C gets the marker next)
Node C : I'm bigger and stronger. I can do all 5000, but since Node B is playing this round,
lets distribute. I will do 5k*5k/(4k+5k) = 2778, Node B does 5k*4k/(4k+5k) = 2222. Total: 5k
Node A: 2778, Can do 4000
Node B: 2222, Can do 5000
Total: 5000
(Node B gets the marker next)
Node B : I'm not as big as you folks, can do only 3000. But I'm old and wise, and I got the last
turn and what I write goes. I will distribute equally per our strengths. So here is what I write.
Node A: 1666
Node B: 2084
Node C: 1250
Total: 5000
Partition Issues and Quorum
Issue arise if there is a network disconnection and only Node A and B make it to the table,
and Node C couldn't. In this scenario A and B will be on one table and C will get
it's own table. This will end up sending 5k traffic from A and B and C will send 3k,
making the server smoke and burn.
So we introduce Quorum in each GoSiege Cluster. Quorum, as you would know, is a minimum
number of votes that should be obtained to be operational.
In the above GoSiege cluster, the quorum should be set to 2. In this instance of
Node C will not proceed with sending traffic since it has only one vote on it's decision. In the case of generating load using just one node, Quorum should be set to 1 so that the node does not stop the load.
As you would have guessed, whiteboard here is the Session State, that gets updated by
each node.
Dependencies
Docker - Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating-system-level virtualization on Linux.
Siege - Siege is an http load testing and benchmarking utility. It was designed to let web developers measure their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP, HTTPS and FTP protocols. It lets its user hit a server with a configurable number of simulated clients. Those clients place the server “under siege.”. Taken from siege home page.
etcd - A distributed key-value store daemon from CoreOS written in Go.
NodeJS - Runs admin ui.
Code Organization
main
Initializes all the components and starts them up. Has common components like
config, logging etc.,
state
Distributed state maintainer. Has a set of plugins to use different distributed
key-value store providers: etcd, local file(if not for cluster) etc.,
manager/session
Session instances that gossip across the cluster and share load generation. Each
GoSiege session will have a goroutine associated with it on each node maintaining the
same.
- Each session has a session ID
manager/cluster
Administers the GoSiege cluster. Is a set of go routines that listens to commands from
Admin UI or command line and does CRUD operations on the cluster.
REST API