github.com/the-benchmarker/web-frameworks

Which is the fastest?

Measuring response times (routing times) for each framework (middleware). Each framework has to have two features; routing and parsing path parameters.

Result

Last update: 2018-04-10

OS: Darwin (version: 17.3.0, arch: x86_64)
CPU Cores: 8

Ranking (Framework)

1. router_cr (crystal)
2. iron (rust)
3. raze (crystal)
4. nickel (rust)
5. japronto (python)
6. actix (rust)
7. spider-gazelle (crystal)
8. fasthttprouter (go)
9. mofuw (nim)
10. lucky (crystal)
11. amber (crystal)
12. kemal (crystal)
13. rocket (rust)
14. clusterpolka (node)
15. iris (go)
16. gorilla_mux (go)
17. echo (go)
18. phoenix (elixir)
19. gin (go)
20. plug (elixir)
21. aspnetcore (csharp)
22. vapor (swift)
23. sanic (python)
24. perfect (swift)
25. kitura (swift)
26. jester (nim)
27. polka (node)
28. clusterexpress (node)
29. akkahttp (scala)
30. express (node)
31. roda (ruby)
32. rack-routing (ruby)
33. criollo (objc)
34. sinatra (ruby)
35. tornado (python)
36. rails (ruby)

Ranking (Language)

1. crystal (router_cr)
2. rust (iron)
3. python (japronto)
4. go (fasthttprouter)
5. nim (mofuw)
6. node (clusterpolka)
7. elixir (phoenix)
8. csharp (aspnetcore)
9. swift (vapor)
10. scala (akkahttp)
11. ruby (roda)
12. objc (criollo)

All frameworks

Language (Runtime)	Framework (Middleware)	Max [sec]	Min [sec]	Ave [sec]
ruby	rails	350.161733	349.403240	349.758695
ruby	sinatra	100.885575	100.654648	100.776388
ruby	roda	35.200353	34.684047	34.891702
ruby	rack-routing	45.339112	44.963142	45.080912
crystal	kemal	8.239321	7.847715	8.080067
crystal	router_cr	6.643287	6.410330	6.555974
crystal	raze	6.931865	6.664825	6.848396
crystal	lucky	8.145836	7.735874	8.010866
crystal	amber	8.090061	7.831198	8.011325
crystal	spider-gazelle	7.240015	7.046256	7.192485
go	echo	10.205904	9.653988	9.989534
go	gorilla_mux	9.971818	9.486147	9.743084
go	iris	9.732393	9.387682	9.565645
go	fasthttprouter	7.592036	7.326633	7.452470
go	gin	10.973279	10.560217	10.811344
rust	actix	7.199203	6.997528	7.099141
rust	iron	6.801253	6.679035	6.710507
rust	nickel	6.936677	6.786194	6.876099
rust	rocket	8.657137	8.349306	8.504091
node	express	28.081658	26.611051	27.016107
node	clusterexpress	17.409648	15.773684	16.309491
node	polka	15.867429	15.066904	15.401819
node	clusterpolka	9.363300	9.197916	9.309864
elixir	plug	11.253845	10.597757	10.876218
elixir	phoenix	11.220435	10.087917	10.668334
swift	vapor	12.498265	12.178270	12.336104
swift	perfect	12.900839	12.444516	12.684239
swift	kitura	13.263198	12.493619	12.940476
scala	akkahttp	18.595948	16.553332	17.078369
csharp	aspnetcore	10.932268	10.829186	10.887798
python	sanic	12.986084	11.706889	12.437696
python	japronto	7.116580	7.002742	7.055527
python	tornado	176.956387	176.746874	176.855119
nim	jester	14.122242	14.050893	14.082330
nim	mofuw	8.012292	7.701051	7.814884
objc	criollo	57.233161	55.853540	56.561862

Current target frameworks (middlewares)

• Ruby
  • Rails
  • Sinatra
  • Roda
  • Rack-Routing
• Crystal
  • Kemal
  • raze
  • router.cr
• Go
  • Echo
  • gorilla-mux
  • iris
  • fasthttprouter
  • Gin
• Rust
  • IRON
  • nickel.rs
  • Rocket (nightly)
• node
  • express
  • express/cluster
  • polka
  • polka/cluster
• Elixir
  • Plug
  • Phoenix
• Swift
  • Vapor
  • Perfect
  • Kitura
• Scala
  • Akka-Http (Routing DSL)
• C#
  • ASP.NET Core
• Python
  • sanic
  • japronto
  • flask
  • tornado
• Nim
  • Jester
  • mofuw
• Objective-C
  • Criollo

See Development section when you want to add new languages or frameworks.

The rule

We want to know the response time (routing time), not a usability. So full-stack framework is at a disadvantage.

• Each server has no special logics.
• Each server's executable is named as server_[Lauguage]_[Framework]. (For example, server_ruby_sinatra)
• There are only 3 routes
  • GET '/' return status code 200 with empty body
  • GET '/user/:id' return status code 200 with the id
  • POST '/user' return status code 200 with empty body

Installation

Required environment -> See Current target frameworks(middlewares)

By using Neph

Neph is a modern command line job processor that can be substitute for make command.

To compile servers and benchmarker,

> neph

For each language,

> neph ruby

For each framework,

> neph rails

See neph.yaml

By using make

To compile servers and benchmarker,

> make

For each language,

> make ruby

For each framework,

> make rails

Usage

You can take a benchmark by

> bin/benchmarker

For each language

> bin/benchmarker ruby

For each framework

> bin/benchmarker rails

For comparison (Comparing rails, kemal and router.cr in this example)

> bin/benchmarker rails crystal

If you take it manually, you can run each server by

> bin/server_[Language]_[Framework]

and run client by

> time bin/client

You can set # of threads and # of the loops of the request(there are 3 requests in a loop) by

> time bin/client -t 16 -r 1000

In the above example, 16 threads requests 1000 * 3 times. So 48000 requests are sent in total.

Using Docker

Setup servers by using docker is under WIP. Currently, crystal and ruby servers are supported. For example

docker-compose up rails

Then you can run your client by

time ./bin/client

Development

• Give me PR when you want to add other web frameworks
• Give me PR when you can tuning each framework (under the rule)

Where should I modify when adding new framework

• /[language]/[framework]/[codes] <- Project itself
• benchmarker/benchmarker.cr <- Adding it as a target to
• README.md <- Adding it as a target framework of the list
• Makefile
• neph.yaml (optional)

Anyway, you don't have to care about details since maintainer can fix them after merging it. The result will be updated by maintainer.

Contributing

1. Fork it (https://github.com/tbrand/which_is_the_fastest/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• tbrand Taichiro Suzuki - creator, maintainer
• OvermindDL1 OvermindDL1 - maintainer
• waghanza Marwan Rabbâa - maintainer