Teal.Finance/Garcon
| Garcon works with all HTTP routers ans middleware respecting the Go HTTP standards. Garcon provides the batteries: static website server, contact-form backend, API helpers, debugging helpers (PProf), Git version, exporter server (Prometheus), health endpoints (Kubernetes), URI sanitization and middleware: rate-limiter, JWT cookies, CORS, traffic logs, OPA… |
---|
Motivation
Many projects often start with one of the many nice HTTP routers and middleware already available and develop their own middleware, debugging server, API helpers... At Teal.Finance, we decided to share in one place (here) all our stuff in the idea to let other projects go faster.
Middleware
Our Middleware are very easy to setup. They respect the Go standards. Thus you can easily use them with the HTTP router of your choice and chained them with other middleware:
MiddlewareLogRequest
Log incoming requests (with or without browser fingerprint)MiddlewareLogDuration
Log processing timeMiddlewareExportTrafficMetrics
Export web traffic metricsMiddlewareRejectUnprintableURI
Reject request with unwanted charactersMiddlewareRateLimiter
Limit incoming request to prevent floodingMiddlewareServerHeader
Add the "Server" HTTP header in the responseJWTChecker
JWT management using HttpOnly cookie or Authorization headerIncorruptibleChecker
Session cookie with Incorruptible tokenMiddlewareCORS
Cross-Origin Resource Sharing (CORS)MiddlewareOPA
Authenticate from Datalog/Rego files using Open Policy AgentMiddlewareSecureHTTPHeader
Set some HTTP header to increase the web security
g := garcon.New()
middleware = garcon.NewChain(
g.MiddlewareRejectUnprintableURI(),
g.MiddlewareLogRequest(),
g.MiddlewareRateLimiter())
router := ... you choose
handler := middleware.Then(router)
server := http.Server{Addr: ":8080", Handler: handler}
server.ListenAndServe()
Other features
- Static web files server supporting Brotli and AVIF
- Metrics server exporting data to Prometheus (or other compatible monitoring tool)
- Health status server for Kubernetes liveness and readiness probes
- PProf server for debugging purpose
- Serialize JSON responses, including the error messages
- Chained middleware (fork of justinas/alice)
- Chained round trip handlers
- Retrieve Git version, branch and commit from build flags and Go module information
Basic example
g := garcon.New()
middleware = garcon.NewChain(
g.MiddlewareRejectUnprintableURI(),
g.MiddlewareLogRequests(),
g.MiddlewareRateLimiter())
router := chi.NewRouter()
ws := g.NewStaticWebServer("/var/www")
router.Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
router.Get("/favicon.ico", ws.ServeFile("favicon.ico", "image/x-icon"))
router.Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
router.Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
router.Get("/images/*", ws.ServeImages())
cf := g.NewContactForm("/")
router.Post("/", cf.Notify("https://mattermost.com/hooks/qite178czotd5"))
router.Get("/version", garcon.ServeVersion())
router.Get("/reserved", g.Writer.NotImplemented)
router.NotFound(g.Writer.InvalidPath)
handler := middleware.Then(router)
server := http.Server{Addr: ":8080", Handler: handler}
server.ListenAndServe()
Incorruptible middleware
Garcon uses the Incorruptible package
to create/verify session cookie.
package main
import "github.com/teal-finance/garcon"
func main() {
g, _ := garcon.New(
garcon.WithURLs("https://my-company.com"),
garcon.WithDev())
aes128Key = "00112233445566778899aabbccddeeff"
maxAge := 3600
setIP := true
ck := g.IncorruptibleChecker(aes128Key, maxAge, setIP)
router := chi.NewRouter()
ws := g.NewStaticWebServer("/var/www")
router.With(ck.Set).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
router.With(ck.Chk).Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
router.With(ck.Chk).Get("/assets/*", ws.ServeAssets())
router.With(ck.Vet).Post("/api/items", myFunctionHandler)
server := http.Server{Addr: ":8080", Handler: router}
server.ListenAndServe()
}
JWT middleware
The JWT and Incorruptible checkers share a common interface,
TokenChecker
, providing the same middleware: Set()
, Chk()
and Vet()
.
package main
import "github.com/teal-finance/garcon"
func main() {
g, _ := garcon.New(
garcon.WithURLs("https://my-company.com"),
garcon.WithDev())
hmacSHA256Key := "9d2e0a02121179a3c3de1b035ae1355b1548781c8ce8538a1dc0853a12dfb13d"
ck := g.JWTChecker(hmacSHA256Key, "FreePlan", 10, "PremiumPlan", 100)
router := chi.NewRouter()
ws := g.NewStaticWebServer("/var/www")
router.With(ck.Set).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
router.With(ck.Chk).Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
router.With(ck.Chk).Get("/assets/*", ws.ServeAssets())
router.With(ck.Vet).Post("/api/items", myFunctionHandler)
server := http.Server{Addr: ":8080", Handler: router}
server.ListenAndServe()
}
Who use Garcon
In production, this library is used by
Rainbow,
Quid
and other internal projects at Teal.Finance.
Please propose a Pull Request to add here your project that also uses Garcon.
See a complete real example in the repo
github.com/teal-finance/rainbow.
CPU profiling
Moreover, Garcon simplifies investigation on CPU and memory consumption issues
thanks to https://github.com/pkg/profile.
In your code, add defer garcon.ProbeCPU.Stop()
that will write the cpu.pprof
file.
import "github.com/teal-finance/garcon"
func myFunctionConsumingLotsOfCPU() {
defer garcon.ProbeCPU.Stop()
}
Run pprof
and browse your cpu.pprof
file:
go run github.com/google/pprof@latest -http=: cpu.pprof
Complete example
See the complete example
enabling almost of the Garcon features.
Below is a simplified extract:
package main
import "github.com/teal-finance/garcon"
func main() {
defer garcon.ProbeCPU().Stop()
garcon.LogVersion()
garcon.SetVersionFlag()
jwt := flag.Bool("jwt", false, "Use JWT in lieu of the Incorruptible token")
flag.Parse()
g := garcon.New(
garcon.WithURLs("https://my-company.co"),
garcon.WithDocURL("/doc"),
garcon.WithPProf(8093))
ic := g.IncorruptibleChecker(aes128Key, 60, true)
jc := g.JWTChecker(hmacSHA256Key, "FreePlan", 10, "PremiumPlan", 100)
middleware, connState := g.StartExporter(9093)
middleware = middleware.Append(
g.MiddlewareRejectUnprintableURI(),
g.MiddlewareLogRequests("fingerprint"),
g.MiddlewareRateLimiter(10, 30),
g.MiddlewareServerHeader("MyApp"),
g.MiddlewareCORS(),
g.MiddlewareOPA("auth.rego"),
g.MiddlewareLogDuration(true()))
router := chi.NewRouter()
ws := g.NewStaticWebServer("/var/www")
router.With(ic.Set).With(jc.Set).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
router.With(ic.Chk).Get("/assets/*", ws.ServeAssets())
cf := g.NewContactForm("/")
router.Post("/", cf.Notify("https://mattermost.com/hooks/qite178czotd5"))
router.With(jc.Vet).Post("/api/items", myFunctionHandler)
handler := middleware.Then(router)
server := garcon.Server(handler, 8080, connState)
garcon.ListenAndServe(&server)
}
cd garcon
go run -race ./examples/complete
garcon ℹ️ Probing CPU. To visualize the profile: pprof -http=: cpu.pprof
2022/09/14 18:43:05 profile: cpu profiling enabled, cpu.pprof
garcon 🎬 Version: devel
garcon ℹ️ Enable PProf endpoints: http://localhost:8093/debug/pprof
incorr 🔒 DevMode accepts missing/invalid token from http://localhost:8080/myapp
incorr 🔒 cookie myapp Domain=localhost Path=/myapp Max-Age=60 Secure=false SameSite=3 HttpOnly=true Value=0 bytes
garcon ℹ️ Prometheus export http://localhost:9093 namespace=myapp
garcon 🔒 CORS Allow origin prefixes: [http://localhost:8080 http://localhost: http://192.168.1.]
garcon 🔒 CORS Methods: [GET POST DELETE]
garcon 🔒 CORS Headers: [Origin Content-Type Authorization]
garcon 🔒 CORS Credentials=true MaxAge=86400
incorr 🔒 Middleware Incorruptible.Set cookie "myapp" MaxAge=60 setIP=true
incorr 🔒 Middleware Incorruptible.Set cookie "myapp" MaxAge=60 setIP=true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
garcon ℹ️ Middleware WebForm redirects to http://localhost:8080/myapp
garcon ℹ️ empty URL => use the LogNotifier
incorr 🔒 Middleware Incorruptible.Set cookie "myapp" MaxAge=60 setIP=true
incorr 🔒 Middleware Incorruptible.Vet cookie/bearer DevMode= true
incorr 🔒 Middleware Incorruptible.Vet cookie/bearer DevMode= true
incorr 🔒 Middleware Incorruptible.Vet cookie/bearer DevMode= true
garcon ℹ️ MiddlewareLogDurationSafe: logs requester IP, sanitized URL and duration
garcon ℹ️ MiddlewareServerHeader sets the HTTP header Server=MyApp-devel in the responses
garcon ℹ️ MiddlewareRateLimiter burst=40 rate=2.67/s
garcon ℹ️ MiddlewareLogFingerprint:
1. Accept-Language, the language preferred by the user.
2. User-Agent, name and version of the browser and OS.
3. R=Referer, the website from which the request originated.
4. A=Accept, the content types the browser prefers.
5. E=Accept-Encoding, the compression formats the browser supports.
6. Connection, can be empty, "keep-alive" or "close".
7. Cache-Control, how the browser is caching data.
8. URI=Upgrade-Insecure-Requests, the browser can upgrade from HTTP to HTTPS.
9. Via avoids request loops and identifies protocol capabilities.
10. Authorization or Cookie (both should not be present at the same time).
11. DNT (Do Not Track) is being dropped by web browsers.
garcon ℹ️ MiddlewareRejectUnprintableURI rejects URI having line breaks or unprintable characters
app 🎬 -------------- Open http://localhost:8080/myapp --------------
garcon 📰 Server listening on http://localhost:8080
2. Embedded PProf server
Visit the PProf server at http://localhost:8093/debug/pprof providing the following endpoints:
PProf is easy to use with curl
or wget
:
( cd ~ ; go get -u github.com/google/pprof )
curl http://localhost:8093/debug/pprof/allocs > allocs.pprof
pprof -http=: allocs.pprof
wget http://localhost:8093/debug/pprof/heap
pprof -http=: heap
wget http://localhost:8093/debug/pprof/goroutine
pprof -http=: goroutine
See the PProf post (2013) for further explanations.
3. Exporter server
To facilitate the prod management, metrics and health state are communicated.
Tools like Prometheus and Kubernetes collect every N seconds this information depending on the endpoint:
Kubernetes uses that readiness status to orchestrate the deployment.
The default update policy is to update one pod at a time:
Kubernetes waits for the new pod to be ready to receive traffic before updating the next one.
4. Static website server
:warning: WARNING: This section is outdated! :warning:
The complete example is running.
Open http://localhost:8080/myapp with your browser, and play with the API endpoints.
The resources and API endpoints are protected with a HttpOnly cookie.
The complete example sets the cookie to browsers visiting the index.html
.
func handler(gw garcon.Writer, jc *jwtperm.Checker) http.Handler {
r := chi.NewRouter()
ws := garcon.WebServer{Dir: "examples/www", Writer: gw}
r.With(jc.SetCookie).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
r.With(jc.SetCookie).Get("/favicon.ico", ws.ServeFile("favicon.ico", "image/x-icon"))
r.With(jc.ChkCookie).Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
r.With(jc.ChkCookie).Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
r.With(jc.ChkCookie).Get("/images/*", ws.ServeImages())
r.With(jc.ChkJWT).Get("/api/v1/items", items)
r.With(jc.ChkJWT).Get("/api/v1/ducks", gw.NotImplemented)
r.NotFound(gw.InvalidPath)
return r
}
5. Enable Authentication
:warning: WARNING: This section is outdated! :warning:
Restart again the complete example with authentication enabled.
Attention, in this example we use two redundant middleware pieces using the same JWT: jwtperm
and opa
.
This is just an example, don't be confused.
go run -race ./examples/complete -auth
2021/12/02 08:09:47 Prometheus export http://localhost:9093
2021/12/02 08:09:47 CORS: Set origin prefixes: [http://localhost:8080 http://localhost: http://192.168.1.]
2021/12/02 08:09:47 CORS: Methods=[GET] Headers=[Origin Accept Content-Type Authorization Cookie] Credentials=true MaxAge=86400
2021/12/02 08:09:47 JWT not required for dev. origins: [http://localhost:8080 http://localhost: http://192.168.1.]
2021/12/02 08:09:47 Enable PProf endpoints: http://localhost:8093/debug/pprof
2021/12/02 08:09:47 Create cookie plan=FreePlan domain=localhost secure=false jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ5ODd9.5tJk2NoHxkG0o_owtMleBcUaR8z1vRx4rxRRqtZUc_Q
2021/12/02 08:09:47 Create cookie plan=PremiumPlan domain=localhost secure=false jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiUHJlbWl1bVBsYW4iLCJleHAiOjE2Njk5NjQ5ODd9.ifKhbmxQQ64NweL5aQDb_42tvKHwqiEKD-vxHO3KzsM
2021/12/02 08:09:47 OPA: load "examples/sample-auth.rego"
2021/12/02 08:09:47 Middleware OPA: map[sample-auth.rego:package auth
default allow = false
tokens := {"Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y"} { true }
allow = true { __local0__ = input.token; data.auth.tokens[__local0__] }]
2021/12/02 08:09:47 Middleware response HTTP header: Set Server MyBackendName-1.2.0
2021/12/02 08:09:47 MiddlewareRateLimiter burst=100 rate=5/s
2021/12/02 08:09:47 Middleware logger: requester IP and requested URL
2021/12/02 08:09:47 Server listening on http://localhost:8080
Test the API with curl
:
curl -D - http://localhost:8080/myapp/api/v1/items
HTTP/1.1 401 Unauthorized
Content-Type: application/json
Server: MyBackendName-1.2.0
Vary: Origin
Date: Thu, 02 Dec 2021 07:06:20 GMT
Content-Length: 84
{"error":"Unauthorized",
"path":"/api/v1/items",
"doc":"http://localhost:8080/myapp/doc"}
The corresponding garcon logs:
2021/12/02 08:06:20 in 127.0.0.1:42888 GET /api/v1/items
[cors] 2021/12/02 08:06:20 Handler: Actual request
[cors] 2021/12/02 08:06:20 Actual request no headers added: missing origin
2021/12/02 08:06:20 OPA unauthorize 127.0.0.1:42888 /api/v1/items
2021/12/02 08:06:20 out 127.0.0.1:42888 GET /api/v1/items 1.426916ms c=1
The CORS logs can be disabled by passing debug=false
in cors.Handler(origins, false)
.
The value c=1
measures the web traffic (current active HTTP connections).
curl -D - http://localhost:8080/myapp/api/v1/items -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y'
HTTP/1.1 200 OK
Content-Type: application/json
Server: MyBackendName-1.2.0
Vary: Origin
Date: Thu, 02 Dec 2021 07:10:37 GMT
Content-Length: 25
["item1","item2","item3"]
The corresponding garcon logs:
2021/12/02 08:10:37 in 127.0.0.1:42892 GET /api/v1/items
[cors] 2021/12/02 08:10:37 Handler: Actual request
[cors] 2021/12/02 08:10:37 Actual request no headers added: missing origin
2021/12/02 08:10:37 Authorization header has JWT: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y
2021/12/02 08:10:37 JWT Claims: {FreePlan { [] 2022-12-02 08:01:28 +0100 CET <nil> <nil> invalid cookie}}
2021/12/02 08:10:37 JWT has the FreePlan Namespace
2021/12/02 08:10:37 JWT Permission: {10}
2021/12/02 08:10:37 out 127.0.0.1:42892 GET /api/v1/items 1.984568ms c=1 a=1 i=0 h=0
Low-level example
:warning: WARNING: This section is outdated! :warning:
See the low-level example.
The following code is a bit different to the stuff done
by the complete function Garcon.Run()
presented in the previous chapter.
The following code is intended to show
Garcon can be customized to meet your specific requirements.
package main
import (
"log"
"net"
"net/http"
"time"
"github.com/go-chi/chi/v5"
"github.com/teal-finance/garcon"
)
const apiDoc = "https://my-dns.co/doc"
const allowedProdOrigin = "https://my-dns.co"
const allowedDevOrigins = "http://localhost: http://192.168.1."
const serverHeader = "MyBackendName-1.2.0"
const authCfg = "examples/sample-auth.rego"
const pprofPort = 8093
const expPort = 9093
const burst, reqMinute = 10, 30
const devMode = true
func main() {
if devMode {
defer garcon.ProbeCPU().Stop()
}
garcon.StartPProfServer(pprofPort)
gw := garcon.NewWriter(apiDoc)
middleware, connState := setMiddlewares(gw)
h := handler(gw)
h = middleware.Then(h)
runServer(h, connState)
}
func setMiddlewares(gw garcon.Writer) (middleware garcon.Chain, connState func(net.Conn, http.ConnState)) {
middleware, connState = garcon.StartExporter(expPort, devMode)
reqLimiter := garcon.NewReqLimiter(gw, burst, reqMinute, devMode)
corsConfig := allowedProdOrigin
if devMode {
corsConfig += " " + allowedDevOrigins
}
allowedOrigins := garcon.SplitClean(corsConfig)
middleware = middleware.Append(
reqLimiter.Limit,
garcon.ServerHeader(serverHeader),
cors.Handler(allowedOrigins, devMode),
)
files := garcon.SplitClean(authCfg)
policy, err := garcon.NewPolicy(gw, files)
if err != nil {
log.Fatal(err)
}
if policy.Ready() {
middleware = middleware.Append(policy.Auth)
}
return middleware, connState
}
func runServer(h http.Handler, connState func(net.Conn, http.ConnState)) {
const mainPort = "8080"
server := http.Server{
Addr: ":" + mainPort,
Handler: h,
TLSConfig: nil,
ReadTimeout: 1 * time.Second,
ReadHeaderTimeout: 1 * time.Second,
WriteTimeout: 1 * time.Second,
IdleTimeout: 1 * time.Second,
MaxHeaderBytes: 222,
TLSNextProto: nil,
ConnState: connState,
ErrorLog: log.Default(),
BaseContext: nil,
ConnContext: nil,
}
log.Print("Server listening on http://localhost", server.Addr)
log.Fatal(server.ListenAndServe())
}
func handler(gw garcon.Writer) http.Handler {
r := chi.NewRouter()
ws := g.NewStaticWebServer("/var/www")
r.Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
r.Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
r.Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
r.Get("/images/*", ws.ServeImages())
r.Get("/api/v1/items", items)
r.Get("/api/v1/ducks", gw.NotImplemented)
r.NotFound(gw.InvalidPath)
return r
}
func items(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`["item1","item2","item3"]`))
}
KeyStore example
The example KeyStore implements a key/value datastore
providing private storage for each client identified by its unique IP.
cd garcon
go run -race ./examples/keystore
Then open http://localhost:8080 to learn more about the implemented features.
✨ Contributions Welcome
This project needs your help to become better.
Please propose your enhancements,
or even a further refactoring.
We welcome contributions in many forms,
and there's always plenty to do!
🗣️ Feedback
If you have some suggestions, or need a new feature,
please contact us, using the
issues,
or at Teal.Finance@pm.me or
@TealFinance.
Feel free to propose a
Pull Request,
your contributions are welcome. :wink:
🗽 Copyright and license
Copyright (c) 2021 Teal.Finance/Garcon contributors
Teal.Finance/Garcon is free software, and can be redistributed
and/or modified under the terms of the MIT License.
SPDX-License-Identifier: MIT
Teal.Finance/Garcon is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the LICENSE file (alongside the source files)
or https://opensource.org/licenses/MIT.
See also