github.com/teal-finance/garcon
 | Garcon works with all HTTP routers ans middleware respecting the Go HTTP standards. Garcon provides the batteries: static website server, contact-form backend, API helpers, debugging helpers (PProf), Git version, exporter server (Prometheus), health endpoints (Kubernetes), URI sanitization and middleware: rate-limiter, JWT cookies, CORS, traffic logs, OPA… |
|---|
Many projects often start with one of the many nice HTTP routers and middleware already available and develop their own middleware, debugging server, API helpers... At Teal.Finance, we decided to share in one place (here) all our stuff in the idea to let other projects go faster.
Our Middleware are very easy to setup. They respect the Go standards. Thus you can easily use them with the HTTP router of your choice and chained them with other middleware:
MiddlewareLogRequest Log incoming requests (with or without browser fingerprint)MiddlewareLogDuration Log processing timeMiddlewareExportTrafficMetrics Export web traffic metricsMiddlewareRejectUnprintableURI Reject request with unwanted charactersMiddlewareRateLimiter Limit incoming request to prevent floodingMiddlewareServerHeader Add the "Server" HTTP header in the responseJWTChecker JWT management using HttpOnly cookie or Authorization headerIncorruptibleChecker Session cookie with Incorruptible tokenMiddlewareCORS Cross-Origin Resource Sharing (CORS)MiddlewareOPA Authenticate from Datalog/Rego files using Open Policy AgentMiddlewareSecureHTTPHeader Set some HTTP header to increase the web securityg := garcon.New()
middleware = garcon.NewChain(
g.MiddlewareRejectUnprintableURI(),
g.MiddlewareLogRequest(),
g.MiddlewareRateLimiter())
router := ... you choose
handler := middleware.Then(router)
server := http.Server{Addr: ":8080", Handler: handler}
server.ListenAndServe()
g := garcon.New()
// chain some middleware
middleware = garcon.NewChain(
g.MiddlewareRejectUnprintableURI(),
g.MiddlewareLogRequests(),
g.MiddlewareRateLimiter())
// use the HTTP router library of your choice, here we use Chi
router := chi.NewRouter()
// static website, automatically sends the Brotli-compressed file if present and supported by the browser
ws := g.NewStaticWebServer("/var/www")
router.Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
router.Get("/favicon.ico", ws.ServeFile("favicon.ico", "image/x-icon"))
router.Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
router.Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
router.Get("/images/*", ws.ServeImages()) // automatically sends AVIF if present and supported by the browser
// receive contact-forms on your chat channel on the fly
cf := g.NewContactForm("/")
router.Post("/", cf.Notify("https://mattermost.com/hooks/qite178czotd5"))
// Git version and last commit date (HTML or JSON depending on the "Accept" header)
router.Get("/version", garcon.ServeVersion())
// return a JSON message
router.Get("/reserved", g.Writer.NotImplemented)
router.NotFound(g.Writer.InvalidPath)
handler := middleware.Then(router)
server := http.Server{Addr: ":8080", Handler: handler}
server.ListenAndServe()
Garcon uses the Incorruptible package to create/verify session cookie.
package main
import "github.com/teal-finance/garcon"
func main() {
g, _ := garcon.New(
garcon.WithURLs("https://my-company.com"),
garcon.WithDev())
aes128Key = "00112233445566778899aabbccddeeff"
maxAge := 3600
setIP := true
ck := g.IncorruptibleChecker(aes128Key, maxAge, setIP)
router := chi.NewRouter()
// website with static files directory
ws := g.NewStaticWebServer("/var/www")
// ck.Set => set the cookie when visiting /
router.With(ck.Set).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
// ck.Chk => reject request with invalid JWT cookie
router.With(ck.Chk).Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
router.With(ck.Chk).Get("/assets/*", ws.ServeAssets())
// ck.Vet => accepts valid JWT in either the cookie or the Authorization header
router.With(ck.Vet).Post("/api/items", myFunctionHandler)
server := http.Server{Addr: ":8080", Handler: router}
server.ListenAndServe()
}
The JWT and Incorruptible checkers share a common interface,
TokenChecker, providing the same middleware: Set(), Chk() and Vet().
package main
import "github.com/teal-finance/garcon"
func main() {
g, _ := garcon.New(
garcon.WithURLs("https://my-company.com"),
garcon.WithDev())
hmacSHA256Key := "9d2e0a02121179a3c3de1b035ae1355b1548781c8ce8538a1dc0853a12dfb13d"
ck := g.JWTChecker(hmacSHA256Key, "FreePlan", 10, "PremiumPlan", 100)
router := chi.NewRouter()
// website with static files directory
ws := g.NewStaticWebServer("/var/www")
// ck.Set => set the cookie when visiting /
router.With(ck.Set).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
// ck.Chk => reject request with invalid JWT cookie
router.With(ck.Chk).Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
router.With(ck.Chk).Get("/assets/*", ws.ServeAssets())
// ck.Vet => accepts valid JWT in either the cookie or the Authorization header
router.With(ck.Vet).Post("/api/items", myFunctionHandler)
server := http.Server{Addr: ":8080", Handler: router}
server.ListenAndServe()
}
In production, this library is used by Rainbow, Quid and other internal projects at Teal.Finance.
Please propose a Pull Request to add here your project that also uses Garcon.
See a complete real example in the repo github.com/teal-finance/rainbow.
Moreover, Garcon simplifies investigation on CPU and memory consumption issues thanks to https://github.com/pkg/profile.
In your code, add defer garcon.ProbeCPU.Stop() that will write the cpu.pprof file.
import "github.com/teal-finance/garcon"
func myFunctionConsumingLotsOfCPU() {
defer garcon.ProbeCPU.Stop()
// ... lots of sub-functions
}
Run pprof and browse your cpu.pprof file:
go run github.com/google/pprof@latest -http=: cpu.pprof
See the complete example enabling almost of the Garcon features. Below is a simplified extract:
package main
import "github.com/teal-finance/garcon"
func main() {
defer garcon.ProbeCPU().Stop() // collects the CPU-profile and writes it in the file "cpu.pprof"
garcon.LogVersion() // log the Git version
garcon.SetVersionFlag() // the -version flag prints the Git version
jwt := flag.Bool("jwt", false, "Use JWT in lieu of the Incorruptible token")
flag.Parse()
g := garcon.New(
garcon.WithURLs("https://my-company.co"),
garcon.WithDocURL("/doc"),
garcon.WithPProf(8093))
ic := g.IncorruptibleChecker(aes128Key, 60, true)
jc := g.JWTChecker(hmacSHA256Key, "FreePlan", 10, "PremiumPlan", 100)
middleware, connState := g.StartExporter(9093)
middleware = middleware.Append(
g.MiddlewareRejectUnprintableURI(),
g.MiddlewareLogRequests("fingerprint"),
g.MiddlewareRateLimiter(10, 30),
g.MiddlewareServerHeader("MyApp"),
g.MiddlewareCORS(),
g.MiddlewareOPA("auth.rego"),
g.MiddlewareLogDuration(true()))
router := chi.NewRouter()
// website with static files directory
ws := g.NewStaticWebServer("/var/www")
router.With(ic.Set).With(jc.Set).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
router.With(ic.Chk).Get("/assets/*", ws.ServeAssets())
// Contact-form
cf := g.NewContactForm("/")
router.Post("/", cf.Notify("https://mattermost.com/hooks/qite178czotd5"))
// API
router.With(jc.Vet).Post("/api/items", myFunctionHandler)
handler := middleware.Then(router)
server := garcon.Server(handler, 8080, connState)
garcon.ListenAndServe(&server)
}
cd garcon
go run -race ./examples/complete
garcon ℹ️ Probing CPU. To visualize the profile: pprof -http=: cpu.pprof
2022/09/14 18:43:05 profile: cpu profiling enabled, cpu.pprof
garcon 🎬 Version: devel
garcon ℹ️ Enable PProf endpoints: http://localhost:8093/debug/pprof
incorr 🔒 DevMode accepts missing/invalid token from http://localhost:8080/myapp
incorr 🔒 cookie myapp Domain=localhost Path=/myapp Max-Age=60 Secure=false SameSite=3 HttpOnly=true Value=0 bytes
garcon ℹ️ Prometheus export http://localhost:9093 namespace=myapp
garcon 🔒 CORS Allow origin prefixes: [http://localhost:8080 http://localhost: http://192.168.1.]
garcon 🔒 CORS Methods: [GET POST DELETE]
garcon 🔒 CORS Headers: [Origin Content-Type Authorization]
garcon 🔒 CORS Credentials=true MaxAge=86400
incorr 🔒 Middleware Incorruptible.Set cookie "myapp" MaxAge=60 setIP=true
incorr 🔒 Middleware Incorruptible.Set cookie "myapp" MaxAge=60 setIP=true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
incorr 🔒 Middleware Incorruptible.Chk cookie DevMode= true
garcon ℹ️ Middleware WebForm redirects to http://localhost:8080/myapp
garcon ℹ️ empty URL => use the LogNotifier
incorr 🔒 Middleware Incorruptible.Set cookie "myapp" MaxAge=60 setIP=true
incorr 🔒 Middleware Incorruptible.Vet cookie/bearer DevMode= true
incorr 🔒 Middleware Incorruptible.Vet cookie/bearer DevMode= true
incorr 🔒 Middleware Incorruptible.Vet cookie/bearer DevMode= true
garcon ℹ️ MiddlewareLogDurationSafe: logs requester IP, sanitized URL and duration
garcon ℹ️ MiddlewareServerHeader sets the HTTP header Server=MyApp-devel in the responses
garcon ℹ️ MiddlewareRateLimiter burst=40 rate=2.67/s
garcon ℹ️ MiddlewareLogFingerprint:
1. Accept-Language, the language preferred by the user.
2. User-Agent, name and version of the browser and OS.
3. R=Referer, the website from which the request originated.
4. A=Accept, the content types the browser prefers.
5. E=Accept-Encoding, the compression formats the browser supports.
6. Connection, can be empty, "keep-alive" or "close".
7. Cache-Control, how the browser is caching data.
8. URI=Upgrade-Insecure-Requests, the browser can upgrade from HTTP to HTTPS.
9. Via avoids request loops and identifies protocol capabilities.
10. Authorization or Cookie (both should not be present at the same time).
11. DNT (Do Not Track) is being dropped by web browsers.
garcon ℹ️ MiddlewareRejectUnprintableURI rejects URI having line breaks or unprintable characters
app 🎬 -------------- Open http://localhost:8080/myapp --------------
garcon 📰 Server listening on http://localhost:8080
Visit the PProf server at http://localhost:8093/debug/pprof providing the following endpoints:
PProf is easy to use with curl or wget:
( cd ~ ; go get -u github.com/google/pprof )
curl http://localhost:8093/debug/pprof/allocs > allocs.pprof
pprof -http=: allocs.pprof
wget http://localhost:8093/debug/pprof/heap
pprof -http=: heap
wget http://localhost:8093/debug/pprof/goroutine
pprof -http=: goroutine
See the PProf post (2013) for further explanations.
To facilitate the prod management, metrics and health state are communicated. Tools like Prometheus and Kubernetes collect every N seconds this information depending on the endpoint:
http://localhost:9093/metrics to communicate internal metrics
http://localhost:9093/health liveness probes
reponds "200 OK" if it is healthy.
Here healthy means the application is running
and can access to its dependencies (e.g. database),
the application does no need to be killed/restarted.
http://localhost:9093/ready readiness probes
reponds "200 OK" if it is ready to receive traffic.
Here ready to receive traffic means the application is healthy,
the initialization phase is completed
and valid requests do not result in errors.
Kubernetes uses that readiness status to orchestrate the deployment. The default update policy is to update one pod at a time: Kubernetes waits for the new pod to be ready to receive traffic before updating the next one.
:warning: WARNING: This section is outdated! :warning:
The complete example is running.
Open http://localhost:8080/myapp with your browser, and play with the API endpoints.
The resources and API endpoints are protected with a HttpOnly cookie.
The complete example sets the cookie to browsers visiting the index.html.
func handler(gw garcon.Writer, jc *jwtperm.Checker) http.Handler {
r := chi.NewRouter()
// Static website files
ws := garcon.WebServer{Dir: "examples/www", Writer: gw}
r.With(jc.SetCookie).Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
r.With(jc.SetCookie).Get("/favicon.ico", ws.ServeFile("favicon.ico", "image/x-icon"))
r.With(jc.ChkCookie).Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
r.With(jc.ChkCookie).Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
r.With(jc.ChkCookie).Get("/images/*", ws.ServeImages())
// API
r.With(jc.ChkJWT).Get("/api/v1/items", items)
r.With(jc.ChkJWT).Get("/api/v1/ducks", gw.NotImplemented)
// Other endpoints
r.NotFound(gw.InvalidPath)
return r
}
:warning: WARNING: This section is outdated! :warning:
Restart again the complete example with authentication enabled.
Attention, in this example we use two redundant middleware pieces using the same JWT: jwtperm and opa.
This is just an example, don't be confused.
go run -race ./examples/complete -auth
2021/12/02 08:09:47 Prometheus export http://localhost:9093
2021/12/02 08:09:47 CORS: Set origin prefixes: [http://localhost:8080 http://localhost: http://192.168.1.]
2021/12/02 08:09:47 CORS: Methods=[GET] Headers=[Origin Accept Content-Type Authorization Cookie] Credentials=true MaxAge=86400
2021/12/02 08:09:47 JWT not required for dev. origins: [http://localhost:8080 http://localhost: http://192.168.1.]
2021/12/02 08:09:47 Enable PProf endpoints: http://localhost:8093/debug/pprof
2021/12/02 08:09:47 Create cookie plan=FreePlan domain=localhost secure=false jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ5ODd9.5tJk2NoHxkG0o_owtMleBcUaR8z1vRx4rxRRqtZUc_Q
2021/12/02 08:09:47 Create cookie plan=PremiumPlan domain=localhost secure=false jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiUHJlbWl1bVBsYW4iLCJleHAiOjE2Njk5NjQ5ODd9.ifKhbmxQQ64NweL5aQDb_42tvKHwqiEKD-vxHO3KzsM
2021/12/02 08:09:47 OPA: load "examples/sample-auth.rego"
2021/12/02 08:09:47 Middleware OPA: map[sample-auth.rego:package auth
default allow = false
tokens := {"Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y"} { true }
allow = true { __local0__ = input.token; data.auth.tokens[__local0__] }]
2021/12/02 08:09:47 Middleware response HTTP header: Set Server MyBackendName-1.2.0
2021/12/02 08:09:47 MiddlewareRateLimiter burst=100 rate=5/s
2021/12/02 08:09:47 Middleware logger: requester IP and requested URL
2021/12/02 08:09:47 Server listening on http://localhost:8080
Test the API with curl:
curl -D - http://localhost:8080/myapp/api/v1/items
HTTP/1.1 401 Unauthorized
Content-Type: application/json
Server: MyBackendName-1.2.0
Vary: Origin
Date: Thu, 02 Dec 2021 07:06:20 GMT
Content-Length: 84
{"error":"Unauthorized",
"path":"/api/v1/items",
"doc":"http://localhost:8080/myapp/doc"}
The corresponding garcon logs:
2021/12/02 08:06:20 in 127.0.0.1:42888 GET /api/v1/items
[cors] 2021/12/02 08:06:20 Handler: Actual request
[cors] 2021/12/02 08:06:20 Actual request no headers added: missing origin
2021/12/02 08:06:20 OPA unauthorize 127.0.0.1:42888 /api/v1/items
2021/12/02 08:06:20 out 127.0.0.1:42888 GET /api/v1/items 1.426916ms c=1
The CORS logs can be disabled by passing debug=false in cors.Handler(origins, false).
The value c=1 measures the web traffic (current active HTTP connections).
curl -D - http://localhost:8080/myapp/api/v1/items -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y'
HTTP/1.1 200 OK
Content-Type: application/json
Server: MyBackendName-1.2.0
Vary: Origin
Date: Thu, 02 Dec 2021 07:10:37 GMT
Content-Length: 25
["item1","item2","item3"]
The corresponding garcon logs:
2021/12/02 08:10:37 in 127.0.0.1:42892 GET /api/v1/items
[cors] 2021/12/02 08:10:37 Handler: Actual request
[cors] 2021/12/02 08:10:37 Actual request no headers added: missing origin
2021/12/02 08:10:37 Authorization header has JWT: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJuIjoiRnJlZVBsYW4iLCJleHAiOjE2Njk5NjQ0ODh9.elDm_t4vezVgEmS8UFFo_spLJTts7JWybzbyO_aYV3Y
2021/12/02 08:10:37 JWT Claims: {FreePlan { [] 2022-12-02 08:01:28 +0100 CET <nil> <nil> invalid cookie}}
2021/12/02 08:10:37 JWT has the FreePlan Namespace
2021/12/02 08:10:37 JWT Permission: {10}
2021/12/02 08:10:37 out 127.0.0.1:42892 GET /api/v1/items 1.984568ms c=1 a=1 i=0 h=0
:warning: WARNING: This section is outdated! :warning:
See the low-level example.
The following code is a bit different to the stuff done
by the complete function Garcon.Run() presented in the previous chapter.
The following code is intended to show
Garcon can be customized to meet your specific requirements.
package main
import (
"log"
"net"
"net/http"
"time"
"github.com/go-chi/chi/v5"
"github.com/teal-finance/garcon"
)
// Garcon settings
const apiDoc = "https://my-dns.co/doc"
const allowedProdOrigin = "https://my-dns.co"
const allowedDevOrigins = "http://localhost: http://192.168.1."
const serverHeader = "MyBackendName-1.2.0"
const authCfg = "examples/sample-auth.rego"
const pprofPort = 8093
const expPort = 9093
const burst, reqMinute = 10, 30
const devMode = true
func main() {
if devMode {
// the following line collects the CPU-profile and writes it in the file "cpu.pprof"
defer garcon.ProbeCPU().Stop()
}
garcon.StartPProfServer(pprofPort)
// Uniformize error responses with API doc
gw := garcon.NewWriter(apiDoc)
middleware, connState := setMiddlewares(gw)
// Handles both REST API and static web files
h := handler(gw)
h = middleware.Then(h)
runServer(h, connState)
}
func setMiddlewares(gw garcon.Writer) (middleware garcon.Chain, connState func(net.Conn, http.ConnState)) {
// Start an exporter/health server in background if export port > 0.
// This server is for use with Kubernetes and Prometheus-like monitoring tools.
middleware, connState = garcon.StartExporter(expPort, devMode)
// Limit the input request rate per IP
reqLimiter := garcon.NewReqLimiter(gw, burst, reqMinute, devMode)
corsConfig := allowedProdOrigin
if devMode {
corsConfig += " " + allowedDevOrigins
}
allowedOrigins := garcon.SplitClean(corsConfig)
middleware = middleware.Append(
reqLimiter.Limit,
garcon.ServerHeader(serverHeader),
cors.Handler(allowedOrigins, devMode),
)
// Endpoint authentication rules (Open Policy Agent)
files := garcon.SplitClean(authCfg)
policy, err := garcon.NewPolicy(gw, files)
if err != nil {
log.Fatal(err)
}
if policy.Ready() {
middleware = middleware.Append(policy.Auth)
}
return middleware, connState
}
// runServer runs in foreground the main server.
func runServer(h http.Handler, connState func(net.Conn, http.ConnState)) {
const mainPort = "8080"
server := http.Server{
Addr: ":" + mainPort,
Handler: h,
TLSConfig: nil,
ReadTimeout: 1 * time.Second,
ReadHeaderTimeout: 1 * time.Second,
WriteTimeout: 1 * time.Second,
IdleTimeout: 1 * time.Second,
MaxHeaderBytes: 222,
TLSNextProto: nil,
ConnState: connState,
ErrorLog: log.Default(),
BaseContext: nil,
ConnContext: nil,
}
log.Print("Server listening on http://localhost", server.Addr)
log.Fatal(server.ListenAndServe())
}
// handler creates the mapping between the endpoints and the handler functions.
func handler(gw garcon.Writer) http.Handler {
r := chi.NewRouter()
// Website with static files
ws := g.NewStaticWebServer("/var/www")
r.Get("/", ws.ServeFile("index.html", "text/html; charset=utf-8"))
r.Get("/js/*", ws.ServeDir("text/javascript; charset=utf-8"))
r.Get("/css/*", ws.ServeDir("text/css; charset=utf-8"))
r.Get("/images/*", ws.ServeImages())
// API
r.Get("/api/v1/items", items)
r.Get("/api/v1/ducks", gw.NotImplemented)
// Other endpoints
r.NotFound(gw.InvalidPath)
return r
}
func items(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.Write([]byte(`["item1","item2","item3"]`))
}
The example KeyStore implements a key/value datastore providing private storage for each client identified by its unique IP.
cd garcon
go run -race ./examples/keystore
Then open http://localhost:8080 to learn more about the implemented features.
This project needs your help to become better. Please propose your enhancements, or even a further refactoring.
We welcome contributions in many forms, and there's always plenty to do!
If you have some suggestions, or need a new feature, please contact us, using the issues, or at Teal.Finance@pm.me or @TealFinance.
Feel free to propose a Pull Request, your contributions are welcome. :wink:
Copyright (c) 2021 Teal.Finance/Garcon contributors
Teal.Finance/Garcon is free software, and can be redistributed and/or modified under the terms of the MIT License. SPDX-License-Identifier: MIT
Teal.Finance/Garcon is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the LICENSE file (alongside the source files) or https://opensource.org/licenses/MIT.
FAQs
Unknown package
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.