rk-grpc
Inject middlewares & server configuration of gRPC and grpc-gateway from YAML file.
This belongs to rk-boot family. We suggest use this lib with rk-boot.
Architecture
Quick Start
In the bellow example, we will start microservice with bellow functionality and middlewares enabled via YAML.
- gRPC and grpc-gateway server
- gRPC server reflection
- Swagger UI
- CommonService
- Docs
- Prometheus Metrics (middleware)
- Logging (middleware)
- Meta (middleware)
Please refer example at example/boot/simple.
Installation
go get github.com/rookie-ninja/rk-grpc/v2
1.Prepare .proto files
show
syntax = "proto3";
package api.v1;
option go_package = "api/v1/greeter";
service Greeter {
rpc Greeter (GreeterRequest) returns (GreeterResponse) {}
}
message GreeterRequest {
bytes msg = 1;
}
message GreeterResponse {}
type: google.api.Service
config_version: 3
http:
rules:
- selector: api.v1.Greeter.Greeter
get: /v1/greeter
version: v1beta1
name: github.com/rk-dev/rk-boot
build:
roots:
- api
version: v1beta1
plugins:
- name: go
out: api/gen
opt:
- paths=source_relative
- name: go-grpc
out: api/gen
opt:
- paths=source_relative
- require_unimplemented_servers=false
- name: grpc-gateway
out: api/gen
opt:
- paths=source_relative
- grpc_api_configuration=api/v1/gw_mapping.yaml
- name: openapiv2
out: api/gen
opt:
- grpc_api_configuration=api/v1/gw_mapping.yaml
2.Generate .pb.go files with buf
show
$ buf generate --path api/v1
.
├── api
│ ├── gen
│ │ └── v1
│ │ ├── greeter.pb.go
│ │ ├── greeter.pb.gw.go
│ │ ├── greeter.swagger.json
│ │ └── greeter_grpc.pb.go
│ └── v1
│ ├── greeter.proto
│ └── gw_mapping.yaml
├── boot.yaml
├── buf.gen.yaml
├── buf.yaml
├── go.mod
├── go.sum
└── main.go
3.Create boot.yaml
show
---
grpc:
- name: greeter
port: 8080
enabled: true
enableReflection: true
enableRkGwOption: true
commonService:
enabled: true
docs:
enabled: true
sw:
enabled: true
prom:
enabled: true
middleware:
logging:
enabled: true
prom:
enabled: true
meta:
enabled: true
4.Create main.go
show
package main
import (
"context"
"embed"
_ "embed"
"github.com/rookie-ninja/rk-entry/v2/entry"
"github.com/rookie-ninja/rk-grpc/v2/boot"
proto "github.com/rookie-ninja/rk-grpc/v2/example/boot/simple/api/gen/v1"
"google.golang.org/grpc"
)
var boot []byte
var docsFS embed.FS
var staticFS embed.FS
func init() {
rkentry.GlobalAppCtx.AddEmbedFS(rkentry.DocsEntryType, "greeter", &docsFS)
rkentry.GlobalAppCtx.AddEmbedFS(rkentry.SWEntryType, "greeter", &docsFS)
rkentry.GlobalAppCtx.AddEmbedFS(rkentry.StaticFileHandlerEntryType, "greeter", &staticFS)
}
func main() {
rkentry.BootstrapPreloadEntryYAML(boot)
res := rkgrpc.RegisterGrpcEntryYAML(boot)
grpcEntry := res["greeter"].(*rkgrpc.GrpcEntry)
grpcEntry.AddRegFuncGrpc(func(server *grpc.Server) {
proto.RegisterGreeterServer(server, &GreeterServer{})
})
grpcEntry.AddRegFuncGw(proto.RegisterGreeterHandlerFromEndpoint)
grpcEntry.Bootstrap(context.Background())
rkentry.GlobalAppCtx.WaitForShutdownSig()
grpcEntry.Interrupt(context.Background())
}
type GreeterServer struct{}
func (server *GreeterServer) Greeter(context.Context, *proto.GreeterRequest) (*proto.GreeterResponse, error) {
return &proto.GreeterResponse{}, nil
}
5.Start server
$ go run main.go
6.Validation
show
6.1 gRPC & grpc-gateway server
Try to test gRPC & grpc-gateway Service with curl & grpcurl
# Curl to common service
$ curl localhost:8080/rk/v1/ready
{"ready":true}
6.2 Swagger UI
Please refer sw section at Full YAML.
By default, we could access swagger UI at http://localhost:8080/sw
6.3 Docs UI
Please refer docs section at Full YAML.
By default, we could access docs UI at http://localhost:8080/docs
6.4 Prometheus Metrics
Please refer middleware.prom section at Full YAML.
By default, we could access prometheus client at http://localhost:8080/metrics
6.5 Logging
Please refer middleware.logging section at Full YAML.
By default, we enable zap logger and event logger with encoding type of [console]. Encoding type of [json] and [flatten] is also supported.
2021-12-28T05:36:21.561+0800 INFO boot/grpc_entry.go:1515 Bootstrap grpcEntry {"eventId": "db2c977c-e0ff-4b21-bc0d-5966f1cad093", "entryName": "greeter"}
------------------------------------------------------------------------
endTime=2021-12-28T05:36:21.563575+08:00
startTime=2021-12-28T05:36:21.561362+08:00
elapsedNano=2213846
timezone=CST
ids={"eventId":"db2c977c-e0ff-4b21-bc0d-5966f1cad093"}
app={"appName":"rk","appVersion":"","entryName":"greeter","entryType":"GrpcEntry"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.2","os":"darwin","realm":"*","region":"*"}
payloads={"commonServiceEnabled":true,"commonServicePathPrefix":"/rk/v1/","grpcPort":8080,"gwPort":8080,"promEnabled":true,"promPath":"/metrics","promPort":8080,"swEnabled":true,"swPath":"/sw/","tvEnabled":true,"tvPath":"/rk/v1/tv/"}
error={}
counters={}
pairs={}
timing={}
remoteAddr=localhost
operation=Bootstrap
resCode=OK
eventStatus=Ended
EOE
6.6 Meta
Please refer meta section at Full YAML.
By default, we will send back some metadata to client with headers.
$ curl -vs localhost:8080/rk/v1/ready
...
< HTTP/1.1 200 OK
< Content-Type: application/json
< X-Request-Id: 7e4f5ac5-3369-485f-89f7-55551cc4a9a1
< X-Rk-App-Name: rk
< X-Rk-App-Unix-Time: 2021-12-28T05:39:50.508328+08:00
< X-Rk-App-Version:
< X-Rk-Received-Time: 2021-12-28T05:39:50.508328+08:00
< Date: Mon, 27 Dec 2021 21:39:50 GMT
...
6.7 Send request
We registered /v1/greeter API in grpc-gateway server and let's validate it!
$ curl -vs localhost:8080/v1/greeter
* Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 8080 failed: Connection refused
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /v1/greeter HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< X-Request-Id: 07b0fbf6-cebf-40ac-84a2-533bbd4b8958
< X-Rk-App-Name: rk
< X-Rk-App-Unix-Time: 2021-12-28T05:41:04.653652+08:00
< X-Rk-App-Version:
< X-Rk-Received-Time: 2021-12-28T05:41:04.653652+08:00
< Date: Mon, 27 Dec 2021 21:41:04 GMT
< Content-Length: 2
<
* Connection #0 to host localhost left intact
{}
We registered api.v1.Greeter.Greeter API in gRPC server and let's validate it!
$ grpcurl -plaintext localhost:8080 api.v1.Greeter.Greeter
{}
6.8 RPC logs
Bellow logs would be printed in stdout.
The first block of log is from grpc-gateway request.
The second block of log is from gRPC request.
------------------------------------------------------------------------
endTime=2021-12-28T05:45:52.986041+08:00
startTime=2021-12-28T05:45:52.985956+08:00
elapsedNano=85065
timezone=CST
ids={"eventId":"88362f69-7eda-4f03-bdbe-7ef667d06bac","requestId":"88362f69-7eda-4f03-bdbe-7ef667d06bac"}
app={"appName":"rk","appVersion":"","entryName":"greeter","entryType":"GrpcEntry"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.2","os":"darwin","realm":"*","region":"*"}
payloads={"grpcMethod":"Greeter","grpcService":"api.v1.Greeter","grpcType":"unaryServer","gwMethod":"GET","gwPath":"/v1/greeter","gwScheme":"http","gwUserAgent":"curl/7.64.1"}
error={}
counters={}
pairs={}
timing={}
remoteAddr=127.0.0.1:61520
operation=/api.v1.Greeter/Greeter
resCode=OK
eventStatus=Ended
EOE
------------------------------------------------------------------------
endTime=2021-12-28T05:44:45.686734+08:00
startTime=2021-12-28T05:44:45.686592+08:00
elapsedNano=141716
timezone=CST
ids={"eventId":"7765862c-9e83-443a-a6e5-bb28f17f8ea0","requestId":"7765862c-9e83-443a-a6e5-bb28f17f8ea0"}
app={"appName":"rk","appVersion":"","entryName":"greeter","entryType":"GrpcEntry"}
env={"arch":"amd64","az":"*","domain":"*","hostname":"lark.local","localIP":"10.8.0.2","os":"darwin","realm":"*","region":"*"}
payloads={"grpcMethod":"Greeter","grpcService":"api.v1.Greeter","grpcType":"unaryServer","gwMethod":"","gwPath":"","gwScheme":"","gwUserAgent":""}
error={}
counters={}
pairs={}
timing={}
remoteAddr=127.0.0.1:57149
operation=/api.v1.Greeter/Greeter
resCode=OK
eventStatus=Ended
EOE
6.9 RPC prometheus metrics
Prometheus client will automatically register into grpc-gateway instance at /metrics.
Access http://localhost:8080/metrics
Supported features
User can enable anyone of those as needed! No mandatory binding!
Instance | Description |
---|
gRPC | gRPC defined with protocol buffer. |
gRPC proxy | Proxy gRPC request to another gRPC server. |
grpc-gateway | grpc-gateway service with same port. |
grpc-gateway options | Well defined grpc-gateway options. |
Config | Configure spf13/viper as config instance and reference it from YAML |
Logger | Configure uber-go/zap logger configuration and reference it from YAML |
Event | Configure logging of RPC with rk-query and reference it from YAML |
Cert | Fetch TLS/SSL certificates from remote datastore like ETCD and start microservice. |
Prometheus | Start prometheus client at client side and push metrics to pushgateway as needed. |
Swagger | Builtin swagger UI handler. |
Docs | Builtin RapiDoc instance which can be used to replace swagger and RK TV. |
CommonService | List of common APIs. |
StaticFileHandler | A Web UI shows files could be downloaded from server, currently support source of local and embed.FS. |
PProf | PProf web UI. |
gRPC Web | gRPC Web |
Supported middlewares
All middlewares could be configured via YAML or Code.
User can enable anyone of those as needed! No mandatory binding!
Middleware | Description |
---|
Metrics | Collect RPC metrics and export to prometheus client. |
Log | Log every RPC requests as event with rk-query. |
Trace | Collect RPC trace and export it to stdout, file or jaeger with open-telemetry/opentelemetry-go. |
Panic | Recover from panic for RPC requests and log it. |
Meta | Send micsroservice metadata as header to client. |
Auth | Support [Basic Auth] and [API Key] authorization types. |
RateLimit | Limiting RPC rate globally or per path. |
Timeout | Timing out request by configuration. |
CORS | Server side CORS validation. |
JWT | Server side JWT validation. |
Secure | Server side secure validation. |
CSRF | Server side csrf validation. |
YAML options
User can start multiple gRPC and grpc-gateway instances at the same time. Please make sure use different port and name.
show
---
grpc:
- name: greeter
enabled: true
port: 8080
Development Status: Stable
Build instruction
Simply run make all to validate your changes. Or run codes in example/ folder.
Run unit-test, golangci-lint, doctoc and gofmt.
Test instruction
Run unit test with make test command.
Github workflow will automatically run unit test and golangci-lint for testing and lint validation.
Contributing
We encourage and support an active, healthy community of contributors;
including you! Details are in the contribution guide and
the code of conduct. The rk maintainers keep an eye on
issues and pull requests, but you can also report any negative conduct to
lark@rkdev.info.
Released under the Apache 2.0 License.