github.com/bold-commerce/protoc-gen-struct-transformer
Protocol buffers complier protoc generated structures based on message
definition in *.proto file. It's possible to use these generated structures
directly, but it's better to have clear separation between transport level
(gRPC) and business logic with its own structures. In this case you have to
convert protobuf structures into business logic structures and vice versa.
protoc-gen-struct-transformer is a plugin for protoc which generates functions
for structure transformation.
Let's look at simple example.
Source proto file:
// message.proto
syntax = "proto3";
package messages;
message Product {
int32 id = 1;
string name = 2;
}
Command protoc --gogofaster_out=. message.proto will generate message.pb.go with
following structure:
type Product struct {
Id int32 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,json=name,proto3" json:"name,omitempty"`
}
and let's suppose you service has as repo package with ProductModel struct inside:
type ProductModel struct {
ID int `db:"id" json:"id"`
Name string `db:"name" json:"name"`
}
In order to publish data from the repo to API you have to convert ProductModel
to Product, for saving data which hit API you have to make back transformation.
func ProductModelToProduct(m repo.ProductModel) proto.Product {
return proto.Product {
Id: m.ID,
Name: m.Name,
}
}
func ProductToProductModel(p proto.Product) repo.ProductModel {
return repo.ProductModel {
ID: p.Id,
Name: p.Name,
}
}
List of function type should be generated:
| Source | Destination | Suffix name |
|---|---|---|
| *proto | *model | Ptr |
| []*proto | []*model | PtrList |
| *proto | model | PtrVal |
| []*proto | []model | PtrValList |
| proto | model | |
| proto | *model | ValPtr |
| []proto | []model | ValList |
| *model | *proto | Ptr |
| []*model | []*proto | PtrList |
| *model | proto | PtrVal |
| []*model | []proto | PtrValList |
| model | proto | |
| model | *proto | ValPtr |
| []model | []proto | ValList |
function name has a format <Source>To<Destination><Suffix>.
For instance, function which converts list of pointers to Product into list of
Product values will be named PbToProductPtrValList,
where
Pb is a replacement for proto messageProducts is a name of model structurePtrValList is a suffix pointed that convertion is made from slice of pointer to slice of values.Full set of function for Product message will be as:
func PbToProductPtr(src *example.Product, opts ...TransformParam) *model.Product
func PbToProductPtrList(src []*example.Product, opts ...TransformParam) []*model.Product
func PbToProductPtrVal(src *example.Product, opts ...TransformParam) model.Product
func PbToProductPtrValList(src []*example.Product, opts ...TransformParam) []model.Product
func PbToProductList(src []*example.Product, opts ...TransformParam) []model.Product
func PbToProduct(src example.Product, opts ...TransformParam) model.Product
func PbToProductValPtr(src example.Product, opts ...TransformParam) *model.Product
func PbToProductValList(src []example.Product, opts ...TransformParam) []model.Product
func ProductToPbPtr(src *model.Product, opts ...TransformParam) *example.Product
func ProductToPbPtrList(src []*model.Product, opts ...TransformParam) []*example.Product
func ProductToPbPtrVal(src *model.Product, opts ...TransformParam) example.Product
func ProductToPbValPtrList(src []model.Product, opts ...TransformParam) []*example.Product
func ProductToPbList(src []model.Product, opts ...TransformParam) []*example.Product
func ProductToPb(src model.Product, opts ...TransformParam) example.Product
func ProductToPbValPtr(src model.Product, opts ...TransformParam) *example.Product
func ProductToPbValList(src []model.Product, opts ...TransformParam) []example.Product
where
example is a package generated by protoc-gen-go or protoc-gen-gogo pluginmodel is a package which contains manually created models structures.Full example you can find in example directory.
I assume you already have protoc installed.
First of all, it's necessary to install plugin itself, it's just a binary file,
which should be placed into $PATH to be available for protoc.
% brew tap bold-commerce/tap
% brew install protoc-gen-struct-transformer
If you're going to make changes to plugin, use go get ... or git clone ...
% export GO111MODULE=on
% go get -u -d github.com/bold-commerce/protoc-gen-struct-transformer
% cd $GOPATH/src/github.com/bold-commerce/protoc-gen-struct-transformer
// make changes
% go install
Next, we need protoc-gen-go plugin (or protoc-gen-gogofaster if you use
gogo specific options) which creates *.pb.go file.
go get -u github.com/golang/protobuf/protoc-gen-go
// or
go get -u github.com/gogo/protobuf/protoc-gen-gogofaster
To configure plugin you have to use file level options listed below. The plugin will not process file without these options.
// This import allows to use this options.
// Relatively to import_path: github.com/bold-commerce/protoc-gen-struct-transformer.
// See Makefile for mapping details.
import "options/annotations.proto";
// Go package name which contains business logic structures.
option (transformer.go_repo_package) = "models";
// Go package name with protobuf generated srtuctures. Could be equal to
// options go_package.
option (transformer.go_protobuf_package) = "example";
// Path to source file with Go structures which will be used as destination.
option (transformer.go_models_file_path) = "example/model/model.go";
as well as message level option
// Name of structure from business logic package. This option links business
// logic and generated structure.
message Product {
option (transformer.go_struct) = "ProductModel";
// ...
}
options above are minimal requirement for use this plugin.
Also plugin has additional field level options:
message Product {
// SomeField will not be added to transformation function.
string some_field = 4 [ (transformer.skip) = true ];
// "map_as" option is used in cases when protoc-gen-go* plugin creates
// "unpredictable" field name, i.e. by default protoc-gen-go* converts
// protobuf named writen in snake_case into CamelCase, but for fields like
// map_field_1 this rule has aa exception, in pb.go file it will be
// "MapField_1" instead of "MapField1".
// "map_to" options is used when you need to map current message field to
// field in model with arbitrary name.
// Both options "map_as" and "map_to" can be used independently.
string map_field_1 = 6 [ (transformer.map_as) = "MapField_1", (transformer.map_to) = "MapField1"];
// "custom" allows to use custom transformers for fields, which require extended transformation
// The plugin won't generate methods for this field,
// but rather expect it to be in the same package with the transformer file
CustomType custom_field [(transformer.custom) = true]
}
protoc \
--proto_path=github.com/gogo:. \
--go_out=Moptions/annotations.proto=github.com/bold-commerce/protoc-gen-struct-transformer/options,plugins=grpc:. \
--struct-transformer_out=package=transform:. \
./message.proto
this command generates two files:
message.pb.go contains auto-generated structures.transform/message_transformer.go contains transformation functions.by default message_transformer.go does not contain imports. To add imports
run protoc with:
--struct-transformer_out=package=transform,goimports=true:. \
func (s *server) CreateProduct(ctx context.Context, req *pb.Request) (*pb.Response, error) {
p, err := s.svc.Create(ctx, transform.PbToProduct(req.Product))
if err != nil {
return nil, err
}
return &pb.Response{
Product: transform.ProductToPb(p),
}, nil
}
Usage of protoc-gen-struct-transformer:
-debug
Add debug information to generated file.
-goimports
Perform goimports on generated file.
-helper-package string
Package name for helper functions.
-package string
Package name for generated functions. (default "fallback")
-use-package-in-path
If true, package parameter will be used in path for output file. (default true)
-version
Print current version.
gogo.proto file which is used for gogo-specific options is imported from
go modules cache. In order to fill out the cache run:
% export GO111MODULE=on
% go build
and run make generate again.
FAQs
Unknown package
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.