Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package openapi provides a Rex extension that generates an OpenAPI 3.1 specification document from routes implementing the OpenAPIRoute interface.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Command spec-generator generates PLATEAU specification documents as Markdown files.
Package gendoc is a protoc plugin for generating documentation from your proto files.
envdoc is a tool to generate documentation for environment variables from a Go source file.
The claat command generates one or more codelabs from "source" documents, specified as either Google Doc IDs or local markdown files. The command also allows one to preview generated codelabs from local drive using "claat serve". See more details at https://github.com/googlecodelabs/tools.
Package gendoc is a protoc plugin for generating documentation from your proto files.
Package schemadoc generates reference documentation and example configurations from JSON Schema.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
(godoc-to-GitHub-MarkDown) generates package documentation in GitHub flavoured Markdown.
Package gofpdf provides a pure Go implementation for PDF document generation.
Package crypto11 enables access to cryptographic keys from PKCS#11 using Go crypto API. PKCS#11 tokens are accessed via Context objects. Each Context connects to one token. Context objects are created by calling Configure or ConfigureFromFile. In the latter case, the file should contain a JSON representation of a Config. There is support for generating DSA, RSA and ECDSA keys. These keys can be found later using FindKeyPair. All three key types implement the crypto.Signer interface and the RSA keys also implement crypto.Decrypter. RSA keys obtained through FindKeyPair will need a type assertion to be used for decryption. Assert either crypto.Decrypter or SignerDecrypter, as you prefer. Symmetric keys can also be generated. These are found later using FindKey. See the documentation for SecretKey for further information. Note that PKCS#11 session handles must not be used concurrently from multiple threads. Consumers of the Signer interface know nothing of this and expect to be able to sign from multiple threads without constraint. We address this as follows. 1. When a Context is created, a session is created and the user is logged in. This session remains open until the Context is closed, to ensure all object handles remain valid and to avoid repeatedly calling C_Login. 2. The Context also maintains a pool of read-write sessions. The pool expands dynamically as needed, but never beyond the maximum number of r/w sessions supported by the token (as reported by C_GetInfo). If other applications are using the token, a lower limit should be set in the Config. 3. Each operation transiently takes a session from the pool. They have exclusive use of the session, meeting PKCS#11's concurrency requirements. Sessions are returned to the pool afterwards and may be re-used. Behaviour of the pool can be tweaked via Config fields: - PoolWaitTimeout controls how long an operation can block waiting on a session from the pool. A zero value means there is no limit. Timeouts occur if the pool is fully used and additional operations are requested. - MaxSessions sets an upper bound on the number of sessions. If this value is zero, a default maximum is used (see DefaultMaxSessions). In every case the maximum supported sessions as reported by the token is obeyed. The PKCS1v15DecryptOptions SessionKeyLen field is not implemented and an error is returned if it is nonzero. The reason for this is that it is not possible for crypto11 to guarantee the constant-time behavior in the specification. See https://github.com/thalesignite/crypto11/issues/5 for further discussion. Symmetric crypto support via cipher.Block is very slow. You can use the BlockModeCloser API but you must call the Close() interface (not found in cipher.BlockMode). See https://github.com/ThalesIgnite/crypto11/issues/6 for further discussion.
Package poltergeist is a high-performance, lightweight Go framework for building REST APIs and Realtime applications with built-in WebSocket, SSE, and auto-generated OpenAPI documentation.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package docxtpl provides a library for generating DOCX documents from templates using Go's text/template syntax.
Package brdoc provides validation, generation, and formatting for Brazilian fiscal documents. This package implements official algorithms from SERPRO (Serviço Federal de Processamento de Dados) for both CPF (Cadastro de Pessoas Físicas) and alphanumeric CNPJ (Cadastro Nacional de Pessoa Jurídica) validation. The CPF validator supports: The CNPJ validator supports alphanumeric format per SERPRO specification: CPF validation example: CNPJ validation example: Auto-detection example: Generate valid documents: Format documents to standard Brazilian format: The library implements the official SERPRO specification: For more information, see: https://github.com/inovacc/brdoc
Package oastools provides tools for parsing, validating, fixing, converting, joining, comparing, generating code from, and building OpenAPI Specification (OAS) documents from OAS 2.0 through OAS 3.2.0.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package datastore provides a client for Google Cloud Datastore. See https://godoc.org/cloud.google.com/go for authentication, timeouts, connection pooling and similar aspects of this package. Entities are the unit of storage and are associated with a key. A key consists of an optional parent key, a string application ID, a string kind (also known as an entity type), and either a StringID or an IntID. A StringID is also known as an entity name or key name. It is valid to create a key with a zero StringID and a zero IntID; this is called an incomplete key, and does not refer to any saved entity. Putting an entity into the datastore under an incomplete key will cause a unique key to be generated for that entity, with a non-zero IntID. An entity's contents are a mapping from case-sensitive field names to values. Valid value types are: Slices of structs are valid, as are structs that contain slices. The Get and Put functions load and save an entity's contents. An entity's contents are typically represented by a struct pointer. Example code: GetMulti, PutMulti and DeleteMulti are batch versions of the Get, Put and Delete functions. They take a []*Key instead of a *Key, and may return a datastore.MultiError when encountering partial failure. Mutate generalizes PutMulti and DeleteMulti to a sequence of any Datastore mutations. It takes a series of mutations created with NewInsert, NewUpdate, NewUpsert and NewDelete and applies them atomically. An entity's contents can be represented by a variety of types. These are typically struct pointers, but can also be any type that implements the PropertyLoadSaver interface. If using a struct pointer, you do not have to explicitly implement the PropertyLoadSaver interface; the datastore will automatically convert via reflection. If a struct pointer does implement PropertyLoadSaver then those methods will be used in preference to the default behavior for struct pointers. Struct pointers are more strongly typed and are easier to use; PropertyLoadSavers are more flexible. The actual types passed do not have to match between Get and Put calls or even across different calls to datastore. It is valid to put a *PropertyList and get that same entity as a *myStruct, or put a *myStruct0 and get a *myStruct1. Conceptually, any entity is saved as a sequence of properties, and is loaded into the destination value on a property-by-property basis. When loading into a struct pointer, an entity that cannot be completely represented (such as a missing field) will result in an ErrFieldMismatch error but it is up to the caller whether this error is fatal, recoverable or ignorable. By default, for struct pointers, all properties are potentially indexed, and the property name is the same as the field name (and hence must start with an upper case letter). Fields may have a `datastore:"name,options"` tag. The tag name is the property name, which must be one or more valid Go identifiers joined by ".", but may start with a lower case letter. An empty tag name means to just use the field name. A "-" tag name means that the datastore will ignore that field. The only valid options are "omitempty", "noindex" and "flatten". If the options include "omitempty" and the value of the field is a zero value, then the field will be omitted on Save. Zero values are best defined in the golang spec (https://golang.org/ref/spec#The_zero_value). Struct field values will never be empty, except for nil pointers. If options include "noindex" then the field will not be indexed. All fields are indexed by default. Strings or byte slices longer than 1500 bytes cannot be indexed; fields used to store long strings and byte slices must be tagged with "noindex" or they will cause Put operations to fail. For a nested struct field, the options may also include "flatten". This indicates that the immediate fields and any nested substruct fields of the nested struct should be flattened. See below for examples. To use multiple options together, separate them by a comma. The order does not matter. If the options is "" then the comma may be omitted. Example code: A field of slice type corresponds to a Datastore array property, except for []byte, which corresponds to a Datastore blob. Zero-length slice fields are not saved. Slice fields of length 1 or greater are saved as Datastore arrays. When a zero-length Datastore array is loaded into a slice field, the slice field remains unchanged. If a non-array value is loaded into a slice field, the result will be a slice with one element, containing the value. Loading a Datastore Null into a basic type (int, float, etc.) results in a zero value. Loading a Null into a slice of basic type results in a slice of size 1 containing the zero value. Loading a Null into a pointer field results in nil. Loading a Null into a field of struct type is an error. A struct field can be a pointer to a signed integer, floating-point number, string or bool. Putting a non-nil pointer will store its dereferenced value. Putting a nil pointer will store a Datastore Null property, unless the field is marked omitempty, in which case no property will be stored. Loading a Null into a pointer field sets the pointer to nil. Loading any other value allocates new storage with the value, and sets the field to point to it. If the struct contains a *datastore.Key field tagged with the name "__key__", its value will be ignored on Put. When reading the Entity back into the Go struct, the field will be populated with the *datastore.Key value used to query for the Entity. Example code: If the struct pointed to contains other structs, then the nested or embedded structs are themselves saved as Entity values. For example, given these definitions: then an Outer would have one property, Inner, encoded as an Entity value. If an outer struct is tagged "noindex" then all of its implicit flattened fields are effectively "noindex". If the Inner struct contains a *Key field with the name "__key__", like so: then the value of K will be used as the Key for Inner, represented as an Entity value in datastore. If any nested struct fields should be flattened, instead of encoded as Entity values, the nested struct field should be tagged with the "flatten" option. For example, given the following: an Outer's properties would be equivalent to those of: Note that the "flatten" option cannot be used for Entity value fields. The server will reject any dotted field names for an Entity value. An entity's contents can also be represented by any type that implements the PropertyLoadSaver interface. This type may be a struct pointer, but it does not have to be. The datastore package will call Load when getting the entity's contents, and Save when putting the entity's contents. Possible uses include deriving non-stored fields, verifying fields, or indexing a field only if its value is positive. Example code: The *PropertyList type implements PropertyLoadSaver, and can therefore hold an arbitrary entity's contents. If a type implements the PropertyLoadSaver interface, it may also want to implement the KeyLoader interface. The KeyLoader interface exists to allow implementations of PropertyLoadSaver to also load an Entity's Key into the Go type. This type may be a struct pointer, but it does not have to be. The datastore package will call LoadKey when getting the entity's contents, after calling Load. Example code: To load a Key into a struct which does not implement the PropertyLoadSaver interface, see the "Key Field" section above. Queries retrieve entities based on their properties or key's ancestry. Running a query yields an iterator of results: either keys or (key, entity) pairs. Queries are re-usable and it is safe to call Query.Run from concurrent goroutines. Iterators are not safe for concurrent use. Queries are immutable, and are either created by calling NewQuery, or derived from an existing query by calling a method like Filter or Order that returns a new query value. A query is typically constructed by calling NewQuery followed by a chain of zero or more such methods. These methods are: Example code: Client.RunInTransaction runs a function in a transaction. Example code: Pass the ReadOnly option to RunInTransaction if your transaction is used only for Get, GetMulti or queries. Read-only transactions are more efficient. This package supports the Cloud Datastore emulator, which is useful for testing and development. Environment variables are used to indicate that datastore traffic should be directed to the emulator instead of the production Datastore service. To install and set up the emulator and its environment variables, see the documentation at https://cloud.google.com/datastore/docs/tools/datastore-emulator.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images. - UTF-8 support - Choice of measurement unit, page format and margins - Page header and footer management - Automatic page breaks, line breaks, and text justification - Inclusion of JPEG, PNG, GIF, TIFF and basic path-only SVG images - Colors, gradients and alpha channel transparency - Outline bookmarks - Internal and external links - TrueType, Type1 and encoding support - Page compression - Lines, Bézier curves, arcs, and ellipses - Rotation, scaling, skewing, translation, and mirroring - Clipping - Document protection - Layers - Templates - Barcodes - Charting facility - Import PDFs as templates gofpdf has no dependencies other than the Go standard library. All tests pass on Linux, Mac and Windows platforms. gofpdf supports UTF-8 TrueType fonts and “right-to-left” languages. Note that Chinese, Japanese, and Korean characters may not be included in many general purpose fonts. For these languages, a specialized font (for example, NotoSansSC for simplified Chinese) can be used. Also, support is provided to automatically translate UTF-8 runes to code page encodings for languages that have fewer than 256 glyphs. This repository will not be maintained, at least for some unknown duration. But it is hoped that gofpdf has a bright future in the open source world. Due to Go’s promise of compatibility, gofpdf should continue to function without modification for a longer time than would be the case with many other languages. Forks should be based on the last viable commit. Tools such as active-forks can be used to select a fork that looks promising for your needs. If a particular fork looks like it has taken the lead in attracting followers, this README will be updated to point people in that direction. The efforts of all contributors to this project have been deeply appreciated. Best wishes to all of you. To install the package on your system, run Later, to receive updates, run The following Go code generates a simple PDF file. See the functions in the fpdf_test.go file (shown as examples in this documentation) for more advanced PDF examples. If an error occurs in an Fpdf method, an internal error field is set. After this occurs, Fpdf method calls typically return without performing any operations and the error state is retained. This error management scheme facilitates PDF generation since individual method calls do not need to be examined for failure; it is generally sufficient to wait until after Output() is called. For the same reason, if an error occurs in the calling application during PDF generation, it may be desirable for the application to transfer the error to the Fpdf instance by calling the SetError() method or the SetErrorf() method. At any time during the life cycle of the Fpdf instance, the error state can be determined with a call to Ok() or Err(). The error itself can be retrieved with a call to Error(). This package is a relatively straightforward translation from the original FPDF library written in PHP (despite the caveat in the introduction to Effective Go). The API names have been retained even though the Go idiom would suggest otherwise (for example, pdf.GetX() is used rather than simply pdf.X()). The similarity of the two libraries makes the original FPDF website a good source of information. It includes a forum and FAQ. However, some internal changes have been made. Page content is built up using buffers (of type bytes.Buffer) rather than repeated string concatenation. Errors are handled as explained above rather than panicking. Output is generated through an interface of type io.Writer or io.WriteCloser. A number of the original PHP methods behave differently based on the type of the arguments that are passed to them; in these cases additional methods have been exported to provide similar functionality. Font definition files are produced in JSON rather than PHP. A side effect of running go test ./... is the production of a number of example PDFs. These can be found in the gofpdf/pdf directory after the tests complete. Please note that these examples run in the context of a test. In order run an example as a standalone application, you’ll need to examine fpdf_test.go for some helper routines, for example exampleFilename() and summary(). Example PDFs can be compared with reference copies in order to verify that they have been generated as expected. This comparison will be performed if a PDF with the same name as the example PDF is placed in the gofpdf/pdf/reference directory and if the third argument to ComparePDFFiles() in internal/example/example.go is true. (By default it is false.) The routine that summarizes an example will look for this file and, if found, will call ComparePDFFiles() to check the example PDF for equality with its reference PDF. If differences exist between the two files they will be printed to standard output and the test will fail. If the reference file is missing, the comparison is considered to succeed. In order to successfully compare two PDFs, the placement of internal resources must be consistent and the internal creation timestamps must be the same. To do this, the methods SetCatalogSort() and SetCreationDate() need to be called for both files. This is done automatically for all examples. Nothing special is required to use the standard PDF fonts (courier, helvetica, times, zapfdingbats) in your documents other than calling SetFont(). You should use AddUTF8Font() or AddUTF8FontFromBytes() to add a TrueType UTF-8 encoded font. Use RTL() and LTR() methods switch between “right-to-left” and “left-to-right” mode. In order to use a different non-UTF-8 TrueType or Type1 font, you will need to generate a font definition file and, if the font will be embedded into PDFs, a compressed version of the font file. This is done by calling the MakeFont function or using the included makefont command line utility. To create the utility, cd into the makefont subdirectory and run “go build”. This will produce a standalone executable named makefont. Select the appropriate encoding file from the font subdirectory and run the command as in the following example. In your PDF generation code, call AddFont() to load the font and, as with the standard fonts, SetFont() to begin using it. Most examples, including the package example, demonstrate this method. Good sources of free, open-source fonts include Google Fonts and DejaVu Fonts. The draw2d package is a two dimensional vector graphics library that can generate output in different forms. It uses gofpdf for its document production mode. gofpdf is a global community effort and you are invited to make it even better. If you have implemented a new feature or corrected a problem, please consider contributing your change to the project. A contribution that does not directly pertain to the core functionality of gofpdf should be placed in its own directory directly beneath the contrib directory. Here are guidelines for making submissions. Your change should - be compatible with the MIT License - be properly documented - be formatted with go fmt - include an example in fpdf_test.go if appropriate - conform to the standards of golint and go vet, that is, golint . and go vet . should not generate any warnings - not diminish test coverage Pull requests are the preferred means of accepting your changes. gofpdf is released under the MIT License. It is copyrighted by Kurt Jung and the contributors acknowledged below. This package’s code and documentation are closely derived from the FPDF library created by Olivier Plathey, and a number of font and image resources are copied directly from it. Bruno Michel has provided valuable assistance with the code. Drawing support is adapted from the FPDF geometric figures script by David Hernández Sanz. Transparency support is adapted from the FPDF transparency script by Martin Hall-May. Support for gradients and clipping is adapted from FPDF scripts by Andreas Würmser. Support for outline bookmarks is adapted from Olivier Plathey by Manuel Cornes. Layer support is adapted from Olivier Plathey. Support for transformations is adapted from the FPDF transformation script by Moritz Wagner and Andreas Würmser. PDF protection is adapted from the work of Klemen Vodopivec for the FPDF product. Lawrence Kesteloot provided code to allow an image’s extent to be determined prior to placement. Support for vertical alignment within a cell was provided by Stefan Schroeder. Ivan Daniluk generalized the font and image loading code to use the Reader interface while maintaining backward compatibility. Anthony Starks provided code for the Polygon function. Robert Lillack provided the Beziergon function and corrected some naming issues with the internal curve function. Claudio Felber provided implementations for dashed line drawing and generalized font loading. Stani Michiels provided support for multi-segment path drawing with smooth line joins, line join styles, enhanced fill modes, and has helped greatly with package presentation and tests. Templating is adapted by Marcus Downing from the FPDF_Tpl library created by Jan Slabon and Setasign. Jelmer Snoeck contributed packages that generate a variety of barcodes and help with registering images on the web. Jelmer Snoek and Guillermo Pascual augmented the basic HTML functionality with aligned text. Kent Quirk implemented backwards-compatible support for reading DPI from images that support it, and for setting DPI manually and then having it properly taken into account when calculating image size. Paulo Coutinho provided support for static embedded fonts. Dan Meyers added support for embedded JavaScript. David Fish added a generic alias-replacement function to enable, among other things, table of contents functionality. Andy Bakun identified and corrected a problem in which the internal catalogs were not sorted stably. Paul Montag added encoding and decoding functionality for templates, including images that are embedded in templates; this allows templates to be stored independently of gofpdf. Paul also added support for page boxes used in printing PDF documents. Wojciech Matusiak added supported for word spacing. Artem Korotkiy added support of UTF-8 fonts. Dave Barnes added support for imported objects and templates. Brigham Thompson added support for rounded rectangles. Joe Westcott added underline functionality and optimized image storage. Benoit KUGLER contributed support for rectangles with corners of unequal radius, modification times, and for file attachments and annotations. - Remove all legacy code page font support; use UTF-8 exclusively - Improve test coverage as reported by the coverage tool. Example demonstrates the generation of a simple PDF document. Note that since only core fonts are used (in this case Arial, a synonym for Helvetica), an empty string can be specified for the font directory in the call to New(). Note also that the example.Filename() and example.Summary() functions belong to a separate, internal package and are not part of the gofpdf library. If an error occurs at some point during the construction of the document, subsequent method calls exit immediately and the error is finally retrieved with the output call where it can be handled by the application.
gojson generates go struct defintions from JSON documents Reads from stdin and prints to stdout Example: curl -s https://api.github.com/repos/chimeracoder/gojson | gojson -name=Repository Output: package main type Repository struct { ArchiveURL string `json:"archive_url"` AssigneesURL string `json:"assignees_url"` BlobsURL string `json:"blobs_url"` BranchesURL string `json:"branches_url"` CloneURL string `json:"clone_url"` CollaboratorsURL string `json:"collaborators_url"` CommentsURL string `json:"comments_url"` CommitsURL string `json:"commits_url"` CompareURL string `json:"compare_url"` ContentsURL string `json:"contents_url"` ContributorsURL string `json:"contributors_url"` CreatedAt string `json:"created_at"` DefaultBranch string `json:"default_branch"` Description string `json:"description"` DownloadsURL string `json:"downloads_url"` EventsURL string `json:"events_url"` Fork bool `json:"fork"` Forks float64 `json:"forks"` ForksCount float64 `json:"forks_count"` ForksURL string `json:"forks_url"` FullName string `json:"full_name"` GitCommitsURL string `json:"git_commits_url"` GitRefsURL string `json:"git_refs_url"` GitTagsURL string `json:"git_tags_url"` GitURL string `json:"git_url"` HasDownloads bool `json:"has_downloads"` HasIssues bool `json:"has_issues"` HasWiki bool `json:"has_wiki"` Homepage interface{} `json:"homepage"` HooksURL string `json:"hooks_url"` HtmlURL string `json:"html_url"` ID float64 `json:"id"` IssueCommentURL string `json:"issue_comment_url"` IssueEventsURL string `json:"issue_events_url"` IssuesURL string `json:"issues_url"` KeysURL string `json:"keys_url"` LabelsURL string `json:"labels_url"` Language string `json:"language"` LanguagesURL string `json:"languages_url"` MasterBranch string `json:"master_branch"` MergesURL string `json:"merges_url"` MilestonesURL string `json:"milestones_url"` MirrorURL interface{} `json:"mirror_url"` Name string `json:"name"` NetworkCount float64 `json:"network_count"` NotificationsURL string `json:"notifications_url"` OpenIssues float64 `json:"open_issues"` OpenIssuesCount float64 `json:"open_issues_count"` Owner struct { AvatarURL string `json:"avatar_url"` EventsURL string `json:"events_url"` FollowersURL string `json:"followers_url"` FollowingURL string `json:"following_url"` GistsURL string `json:"gists_url"` GravatarID string `json:"gravatar_id"` HtmlURL string `json:"html_url"` ID float64 `json:"id"` Login string `json:"login"` OrganizationsURL string `json:"organizations_url"` ReceivedEventsURL string `json:"received_events_url"` ReposURL string `json:"repos_url"` SiteAdmin bool `json:"site_admin"` StarredURL string `json:"starred_url"` SubscriptionsURL string `json:"subscriptions_url"` Type string `json:"type"` URL string `json:"url"` } ` json:"owner"` Private bool `json:"private"` PullsURL string `json:"pulls_url"` PushedAt string `json:"pushed_at"` Size float64 `json:"size"` SshURL string `json:"ssh_url"` StargazersURL string `json:"stargazers_url"` StatusesURL string `json:"statuses_url"` SubscribersURL string `json:"subscribers_url"` SubscriptionURL string `json:"subscription_url"` SvnURL string `json:"svn_url"` TagsURL string `json:"tags_url"` TeamsURL string `json:"teams_url"` TreesURL string `json:"trees_url"` UpdatedAt string `json:"updated_at"` URL string `json:"url"` Watchers float64 `json:"watchers"` WatchersCount float64 `json:"watchers_count"` }
Package dart contains a Dart generator for GraphQL Documents.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package gofpdf implements a PDF document generator with high level support for text, drawing and images.
Package comet implements a BM25-based full-text search index. WHAT IS BM25? BM25 (Best Matching 25) is a probabilistic ranking function used to estimate the relevance of documents to a given search query. It is one of the most widely used ranking functions in information retrieval. HOW BM25 WORKS: For a given query Q with terms {t1, t2, ..., tn} and document D: 1. Tokenizes and normalizes both query and documents using UAX#29 word segmentation 2. For each query term, calculates: 3. Final score is the sum of (IDF × TF) for all query terms KEY PARAMETERS: TIME COMPLEXITY: MEMORY REQUIREMENTS: - Stores inverted index (term -> docIDs) using roaring bitmaps for compression - Stores term frequencies (term -> docID -> count) - Stores document lengths and tokens (not full text) - Much more memory efficient than storing full document text GUARANTEES & TRADE-OFFS: ✓ Pros: ✗ Cons: WHEN TO USE: Use BM25 index when: 1. You need full-text search with relevance ranking 2. You want fast keyword-based search 3. Memory efficiency is important (vs storing full text) 4. You have your own document store and just need search Package comet provides a high-performance hybrid vector search library for Go. Comet combines multiple indexing strategies and search modalities into a unified, efficient package. It supports semantic search (vector embeddings), full-text search (BM25), metadata filtering, and hybrid search with score fusion. Comet is built for developers who want to understand how vector databases work from the inside out. It provides production-ready implementations of modern vector search algorithms with comprehensive documentation and examples. Create a vector index and perform similarity search: Comet provides five vector index implementations, each with different tradeoffs: FlatIndex: Brute-force exhaustive search with 100% recall. Best for small datasets (<10K vectors) or when perfect accuracy is required. HNSWIndex: Hierarchical graph-based search with 95-99% recall and O(log n) performance. Best for most production workloads (10K-10M vectors). IVFIndex: Inverted file index using k-means clustering with 85-95% recall. Best for large datasets (>100K vectors) with moderate accuracy requirements. PQIndex: Product quantization for massive memory compression (10-500x smaller) with 70-85% recall. Best for memory-constrained environments. IVFPQIndex: Combines IVF and PQ for maximum scalability with 70-90% recall. Best for billion-scale datasets. Three distance metrics are supported: Euclidean (L2): Measures absolute spatial distance. Use when magnitude matters. L2Squared: Squared Euclidean distance (faster, preserves ordering). Use for better performance when only relative distances matter. Cosine: Measures angular similarity, independent of magnitude. Use for normalized vectors like text embeddings. BM25-based full-text search with Unicode tokenization: Fast filtering using Roaring Bitmaps and Bit-Sliced Indexes: Combine vector, text, and metadata search with score fusion: When combining results from multiple search modalities, different fusion strategies are available: WeightedSumFusion: Linear combination with configurable weights ReciprocalRankFusion: Rank-based fusion (scale-independent, recommended) MaxFusion/MinFusion: Simple maximum or minimum across modalities All indexes support persistence: HNSW parameters for tuning search quality: IVF parameters for tuning speed/accuracy: All indexes are safe for concurrent use. Multiple goroutines can search simultaneously while one goroutine adds or removes vectors. Document Search: Use vector embeddings for semantic search in documentation, knowledge bases, or content management systems. Product Recommendations: Combine product image embeddings with metadata filters for personalized recommendations. Question Answering: Use hybrid search (vector + BM25) for retrieval-augmented generation (RAG) systems. Duplicate Detection: Use high-recall vector search to find near-duplicate documents or images. Multi-modal Search: Combine text, image embeddings, and structured metadata for comprehensive search experiences. Choose the right index type: Use appropriate distance metrics: Batch operations: Training indexes: Metadata filtering: For detailed API documentation, see the godoc comments on each type and function. For more examples and use cases, visit: https://github.com/wizenheimer/comet Package comet implements a k-Nearest Neighbors (kNN) flat index for similarity search. WHAT IS A FLAT INDEX? A flat index is the most naive and simple approach to similarity search. The term "flat" indicates that vectors are stored without any compression or transformation - they are stored "as-is" in their original form. This is also known as brute-force or exhaustive search. HOW kNN WORKS: For a given query vector Q, the algorithm: 1. Calculates the distance from Q to EVERY vector in the dataset 2. Sorts all distances 3. Returns the k vectors with the smallest distances TIME COMPLEXITY: MEMORY REQUIREMENTS: - 4 bytes per float32 component - Total per vector: 4 * d bytes (where d is the dimensionality) - No compression, so memory scales linearly with dataset size GUARANTEES & TRADE-OFFS: ✓ Pros: ✗ Cons: WHEN TO USE: Use flat index only when: 1. Dataset size or embedding dimensionality is relatively small 2. You MUST have 100% accuracy (e.g., fingerprint matching, security applications) 3. Speed is not a critical concern Package comet implements HNSW (Hierarchical Navigable Small World). WHAT IS HNSW? HNSW is a state-of-the-art graph-based algorithm for approximate nearest neighbor search. It builds a multi-layered graph where search is O(log n) - incredibly fast! Layer 2: Few nodes, long-range connections (highways) Layer 1: More nodes, medium-range connections (state roads) Layer 0: All nodes, short-range connections (local streets) Search starts at top layer and descends, getting more refined at each level! PERFORMANCE: TIME COMPLEXITY: Package comet implements a hybrid search index that combines vector, text, and metadata search. WHAT IS HYBRIDSEARCHINDEX? HybridSearchIndex is a facade that provides a unified interface over three specialized indexes: 1. VectorIndex: For semantic similarity search using vector embeddings 2. TextIndex: For keyword-based BM25 full-text search 3. MetadataIndex: For filtering by structured metadata attributes HOW IT WORKS: The index maintains three separate indexes internally and coordinates search across them. When searching, it follows this flow: 1. Apply metadata filters first (if any) to get candidate document IDs 2. Pass candidate IDs to vector and/or text search for relevance ranking 3. Combine results from multiple search modes using score aggregation SEARCH MODES: - Vector-only: Semantic similarity search using embeddings - Text-only: Keyword-based BM25 search - Metadata-only: Pure filtering without ranking - Hybrid: Combine any or all of the above with score aggregation WHEN TO USE: Use HybridSearchIndex when: 1. You need to combine multiple search modalities 2. You want to filter by metadata before expensive vector search 3. You need both semantic and keyword-based search 4. You want a simple unified API instead of managing multiple indexes Package comet implements a k-Nearest Neighbors (kNN) IVF index for similarity search. WHAT IS AN IVF INDEX? IVF (Inverted File Index) is a partitioning-based approximate nearest neighbor search algorithm. It divides the vector space into Voronoi cells using k-means clustering, then searches only the nearest cells instead of scanning all vectors. HOW IVF WORKS: Training Phase: 1. Run k-means on training vectors to learn nlist cluster centroids 2. These centroids define Voronoi partitions of the vector space Indexing Phase: 1. For each vector, find its nearest centroid 2. Add the vector to that centroid's inverted list Search Phase: 1. Find the nprobe nearest centroids to the query vector 2. Search only the vectors in those nprobe inverted lists 3. Return the top-k nearest neighbors from candidates TIME COMPLEXITY: MEMORY REQUIREMENTS: - Vectors: 4 × n × dim bytes (stored as-is) - Centroids: 4 × nlist × dim bytes - Lists: negligible overhead (just pointers) - Total: ~4 × (n + nlist) × dim bytes ACCURACY VS SPEED TRADEOFF: - nprobe = 1: Fastest, lowest recall (~30-50%) - nprobe = sqrt(nlist): Good balance (~70-90% recall) - nprobe = nlist: Same as flat search (100% recall) CHOOSING NLIST: Rule of thumb: nlist = sqrt(n) or nlist = 4*sqrt(n) - For 1M vectors: nlist = 1,000 to 4,000 - For 100K vectors: nlist = 316 to 1,264 - For 10K vectors: nlist = 100 to 400 WHEN TO USE IVF: Use IVF when: 1. Dataset is large (>10K vectors) 2. You can tolerate ~90-95% recall (not 100%) 3. You want 10-100x speedup over flat search 4. Memory usage is not a primary concern DON'T use IVF when: 1. Dataset is small (<10K vectors) - use flat index 2. You need 100% recall - use flat index 3. Memory is very limited - use PQ or IVFPQ Package comet implements IVFPQ (Inverted File with Product Quantization). WHAT IS IVFPQ? IVFPQ combines IVF (scope reduction) with PQ (compression) to create one of the most powerful similarity search algorithms. It's the workhorse of large-scale vector search systems. RESIDUAL VECTORS IVFPQ encodes RESIDUALS (vector - centroid) instead of original vectors. This dramatically improves compression quality because: PERFORMANCE: TIME COMPLEXITY: Package comet implements a metadata filtering index for vector search. WHAT IS A METADATA INDEX? A metadata index enables fast filtering of documents based on structured metadata attributes before performing expensive vector similarity searches. This dramatically improves search performance by reducing the candidate set. HOW IT WORKS: The index uses two specialized data structures: 1. Roaring Bitmaps: For categorical fields (strings, booleans) 2. Bit-Sliced Index (BSI): For numeric fields (integers, floats) QUERY TYPES: - Equality: field = value - Inequality: field != value - Comparisons: field > value, field >= value, field < value, field <= value - Range: field BETWEEN min AND max - Set membership: field IN (val1, val2, val3) - Set exclusion: field NOT IN (val1, val2) - Existence: field EXISTS, field NOT EXISTS TIME COMPLEXITY: MEMORY REQUIREMENTS: - Roaring bitmaps: Highly compressed, typically 1-10% of uncompressed size - BSI: ~64 bits per numeric value (compressed with roaring) - Much more efficient than traditional B-tree indexes for high-cardinality data GUARANTEES & TRADE-OFFS: ✓ Pros: ✗ Cons: WHEN TO USE: Use metadata index when: 1. Pre-filtering documents before vector search 2. Need to filter by structured attributes (price, date, category, etc.) 3. Working with large datasets (100K+ documents) 4. Need sub-millisecond filter performance Package comet implements Product Quantization (PQ) for similarity search. WHAT IS PRODUCT QUANTIZATION? PQ is a lossy compression technique that dramatically reduces memory usage for vector storage while enabling approximate similarity search. It achieves compression ratios of 10-500x by dividing vectors into subspaces and quantizing each independently. THE CORE IDEA - DIVIDE AND COMPRESS: Instead of storing full high-dimensional vectors: 1. Divide each vector into M equal-sized subvectors (subspaces) 2. Learn a codebook of K centroids for each subspace via k-means 3. Encode each subvector with the ID of its nearest centroid 4. Store only these compact codes instead of original vectors COMPRESSION EXAMPLE: Original: 768 dims × 4 bytes = 3,072 bytes PQ (M=8, K=256): 8 subspaces × 1 byte = 8 bytes Compression: 384x smaller! TIME COMPLEXITY: WHEN TO USE PQ: - Dataset too large for RAM - Can tolerate 85-95% recall - L2 or inner product metric - Want massive compression
Package gendoc is a protoc plugin for generating documentation from your proto files.
Command json-to-struct generates Go struct definitions from JSON documents.
Package llmschema extracts database schemas and generates markdown documentation optimized for AI agent consumption.
Package gendoc is a protoc plugin for generating documentation from your proto files.
Command md-ld generates placeholders for all unmatched link references in a Markdown document.
Package skipper provides an HTTP routing library with flexible configuration as well as a runtime update of the routing rules. Skipper works as an HTTP reverse proxy that is responsible for mapping incoming requests to multiple HTTP backend services, based on routes that are selected by the request attributes. At the same time, both the requests and the responses can be augmented by a filter chain that is specifically defined for each route. Optionally, it can provide circuit breaker mechanism individually for each backend host. Skipper can load and update the route definitions from multiple data sources without being restarted. It provides a default executable command with a few built-in filters, however, its primary use case is to be extended with custom filters, predicates or data sources. For further information read 'Extending Skipper'. Skipper took the core design and inspiration from Vulcand: https://github.com/mailgun/vulcand. Skipper is 'go get' compatible. If needed, create a 'go workspace' first: Get the Skipper packages: Create a file with a route: Optionally, verify the syntax of the file: Start Skipper and make an HTTP request: The core of Skipper's request processing is implemented by a reverse proxy in the 'proxy' package. The proxy receives the incoming request, forwards it to the routing engine in order to receive the most specific matching route. When a route matches, the request is forwarded to all filters defined by it. The filters can modify the request or execute any kind of program logic. Once the request has been processed by all the filters, it is forwarded to the backend endpoint of the route. The response from the backend goes once again through all the filters in reverse order. Finally, it is mapped as the response of the original incoming request. Besides the default proxying mechanism, it is possible to define routes without a real network backend endpoint. One of these cases is called a 'shunt' backend, in which case one of the filters needs to handle the request providing its own response (e.g. the 'static' filter). Actually, filters themselves can instruct the request flow to shunt by calling the Serve(*http.Response) method of the filter context. Another case of a route without a network backend is the 'loopback'. A loopback route can be used to match a request, modified by filters, against the lookup tree with different conditions and then execute a different route. One example scenario can be to use a single route as an entry point to execute some calculation to get an A/B testing decision and then matching the updated request metadata for the actual destination route. This way the calculation can be executed for only those requests that don't contain information about a previously calculated decision. For further details, see the 'proxy' and 'filters' package documentation. Finding a request's route happens by matching the request attributes to the conditions in the route's definitions. Such definitions may have the following conditions: - method - path (optionally with wildcards) - path regular expressions - host regular expressions - headers - header regular expressions It is also possible to create custom predicates with any other matching criteria. The relation between the conditions in a route definition is 'and', meaning, that a request must fulfill each condition to match a route. For further details, see the 'routing' package documentation. Filters are applied in order of definition to the request and in reverse order to the response. They are used to modify request and response attributes, such as headers, or execute background tasks, like logging. Some filters may handle the requests without proxying them to service backends. Filters, depending on their implementation, may accept/require parameters, that are set specifically to the route. For further details, see the 'filters' package documentation. Each route has one of the following backends: HTTP endpoint, shunt or loopback. Backend endpoints can be any HTTP service. They are specified by their network address, including the protocol scheme, the domain name or the IP address, and optionally the port number: e.g. "https://www.example.org:4242". (The path and query are sent from the original request, or set by filters.) A shunt route means that Skipper handles the request alone and doesn't make requests to a backend service. In this case, it is the responsibility of one of the filters to generate the response. A loopback route executes the routing mechanism on current state of the request from the start, including the route lookup. This way it serves as a form of an internal redirect. Route definitions consist of the following: - request matching conditions (predicates) - filter chain (optional) - backend (either an HTTP endpoint or a shunt) The eskip package implements the in-memory and text representations of route definitions, including a parser. (Note to contributors: in order to stay compatible with 'go get', the generated part of the parser is stored in the repository. When changing the grammar, 'go generate' needs to be executed explicitly to update the parser.) For further details, see the 'eskip' package documentation Skipper has filter implementations of basic auth and OAuth2. It can be integrated with tokeninfo based OAuth2 providers. For details, see: https://godoc.org/github.com/zalando/skipper/filters/auth. Skipper's route definitions of Skipper are loaded from one or more data sources. It can receive incremental updates from those data sources at runtime. It provides three different data clients: - Kubernetes: Skipper can be used as part of a Kubernetes Ingress Controller implementation together with https://github.com/zalando-incubator/kube-ingress-aws-controller . In this scenario, Skipper uses the Kubernetes API's Ingress extensions as a source for routing. For a complete deployment example, see more details in: https://github.com/zalando-incubator/kubernetes-on-aws/ . - Innkeeper: the Innkeeper service implements a storage for large sets of Skipper routes, with an HTTP+JSON API, OAuth2 authentication and role management. See the 'innkeeper' package and https://github.com/zalando/innkeeper. - etcd: Skipper can load routes and receive updates from etcd clusters (https://github.com/coreos/etcd). See the 'etcd' package. - static file: package eskipfile implements a simple data client, which can load route definitions from a static file in eskip format. Currently, it loads the routes on startup. It doesn't support runtime updates. Skipper can use additional data sources, provided by extensions. Sources must implement the DataClient interface in the routing package. Skipper provides circuit breakers, configured either globally, based on backend hosts or based on individual routes. It supports two types of circuit breaker behavior: open on N consecutive failures, or open on N failures out of M requests. For details, see: https://godoc.org/github.com/zalando/skipper/circuit. Skipper can be started with the default executable command 'skipper', or as a library built into an application. The easiest way to start Skipper as a library is to execute the 'Run' function of the current, root package. Each option accepted by the 'Run' function is wired in the default executable as well, as a command line flag. E.g. EtcdUrls becomes -etcd-urls as a comma separated list. For command line help, enter: An additional utility, eskip, can be used to verify, print, update and delete routes from/to files or etcd (Innkeeper on the roadmap). See the cmd/eskip command package, and/or enter in the command line: Skipper doesn't use dynamically loaded plugins, however, it can be used as a library, and it can be extended with custom predicates, filters and/or custom data sources. To create a custom predicate, one needs to implement the PredicateSpec interface in the routing package. Instances of the PredicateSpec are used internally by the routing package to create the actual Predicate objects as referenced in eskip routes, with concrete arguments. Example, randompredicate.go: In the above example, a custom predicate is created, that can be referenced in eskip definitions with the name 'Random': To create a custom filter we need to implement the Spec interface of the filters package. 'Spec' is the specification of a filter, and it is used to create concrete filter instances, while the raw route definitions are processed. Example, hellofilter.go: The above example creates a filter specification, and in the routes where they are included, the filter instances will set the 'X-Hello' header for each and every response. The name of the filter is 'hello', and in a route definition it is referenced as: The easiest way to create a custom Skipper variant is to implement the required filters (as in the example above) by importing the Skipper package, and starting it with the 'Run' command. Example, hello.go: A file containing the routes, routes.eskip: Start the custom router: The 'Run' function in the root Skipper package starts its own listener but it doesn't provide the best composability. The proxy package, however, provides a standard http.Handler, so it is possible to use it in a more complex solution as a building block for routing. Skipper provides detailed logging of failures, and access logs in Apache log format. Skipper also collects detailed performance metrics, and exposes them on a separate listener endpoint for pulling snapshots. For details, see the 'logging' and 'metrics' packages documentation. The router's performance depends on the environment and on the used filters. Under ideal circumstances, and without filters, the biggest time factor is the route lookup. Skipper is able to scale to thousands of routes with logarithmic performance degradation. However, this comes at the cost of increased memory consumption, due to storing the whole lookup tree in a single structure. Benchmarks for the tree lookup can be run by: In case more aggressive scale is needed, it is possible to setup Skipper in a cascade model, with multiple Skipper instances for specific route segments.