High Performance, Feature-Rich Idiomatic Go encoding library for msgpack and binc . Supported Serialization formats are: To install: The idiomatic Go support is as seen in other encoding packages in the standard library (ie json, xml, gob, etc). Rich Feature Set includes: Users can register a function to handle the encoding or decoding of their custom types. There are no restrictions on what the custom type can be. Some examples: As an illustration, MyStructWithUnexportedFields would normally be encoded as an empty map because it has no exported fields, while UUID would be encoded as a string. However, with extension support, you can encode any of these however you like. RPC Client and Server Codecs are implemented, so the codecs can be used with the standard net/rpc package. Typical usage model: Run the benchmark suite using: To run full benchmark suite (including against vmsgpack and bson), see notes in ext_dep_test.go Msgpack-c implementation powers the c, c++, python, ruby, etc libraries. We need to maintain compatibility with it and how it encodes integer values without caring about the type. For compatibility with behaviour of msgpack-c reference implementation:
Package pixfont is a simple pixel font library that allows for text drawing in the standard image and image/draw packages. It is very useful when heavyweight TrueType fonts and the freetype-go package are not available or desired. Since fonts are simple bitmap images, scaling is not possible and aliasing will occur. See the included fontgen tool if you wish to convert or include your own pixel font in your project.
Package logrus is a structured logger for Go, completely API compatible with the standard library logger. The simplest way to use Logrus is simply the package-level exported logger: Output: For a full guide visit https://github.com/sirupsen/logrus
Package hackpadfs defines many essential file and file system interfaces as well as helpers for use with the standard library's 'io/fs' package.
Package sdk is the official AWS SDK for the Go programming language. The AWS SDK for Go provides APIs and utilities that developers can use to build Go applications that use AWS services, such as Amazon Simple Storage Service (Amazon S3). The SDK removes the complexity of coding directly against a web service interface. It hides a lot of the lower-level plumbing, such as authentication, request retries, and error handling. The SDK also includes helpful utilities on top of the AWS APIs that add additional capabilities and functionality. For example, the Amazon S3 Download and Upload Manager will automatically split up large objects into multiple parts and transfer them concurrently. See the s3manager package documentation for more information. https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/ Checkout the Getting Started Guide and API Reference Docs detailed the SDK's components and details on each AWS client the SDK supports. The Getting Started Guide provides examples and detailed description of how to get setup with the SDK. https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/welcome.html The API Reference Docs include a detailed breakdown of the SDK's components such as utilities and AWS clients. Use this as a reference of the Go types included with the SDK, such as AWS clients, API operations, and API parameters. https://docs.aws.amazon.com/sdk-for-go/api/ The SDK is composed of two main components, SDK core, and service clients. The SDK core packages are all available under the aws package at the root of the SDK. Each client for a supported AWS service is available within its own package under the service folder at the root of the SDK. aws - SDK core, provides common shared types such as Config, Logger, and utilities to make working with API parameters easier. awserr - Provides the error interface that the SDK will use for all errors that occur in the SDK's processing. This includes service API response errors as well. The Error type is made up of a code and message. Cast the SDK's returned error type to awserr.Error and call the Code method to compare returned error to specific error codes. See the package's documentation for additional values that can be extracted such as RequestId. credentials - Provides the types and built in credentials providers the SDK will use to retrieve AWS credentials to make API requests with. Nested under this folder are also additional credentials providers such as stscreds for assuming IAM roles, and ec2rolecreds for EC2 Instance roles. endpoints - Provides the AWS Regions and Endpoints metadata for the SDK. Use this to lookup AWS service endpoint information such as which services are in a region, and what regions a service is in. Constants are also provided for all region identifiers, e.g UsWest2RegionID for "us-west-2". session - Provides initial default configuration, and load configuration from external sources such as environment and shared credentials file. request - Provides the API request sending, and retry logic for the SDK. This package also includes utilities for defining your own request retryer, and configuring how the SDK processes the request. service - Clients for AWS services. All services supported by the SDK are available under this folder. The SDK includes the Go types and utilities you can use to make requests to AWS service APIs. Within the service folder at the root of the SDK you'll find a package for each AWS service the SDK supports. All service clients follows a common pattern of creation and usage. When creating a client for an AWS service you'll first need to have a Session value constructed. The Session provides shared configuration that can be shared between your service clients. When service clients are created you can pass in additional configuration via the aws.Config type to override configuration provided by in the Session to create service client instances with custom configuration. Once the service's client is created you can use it to make API requests the AWS service. These clients are safe to use concurrently. In the AWS SDK for Go, you can configure settings for service clients, such as the log level and maximum number of retries. Most settings are optional; however, for each service client, you must specify a region and your credentials. The SDK uses these values to send requests to the correct AWS region and sign requests with the correct credentials. You can specify these values as part of a session or as environment variables. See the SDK's configuration guide for more information. https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html See the session package documentation for more information on how to use Session with the SDK. https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ See the Config type in the aws package for more information on configuration options. https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config When using the SDK you'll generally need your AWS credentials to authenticate with AWS services. The SDK supports multiple methods of supporting these credentials. By default the SDK will source credentials automatically from its default credential chain. See the session package for more information on this chain, and how to configure it. The common items in the credential chain are the following: Environment Credentials - Set of environment variables that are useful when sub processes are created for specific roles. Shared Credentials file (~/.aws/credentials) - This file stores your credentials based on a profile name and is useful for local development. Credentials can be configured in code as well by setting the Config's Credentials value to a custom provider or using one of the providers included with the SDK to bypass the default credential chain and use a custom one. This is helpful when you want to instruct the SDK to only use a specific set of credentials or providers. This example creates a credential provider for assuming an IAM role, "myRoleARN" and configures the S3 service client to use that role for API requests. The SDK has support for the shared configuration file (~/.aws/config). This support can be enabled by setting the environment variable, "AWS_SDK_LOAD_CONFIG=1", or enabling the feature in code when creating a Session via the Option's SharedConfigState parameter. In addition to the credentials you'll need to specify the region the SDK will use to make AWS API requests to. In the SDK you can specify the region either with an environment variable, or directly in code when a Session or service client is created. The last value specified in code wins if the region is specified multiple ways. To set the region via the environment variable set the "AWS_REGION" to the region you want to the SDK to use. Using this method to set the region will allow you to run your application in multiple regions without needing additional code in the application to select the region. The endpoints package includes constants for all regions the SDK knows. The values are all suffixed with RegionID. These values are helpful, because they reduce the need to type the region string manually. To set the region on a Session use the aws package's Config struct parameter Region to the AWS region you want the service clients created from the session to use. This is helpful when you want to create multiple service clients, and all of the clients make API requests to the same region. In addition to setting the region when creating a Session you can also set the region on a per service client bases. This overrides the region of a Session. This is helpful when you want to create service clients in specific regions different from the Session's region. See the Config type in the aws package for more information and additional options such as setting the Endpoint, and other service client configuration options. https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config Once the client is created you can make an API request to the service. Each API method takes a input parameter, and returns the service response and an error. The SDK provides methods for making the API call in multiple ways. In this list we'll use the S3 ListObjects API as an example for the different ways of making API requests. ListObjects - Base API operation that will make the API request to the service. ListObjectsRequest - API methods suffixed with Request will construct the API request, but not send it. This is also helpful when you want to get a presigned URL for a request, and share the presigned URL instead of your application making the request directly. ListObjectsPages - Same as the base API operation, but uses a callback to automatically handle pagination of the API's response. ListObjectsWithContext - Same as base API operation, but adds support for the Context pattern. This is helpful for controlling the canceling of in flight requests. See the Go standard library context package for more information. This method also takes request package's Option functional options as the variadic argument for modifying how the request will be made, or extracting information from the raw HTTP response. ListObjectsPagesWithContext - same as ListObjectsPages, but adds support for the Context pattern. Similar to ListObjectsWithContext this method also takes the request package's Option function option types as the variadic argument. In addition to the API operations the SDK also includes several higher level methods that abstract checking for and waiting for an AWS resource to be in a desired state. In this list we'll use WaitUntilBucketExists to demonstrate the different forms of waiters. WaitUntilBucketExists. - Method to make API request to query an AWS service for a resource's state. Will return successfully when that state is accomplished. WaitUntilBucketExistsWithContext - Same as WaitUntilBucketExists, but adds support for the Context pattern. In addition these methods take request package's WaiterOptions to configure the waiter, and how underlying request will be made by the SDK. The API method will document which error codes the service might return for the operation. These errors will also be available as const strings prefixed with "ErrCode" in the service client's package. If there are no errors listed in the API's SDK documentation you'll need to consult the AWS service's API documentation for the errors that could be returned. Pagination helper methods are suffixed with "Pages", and provide the functionality needed to round trip API page requests. Pagination methods take a callback function that will be called for each page of the API's response. Waiter helper methods provide the functionality to wait for an AWS resource state. These methods abstract the logic needed to to check the state of an AWS resource, and wait until that resource is in a desired state. The waiter will block until the resource is in the state that is desired, an error occurs, or the waiter times out. If a resource times out the error code returned will be request.WaiterResourceNotReadyErrorCode. This example shows a complete working Go file which will upload a file to S3 and use the Context pattern to implement timeout logic that will cancel the request if it takes too long. This example highlights how to use sessions, create a service client, make a request, handle the error, and process the response.
Package trayhost is a cross-platform Go library to place an icon in the host operating system's taskbar. - macOS - Fully implemented and supported by @dmitshur. - Linux - Partially implemented, but unsupported (needs an owner/maintainer). - Windows - Partially implemented, but unsupported (needs an owner/maintainer). On macOS, for Notification Center user notifications to work, your Go binary that uses trayhost must be a part of a standard macOS app bundle. Most other functionality of trayhost will be available if the binary is not a part of app bundle, but you will get a terminal pop up, and you will not be able to configure some aspects of the app. Here's a minimal layout of an app bundle: Here's a minimal Info.plist file as reference (only the entries that are needed, nothing extra): - CFBundleIdentifier needs to be set to some value for Notification Center to work. - The binary must be inside Contents/MacOS directory for Notification Center to work. - NSHighResolutionCapable to enable Retina mode. - LSUIElement is needed to make the app not appear in Cmd+Tab list and the dock while still being able to show a tooltip in the menu bar. On macOS, when you run an app bundle, the working directory of the executed process is the root directory (/), not the app bundle's Contents/Resources directory. Change directory to Resources if you need to load resources from there.
Package gore is a library for analyzing Go binaries. The library has functionality for guessing the compiler version used. It searches the binary for the identifiable string left by the compiler. It is not perfect, so functionality for assuming a specific version is also provided. The version strings used are identical to the identifiers. For example version "1.10.1" is represented as "go1.10.1" and version "1.10" is represented as "go1.10" Function information is recovered from the pclntab. Information that is recovered includes: function start and end location in the text section, source file. The library also tries to estimate the first and last line number in the source file for the function. The methods recovered includes the receiver. All functions and methods belong to a package. The library tries to classify the type of package. If it is a standard library package, 3rd-party package, or part of the main application. If it unable to classify the package, it is classified as unknown. The types in the binary are parsed from the "typelink" list. Not all versions of Go are supported equally. Versions 1.7 and later are fully supported. Versions 1.5 and 1.6 are partially supported. Version prior to 1.5 are not supported at all at the moment. Extract the main package, child packages, and sibling packages: Extract all the types in the binary:
Package siris is a fully-featured HTTP/2 backend web framework written entirely in Google’s Go Language. Source code and other details for the project are available at GitHub: The only requirement is the Go Programming Language, at least version 1.8 Example code: Access to all hosts that serve your application can be provided by the `Application#Hosts` field, after the `Run` method. But the most common scenario is that you may need access to the host before the `Run` method, there are two ways of gain access to the host supervisor, read below. First way is to use the `app.NewHost` to create a new host and use one of its `Serve` or `Listen` functions to start the application via the `siris#Raw` Runner. Note that this way needs an extra import of the `net/http` package. Example Code: Second, and probably easier way is to use the `host.Configurator`. Note that this method requires an extra import statement of "github.com/go-siris/siris/core/host" when using go < 1.9, if you're targeting on go1.9 then you can use the `siris#Supervisor` and omit the extra host import. All common `Runners` we saw earlier (`siris#Addr, siris#Listener, siris#Server, siris#TLS, siris#AutoTLS`) accept a variadic argument of `host.Configurator`, there are just `func(*host.Supervisor)`. Therefore the `Application` gives you the rights to modify the auto-created host supervisor through these. Example Code: All HTTP methods are supported, developers can also register handlers for same paths for different methods. The first parameter is the HTTP Method, second parameter is the request path of the route, third variadic parameter should contains one or more context.Handler executed by the registered order when a user requests for that specific resouce path from the server. Example code: In order to make things easier for the user, Siris provides functions for all HTTP Methods. The first parameter is the request path of the route, second variadic parameter should contains one or more context.Handler executed by the registered order when a user requests for that specific resouce path from the server. Example code: A set of routes that are being groupped by path prefix can (optionally) share the same middleware handlers and template layout. A group can have a nested group too. `.Party` is being used to group routes, developers can declare an unlimited number of (nested) groups. Example code: Siris developers are able to register their own handlers for http statuses like 404 not found, 500 internal server error and so on. Example code: With the help of Siris's expressionist router you can build any form of API you desire, with safety. Example code: At the previous example, we've seen static routes, group of routes, subdomains, wildcard subdomains, a small example of parameterized path with a single known paramete and custom http errors, now it's time to see wildcard parameters and macros. Siris, like net/http std package registers route's handlers by a Handler, the Siris' type of handler is just a func(ctx context.Context) where context comes from github.com/go-siris/siris/context. Until go 1.9 you will have to import that package too, after go 1.9 this will be not be necessary. Siris has the easiest and the most powerful routing process you have ever meet. At the same time, Siris has its own interpeter(yes like a programming language) for route's path syntax and their dynamic path parameters parsing and evaluation, I am calling them "macros" for shortcut. How? It calculates its needs and if not any special regexp needed then it just registers the route with the low-level path syntax, otherwise it pre-compiles the regexp and adds the necessary middleware(s). Standard macro types for parameters: if type is missing then parameter's type is defaulted to string, so {param} == {param:string}. If a function not found on that type then the "string"'s types functions are being used. i.e: Besides the fact that Siris provides the basic types and some default "macro funcs" you are able to register your own too!. Register a named path parameter function: at the func(argument ...) you can have any standard type, it will be validated before the server starts so don't care about performance here, the only thing it runs at serve time is the returning func(paramValue string) bool. Example code: A path parameter name should contain only alphabetical letters, symbols, containing '_' and numbers are NOT allowed. If route failed to be registered, the app will panic without any warnings if you didn't catch the second return value(error) on .Handle/.Get.... Last, do not confuse ctx.Values() with ctx.Params(). Path parameter's values goes to ctx.Params() and context's local storage that can be used to communicate between handlers and middleware(s) goes to ctx.Values(), path parameters and the rest of any custom values are separated for your own good. Run Static Files Example code: More examples can be found here: https://github.com/go-siris/siris/tree/master/_examples/beginner/file-server Middleware is just a concept of ordered chain of handlers. Middleware can be registered globally, per-party, per-subdomain and per-route. Example code: Siris is able to wrap and convert any external, third-party Handler you used to use to your web application. Let's convert the https://github.com/rs/cors net/http external middleware which returns a `next form` handler. Example code: Siris supports 5 template engines out-of-the-box, developers can still use any external golang template engine, as `context.ResponseWriter()` is an `io.Writer`. All of these five template engines have common features with common API, like Layout, Template Funcs, Party-specific layout, partial rendering and more. Example code: View engine supports bundled(https://github.com/jteeuwen/go-bindata) template files too. go-bindata gives you two functions, asset and assetNames, these can be set to each of the template engines using the `.Binary` func. Example code: A real example can be found here: https://github.com/go-siris/siris/tree/master/_examples/intermediate/view/embedding-templates-into-app. Enable auto-reloading of templates on each request. Useful while developers are in dev mode as they no neeed to restart their app on every template edit. Example code: Each one of these template engines has different options located here: https://github.com/go-siris/siris/tree/master/view . This example will show how to store and access data from a session. You don’t need any third-party library, but If you want you can use any session manager compatible or not. In this example we will only allow authenticated users to view our secret message on the /secret page. To get access to it, the will first have to visit /login to get a valid session cookie, which logs him in. Additionally he can visit /logout to revoke his access to our secret message. Example code: Running the example: But you should have a basic idea of the framework by now, we just scratched the surface. If you enjoy what you just saw and want to learn more, please follow the below links: Examples: Built'n Middleware: Community Middleware: Home Page:
Grohl is an opinionated library for gathering metrics and data about how your applications are running in production. It does this through writing logs in a key=value structure. It also provides interfaces for sending stacktraces or metrics to external services. This is a Go version of https://github.com/asenchi/scrolls. The name for this library came from mashing the words "go" and "scrolls" together. Also, Dave Grohl (lead singer of Foo Fighters) is passionate about event driven metrics. Grohl treats logs as the central authority for how an application is behaving. Logs are written in a key=value structure so that they are easily parsed. If you use a set of common log keys, you can relate events from various services together. Here's an example log that you might write: The output would look something like: Note: Other examples leave out the "now" keyword for clarity. A *grohl.Context stores a map of keys and values that are written with every log message. You can set common keys for every request, or create a new context per new request or connection. You can add more context to the example above by setting up the app name and deployed environment. This changes the output from above to: You can also create scoped Context objects. For instance, a network server may want a scoped Context for each request or connection. This is the output (taking the global context above into consideration): As you can see we have some standard nomenclature around logging. Here's a cheat sheet for some of the methods we use: By default, all *grohl.Context objects write to STDOUT. Grohl includes support for both io and channel loggers. If you are writing to *grohl.Context objects in separate go routines, a channel logger can be used for concurrency. Grohl provides a grohl.Statter interface based on https://github.com/peterbourgon/g2s: Without any setup, this outputs: If you import "github.com/peterbourgon/g2s", you can dial into a statsd server over a udp socket: Once being set up, the statter functions above will not output any logs. Grohl makes it easy to measure the run time of a portion of code. This would output: You can change the time unit that Grohl uses to "milliseconds" (the default is "seconds"): You can also write to a custom Statter: You can also set all *grohl.Timer objects to use the same statter. Grohl can report Go errors: Without any ErrorReporter set, this logs the following: You can set the default ErrorReporter too:
Package imageorient provides image decoding functions similar to standard library's image.Decode and image.DecodeConfig with the addition that they also handle the EXIF orientation tag (if present). See also: http://www.daveperrett.com/articles/2012/07/28/exif-orientation-handling-is-a-ghetto/
Package goaction enables writing Github Actions in Go. The idea is: write a standard Go script, one that works with `go run`, and use it as Github action. The script's inputs - flags and environment variables, are set though the Github action API. This project will generate all the required files for the script (This generation can be done automattically with Github action integration). The library also exposes neat API to get workflow information. - [x] Write a Go script. - [x] Add `goaction` configuration in `.github/workflows/goaction.yml`. - [x] Push the project to Github. See simplest example for a Goaction script: (posener/goaction-example) https://github.com/posener/goaction-example, or an example that demonstrait using Github APIs: (posener/goaction-issues-example) https://github.com/posener/goaction-issues-example. Write Github Action by writing Go code! Just start a Go module with a main package, and execute it as a Github action using Goaction, or from the command line using `go run`. A go executable can get inputs from the command line flag and from environment variables. Github actions should have a `action.yml` file that defines this API. Goaction bridges the gap by parsing the Go code and creating this file automatically for you. The main package inputs should be defined with the standard `flag` package for command line arguments, or by `os.Getenv` for environment variables. These inputs define the API of the program and `goaction` automatically detect them and creates the `action.yml` file from them. Additionally, goaction also provides a library that exposes all Github action environment in an easy-to-use API. See the documentation for more information. Code segments which should run only in Github action (called "CI mode"), and not when the main package runs as a command line tool, should be protected by a `if goaction.CI { ... }` block. In order to convert the repository to a Github action, goaction command line should run on the **"main file"** (described above). This command can run manually (by ./cmd/goaction) but luckily `goaction` also comes as a Github action :-) Goaction Github action keeps the Github action file updated according to the main Go file automatically. When a PR is made, goaction will post a review explaining what changes to expect. When a new commit is pushed, Goaction makes sure that the Github action files are updated if needed. Add the following content to `.github/workflows/goaction.yml` ./action.yml: A "metadata" file for Github actions. If this file exists, the repository is considered as Github action, and the file contains information that instructs how to invoke this action. See (metadata syntax) https://help.github.com/en/actions/building-actions/metadata-syntax-for-github-actions. for more info. ./Dockerfile: A file that contains instructions how to build a container, that is used for Github actions. Github action uses this file in order to create a container image to the action. The container can also be built and tested manually: Goaction parses Go script file and looks for annotations that extends the information that exists in the function calls. Goaction annotations are a comments that start with `//goaction:` (no space after slashes). They can only be set on a `var` definition. The following annotations are available: * `//goaction:required` - sets an input definition to be "required". * `//goaction:skip` - skips an input out output definition. * `//goaction:description <description>` - add description for `os.Getenv`. * `//goaction:default <value>` - add default value for `os.Getenv`. A list of projects which are using Goaction (please send a PR if your project uses goaction and does not appear her). * (posener/goreadme) http://github.com/posener/goreadme
Package gcpjwt has Google Cloud Platform (Cloud KMS, IAM API, & AppEngine App Identity API) jwt-go implementations. Should work across virtually all environments, on or off of Google's Cloud Platform. It is highly recommended that you override the default algorithm implementations that you want to leverage a GCP service for in dgrijalva/jwt-go. You otherwise will have to manually pick the verification method for your JWTs and they will place non-standard headers in the rendered JWT (with the exception of signJwt from the IAM API which overwrites the header with its own). You should only need to override the algorithm(s) you plan to use. It is also incorrect to override overlapping, algorithms such as `gcpjwt.SigningMethodKMSRS256.Override()` and `gcpjwt.SigningMethodIAMJWT.Override()` Example: As long as a you override a default algorithm implementation as shown above, using the dgrijalva/jwt-go is mostly unchanged. Token creation is more/less done the same way as in the dgrijalva/jwt-go package. The key that you need to provide is always going to be a context.Context, usuaully with a configuration object loaded in: Example: Finally, the steps to validate a token should be straight forward. This library provides you with helper jwt.Keyfunc implementations to do the heavy lifting around getting the public certificates for verification: Example:
Package kyber provides a toolbox of advanced cryptographic primitives, for applications that need more than straightforward signing and encryption. This top level package defines the interfaces to cryptographic primitives designed to be independent of specific cryptographic algorithms, to facilitate upgrading applications to new cryptographic algorithms or switching to alternative algorithms for experimentation purposes. This toolkits public-key crypto API includes a kyber.Group interface supporting a broad class of group-based public-key primitives including DSA-style integer residue groups and elliptic curve groups. Users of this API can write higher-level crypto algorithms such as zero-knowledge proofs without knowing or caring exactly what kind of group, let alone which precise security parameters or elliptic curves, are being used. The kyber.Group interface supports the standard algebraic operations on group elements and scalars that nontrivial public-key algorithms tend to rely on. The interface uses additive group terminology typical for elliptic curves, such that point addition is homomorphically equivalent to adding their (potentially secret) scalar multipliers. But the API and its operations apply equally well to DSA-style integer groups. As a trivial example, generating a public/private keypair is as simple as: The first statement picks a private key (Scalar) from a the suites's source of cryptographic random or pseudo-random bits, while the second performs elliptic curve scalar multiplication of the curve's standard base point (indicated by the 'nil' argument to Mul) by the scalar private key 'a'. Similarly, computing a Diffie-Hellman shared secret using Alice's private key 'a' and Bob's public key 'B' can be done via: Note that we use 'Mul' rather than 'Exp' here because the library uses the additive-group terminology common for elliptic curve crypto, rather than the multiplicative-group terminology of traditional integer groups - but the two are semantically equivalent and the interface itself works for both elliptic curve and integer groups. Various sub-packages provide several specific implementations of these cryptographic interfaces. In particular, the 'group/mod' sub-package provides implementations of modular integer groups underlying conventional DSA-style algorithms. The `group/nist` package provides NIST-standardized elliptic curves built on the Go crypto library. The 'group/edwards25519' sub-package provides the kyber.Group interface using the popular Ed25519 curve. Other sub-packages build more interesting high-level cryptographic tools atop these primitive interfaces, including: - share: Polynomial commitment and verifiable Shamir secret splitting for implementing verifiable 't-of-n' threshold cryptographic schemes. This can be used to encrypt a message so that any 2 out of 3 receivers must work together to decrypt it, for example. - proof: An implementation of the general Camenisch/Stadler framework for discrete logarithm knowledge proofs. This system supports both interactive and non-interactive proofs of a wide variety of statements such as, "I know the secret x associated with public key X or I know the secret y associated with public key Y", without revealing anything about either secret or even which branch of the "or" clause is true. - sign: The sign directory contains different signature schemes. - sign/anon provides anonymous and pseudonymous public-key encryption and signing, where the sender of a signed message or the receiver of an encrypted message is defined as an explicit anonymity set containing several public keys rather than just one. For example, a member of an organization's board of trustees might prove to be a member of the board without revealing which member she is. - sign/cosi provides collective signature algorithm, where a bunch of signers create a unique, compact and efficiently verifiable signature using the Schnorr signature as a basis. - sign/eddsa provides a kyber-native implementation of the EdDSA signature scheme. - sign/schnorr provides a basic vanilla Schnorr signature scheme implementation. - shuffle: Verifiable cryptographic shuffles of ElGamal ciphertexts, which can be used to implement (for example) voting or auction schemes that keep the sources of individual votes or bids private without anyone having to trust more than one of the shuffler(s) to shuffle votes/bids honestly. For now this library should currently be considered experimental: it will definitely be changing in non-backward-compatible ways, and it will need independent security review before it should be considered ready for use in security-critical applications. However, we intend to bring the library closer to stability and real-world usability as quickly as development resources permit, and as interest and application demand dictates. As should be obvious, this library is intended to be used by developers who are at least moderately knowledgeable about cryptography. If you want a crypto library that makes it easy to implement "basic crypto" functionality correctly - i.e., plain public-key encryption and signing - then [NaCl secretbox](https://godoc.org/golang.org/x/crypto/nacl/secretbox) may be a better choice. This toolkit's purpose is to make it possible - and preferably easy - to do slightly more interesting things that most current crypto libraries don't support effectively. The one existing crypto library that this toolkit is probably most comparable to is the Charm rapid prototyping library for Python (https://charm-crypto.com/category/charm). This library incorporates and/or builds on existing code from a variety of sources, as documented in the relevant sub-packages.
copied (almost) entirely from the Go standard library, encoding/json/decode.go
Package kstat provides a Go interface to the Solaris/OmniOS kstat(s) system for user-level access to a lot of kernel statistics. For more documentation on kstats, see kstat(1) and kstat(3kstat). The package can retrieve what are called 'named' kstat statistics, IO statistics, and the most common additional types of 'raw' statistics, which covers almost all kstats you will normally find in the kernel. You can see the names and types of other kstats, but not currently retrieve data for them. Named statistics are the most common type for general information; IO statistics are exported by disks and some other things. Supported additional raw kstats are unix:0:sysinfo, unix:0:vminfo, unix:0:var, and mnt:*:mntinfo. General usage for named statistics: call Open() to obtain a Token, then call GetNamed() on it to obtain Named(s) for specific statistics. Note that this always gives you the very latest value for the statistic. If you want a number of statistics from the same module:inst:name triplet (eg several network counters from the same network interface) and you want them to all have been gathered at the same time, you need to call .Lookup() to obtain a KStat and then repeatedly call its .GetNamed() (this is also slightly more efficient). The short version: a kstat is a collection of some related statistics, eg various network counters for a particular network interface. A Token is a handle for a collection of kstats. You go collection (Token) -> kstat (KStat) -> specific statistic (Named) in order to retrieve the value of a specific statistic. (IO stats are retrieved all at once with GetIO(), because they come to us from the kernel as one single struct so that's what you get.) This is a cgo-based package. Cross compilation is up to you. Goroutine safety is in no way guaranteed because the underlying C kstat library is probably not thread or goroutine safe (and there are some all-Go concurrency races involving .Close()). This package may leak memory, especially since the Solaris kstat manpage is not clear on the requirements here. However I believe it's reasonably memory safe. It's possible to totally corrupt memory with use-after-free errors if you do operations on kstats after calling Token.Close(), although we try to avoid that. NOTE: this package is quite young. The API may well change as I (and other people) gain more experience with it. In general this is not going to be as lean and mean as calling C directly, partly because of intrinsic CGo overheads and partly because we do more memory allocation and deallocation than a C program would (partly because we prioritize not leaking memory). We support named kstats and IO kstats (KSTAT_TYPE_NAMED and KSTAT_TYPE_IO / kstat_io_t respectively). kstat(1) also knows about a number of magic specific 'raw' stats (which are generally custom C structs); of these we support unix:0:sysinfo, unix:0:vminfo, unix:0:var, and mnt:*:mntinfo for NFS filesystem mounts. In theory kstat supports general timer and interrupt stats. In practice there is no use of KSTAT_TYPE_TIMER in the current Illumos kernel source and very little use of KSTAT_TYPE_INTR (mostly by very old hardware drivers, although the vioif driver uses it too). Since I can't test KSTAT_TYPE_INTR stats, we don't currently support it. There are also a few additional KSTAT_TYPE_RAW raw stats that we don't support, mostly because they seem to be effectively obsolete. These specific raw stats can be found listed in the Illumos source code in cmd/stat/kstat/kstat.h in the ks_raw_lookup array. See cmd/stat/kstat/kstat.c for how they're interpreted. If you need access to one of these kstats, the KStat.CopyTo() and KStat.Raw() methods give you an escape hatch to roll your own. You'll probably need to use cgo to generate an appropriate Go struct that matches the C struct you need. My notes on this process may be helpful: https://utcc.utoronto.ca/~cks/space/blog/programming/GoCGoCompatibleStructs Author: Chris Siebenmann https://github.com/siebenmann/go-kstat Copyright: standard Go copyright. (If you're reading this documentation on a non-Solaris platform, you're probably not seeing the detailed API documentation for constants, types, and so on because of tooling limitations in godoc et al.)
Package rlog provides a partial reimplementation of the log package in the standard library. This package provides simple logging with loglevels and an interface quite similar to the standard library. The following environment variables are supported: GO_RLOG_LEVEL GO_RLOG_JOURNAL In addition to the SetLogLevel() methods, the loglevel can be set globally via an environment variable. The recognized values are: CRIT, ERR, WARN, NOTICE, INFO, and DEBUG. These constants are defined in RFC5435, Section 6.2.1. For your convenience, there is a package level logger available, which can be accessed using the relevant methods. The package level logger is there for programs that do not need several loggers. It aims to simplify this particular usecase. If you need several loggers, please instantiate them as needed and do not use the package level logger. Timestamps are not supported by design, since this is almost always not needed. On the terminal, you can pipe the output through ts(1) from moreutils. Using a logging service, the timestamps are already handled perfectly. This avoids the annoying problem of doubled timestamps in the log database, such as: Aug 10 07:24:37 host service[pid]: TIMESTAMP MESSAGE.
Package kstat provides a Go interface to the Solaris/OmniOS kstat(s) system for user-level access to a lot of kernel statistics. For more documentation on kstats, see kstat(1) and kstat(3kstat). The package can retrieve what are called 'named' kstat statistics, IO statistics, and the most common additional types of 'raw' statistics, which covers almost all kstats you will normally find in the kernel. You can see the names and types of other kstats, but not currently retrieve data for them. Named statistics are the most common type for general information; IO statistics are exported by disks and some other things. Supported additional raw kstats are unix:0:sysinfo, unix:0:vminfo, unix:0:var, and mnt:*:mntinfo. General usage for named statistics: call Open() to obtain a Token, then call GetNamed() on it to obtain Named(s) for specific statistics. Note that this always gives you the very latest value for the statistic. If you want a number of statistics from the same module:inst:name triplet (eg several network counters from the same network interface) and you want them to all have been gathered at the same time, you need to call .Lookup() to obtain a KStat and then repeatedly call its .GetNamed() (this is also slightly more efficient). The short version: a kstat is a collection of some related statistics, eg various network counters for a particular network interface. A Token is a handle for a collection of kstats. You go collection (Token) -> kstat (KStat) -> specific statistic (Named) in order to retrieve the value of a specific statistic. (IO stats are retrieved all at once with GetIO(), because they come to us from the kernel as one single struct so that's what you get.) This is a cgo-based package. Cross compilation is up to you. Goroutine safety is in no way guaranteed because the underlying C kstat library is probably not thread or goroutine safe (and there are some all-Go concurrency races involving .Close()). This package may leak memory, especially since the Solaris kstat manpage is not clear on the requirements here. However I believe it's reasonably memory safe. It's possible to totally corrupt memory with use-after-free errors if you do operations on kstats after calling Token.Close(), although we try to avoid that. NOTE: this package is quite young. The API may well change as I (and other people) gain more experience with it. In general this is not going to be as lean and mean as calling C directly, partly because of intrinsic CGo overheads and partly because we do more memory allocation and deallocation than a C program would (partly because we prioritize not leaking memory). We support named kstats and IO kstats (KSTAT_TYPE_NAMED and KSTAT_TYPE_IO / kstat_io_t respectively). kstat(1) also knows about a number of magic specific 'raw' stats (which are generally custom C structs); of these we support unix:0:sysinfo, unix:0:vminfo, unix:0:var, and mnt:*:mntinfo for NFS filesystem mounts. In theory kstat supports general timer and interrupt stats. In practice there is no use of KSTAT_TYPE_TIMER in the current Illumos kernel source and very little use of KSTAT_TYPE_INTR (mostly by very old hardware drivers, although the vioif driver uses it too). Since I can't test KSTAT_TYPE_INTR stats, we don't currently support it. There are also a few additional KSTAT_TYPE_RAW raw stats that we don't support, mostly because they seem to be effectively obsolete. These specific raw stats can be found listed in the Illumos source code in cmd/stat/kstat/kstat.h in the ks_raw_lookup array. See cmd/stat/kstat/kstat.c for how they're interpreted. If you need access to one of these kstats, the KStat.CopyTo() and KStat.Raw() methods give you an escape hatch to roll your own. You'll probably need to use cgo to generate an appropriate Go struct that matches the C struct you need. My notes on this process may be helpful: https://utcc.utoronto.ca/~cks/space/blog/programming/GoCGoCompatibleStructs Author: Chris Siebenmann https://github.com/siebenmann/go-kstat Copyright: standard Go copyright. (If you're reading this documentation on a non-Solaris platform, you're probably not seeing the detailed API documentation for constants, types, and so on because of tooling limitations in godoc et al.)
The quickselect package provides primitives for finding the smallest k elements in slices and user-defined collections. The primitives used in the package are modeled off of the standard sort library for Go. Quickselect uses Hoare's Selection Algorithm which finds the smallest k elements in expected O(n) time, and is thus an asymptotically optimal algorithm (and is faster than sorting or heap implementations).
Package logrus is a structured logger for Go, completely API compatible with the standard library logger. The simplest way to use Logrus is simply the package-level exported logger: Output: For a full guide visit https://github.com/Sirupsen/logrus
go-crypto is a customized/convenience cryptography package for supporting Tendermint. It wraps select functionality of equivalent functions in the Go standard library, for easy usage with our libraries. Keys: All key generation functions return an instance of the PrivKey interface which implements methods From the above method we can: a) Retrieve the public key if needed For example: We also provide hashing wrappers around algorithms: Sha256 Ripemd160
Package log provides the standard logging framework for cybozu products. As this is a framework rather than a library, most features are hard-coded and non-customizable. cybozu/log is a structured logger, that is, every log entry consists of mandatory and optional fields. Mandatory fields are: To help development, logs go to standard error by default. This can be changed to any io.Writer. Logs are formatted by a Formatter. Following built-in formatters are available. The standard field names are defined as constants in this package. For example, "secret" is defined as FnSecret. Field data can be any type though following types are recommended: The framework automatically redirects Go's standard log output to the default logger provided by this framework.
Package config implements simple TOML-based configuration variables, based on the flag package in the standard Go library (In fact, it's just a simple wrapper around flag.FlagSet). It is used in a similar manner, minus the usage strings and other command-line specific bits. Usage: Given the following TOML file: Define your config variables and give them defaults: After all the config variables are defined, load the config file to overwrite the default values with the user-supplied config settings: You can also create separate ConfigSets for different config files:
Package validator implements value validations for structs and individual fields based on tags. It can also handle Cross-Field and Cross-Struct validation for nested structs and has the ability to dive into arrays and maps of any type. Why not a better error message? Because this library intends for you to handle your own error messages. Why should I handle my own errors? Many reasons. We built an internationalized application and needed to know the field, and what validation failed so we could provide a localized error. Doing things this way is actually the way the standard library does, see the file.Open method here: The authors return type "error" to avoid the issue discussed in the following, where err is always != nil: Validator only returns nil or ValidationErrors as type error; so, in your code all you need to do is check if the error returned is not nil, and if it's not type cast it to type ValidationErrors like so err.(validator.ValidationErrors). Custom functions can be added. Example: Cross-Field Validation can be done via the following tags: If, however, some custom cross-field validation is required, it can be done using a custom validation. Why not just have cross-fields validation tags (i.e. only eqcsfield and not eqfield)? The reason is efficiency. If you want to check a field within the same struct "eqfield" only has to find the field on the same struct (1 level). But, if we used "eqcsfield" it could be multiple levels down. Example: Multiple validators on a field will process in the order defined. Example: Bad Validator definitions are not handled by the library. Example: Baked In Cross-Field validation only compares fields on the same struct. If Cross-Field + Cross-Struct validation is needed you should implement your own custom validator. Comma (",") is the default separator of validation tags. If you wish to have a comma included within the parameter (i.e. excludesall=,) you will need to use the UTF-8 hex representation 0x2C, which is replaced in the code as a comma, so the above will become excludesall=0x2C. Pipe ("|") is the default separator of validation tags. If you wish to have a pipe included within the parameter i.e. excludesall=| you will need to use the UTF-8 hex representation 0x7C, which is replaced in the code as a pipe, so the above will become excludesall=0x7C Here is a list of the current built in validators: Tells the validation to skip this struct field; this is particularly handy in ignoring embedded structs from being validated. (Usage: -) This is the 'or' operator allowing multiple validators to be used and accepted. (Usage: rbg|rgba) <-- this would allow either rgb or rgba colors to be accepted. This can also be combined with 'and' for example ( Usage: omitempty,rgb|rgba) When a field that is a nested struct is encountered, and contains this flag any validation on the nested struct will be run, but none of the nested struct fields will be validated. This is usefull if inside of you program you know the struct will be valid, but need to verify it has been assigned. NOTE: only "required" and "omitempty" can be used on a struct itself. Same as structonly tag except that any struct level validations will not run. Is a special tag without a validation function attached. It is used when a field is a Pointer, Interface or Invalid and you wish to validate that it exists. Example: want to ensure a bool exists if you define the bool as a pointer and use exists it will ensure there is a value; couldn't use required as it would fail when the bool was false. exists will fail is the value is a Pointer, Interface or Invalid and is nil. Allows conditional validation, for example if a field is not set with a value (Determined by the "required" validator) then other validation such as min or max won't run, but if a value is set validation will run. This tells the validator to dive into a slice, array or map and validate that level of the slice, array or map with the validation tags that follow. Multidimensional nesting is also supported, each level you wish to dive will require another dive tag. Example #1 Example #2 This validates that the value is not the data types default zero value. For numbers ensures value is not zero. For strings ensures value is not "". For slices, maps, pointers, interfaces, channels and functions ensures the value is not nil. For numbers, max will ensure that the value is equal to the parameter given. For strings, it checks that the string length is exactly that number of characters. For slices, arrays, and maps, validates the number of items. For numbers, max will ensure that the value is less than or equal to the parameter given. For strings, it checks that the string length is at most that number of characters. For slices, arrays, and maps, validates the number of items. For numbers, min will ensure that the value is greater or equal to the parameter given. For strings, it checks that the string length is at least that number of characters. For slices, arrays, and maps, validates the number of items. For strings & numbers, eq will ensure that the value is equal to the parameter given. For slices, arrays, and maps, validates the number of items. For strings & numbers, ne will ensure that the value is not equal to the parameter given. For slices, arrays, and maps, validates the number of items. For numbers, this will ensure that the value is greater than the parameter given. For strings, it checks that the string length is greater than that number of characters. For slices, arrays and maps it validates the number of items. Example #1 Example #2 (time.Time) For time.Time ensures the time value is greater than time.Now.UTC(). Same as 'min' above. Kept both to make terminology with 'len' easier. Example #1 Example #2 (time.Time) For time.Time ensures the time value is greater than or equal to time.Now.UTC(). For numbers, this will ensure that the value is less than the parameter given. For strings, it checks that the string length is less than that number of characters. For slices, arrays, and maps it validates the number of items. Example #1 Example #2 (time.Time) For time.Time ensures the time value is less than time.Now.UTC(). Same as 'max' above. Kept both to make terminology with 'len' easier. Example #1 Example #2 (time.Time) For time.Time ensures the time value is less than or equal to time.Now.UTC(). This will validate the field value against another fields value either within a struct or passed in field. Example #1: Example #2: Field Equals Another Field (relative) This does the same as eqfield except that it validates the field provided relative to the top level struct. This will validate the field value against another fields value either within a struct or passed in field. Examples: Field Does Not Equal Another Field (relative) This does the same as nefield except that it validates the field provided relative to the top level struct. Only valid for Numbers and time.Time types, this will validate the field value against another fields value either within a struct or passed in field. usage examples are for validation of a Start and End date: Example #1: Example #2: This does the same as gtfield except that it validates the field provided relative to the top level struct. Only valid for Numbers and time.Time types, this will validate the field value against another fields value either within a struct or passed in field. usage examples are for validation of a Start and End date: Example #1: Example #2: This does the same as gtefield except that it validates the field provided relative to the top level struct. Only valid for Numbers and time.Time types, this will validate the field value against another fields value either within a struct or passed in field. usage examples are for validation of a Start and End date: Example #1: Example #2: This does the same as ltfield except that it validates the field provided relative to the top level struct. Only valid for Numbers and time.Time types, this will validate the field value against another fields value either within a struct or passed in field. usage examples are for validation of a Start and End date: Example #1: Example #2: This does the same as ltefield except that it validates the field provided relative to the top level struct. This validates that a string value contains alpha characters only This validates that a string value contains alphanumeric characters only This validates that a string value contains a basic numeric value. basic excludes exponents etc... This validates that a string value contains a valid hexadecimal. This validates that a string value contains a valid hex color including hashtag (#) This validates that a string value contains a valid rgb color This validates that a string value contains a valid rgba color This validates that a string value contains a valid hsl color This validates that a string value contains a valid hsla color This validates that a string value contains a valid email This may not conform to all possibilities of any rfc standard, but neither does any email provider accept all posibilities. This validates that a string value contains a valid url This will accept any url the golang request uri accepts but must contain a schema for example http:// or rtmp:// This validates that a string value contains a valid uri This will accept any uri the golang request uri accepts This validates that a string value contains a valid base64 value. Although an empty string is valid base64 this will report an empty string as an error, if you wish to accept an empty string as valid you can use this with the omitempty tag. This validates that a string value contains the substring value. This validates that a string value contains any Unicode code points in the substring value. This validates that a string value contains the supplied rune value. This validates that a string value does not contain the substring value. This validates that a string value does not contain any Unicode code points in the substring value. This validates that a string value does not contain the supplied rune value. This validates that a string value contains a valid isbn10 or isbn13 value. This validates that a string value contains a valid isbn10 value. This validates that a string value contains a valid isbn13 value. This validates that a string value contains a valid UUID. This validates that a string value contains a valid version 3 UUID. This validates that a string value contains a valid version 4 UUID. This validates that a string value contains a valid version 5 UUID. This validates that a string value contains only ASCII characters. NOTE: if the string is blank, this validates as true. This validates that a string value contains only printable ASCII characters. NOTE: if the string is blank, this validates as true. This validates that a string value contains one or more multibyte characters. NOTE: if the string is blank, this validates as true. This validates that a string value contains a valid DataURI. NOTE: this will also validate that the data portion is valid base64 This validates that a string value contains a valid latitude. This validates that a string value contains a valid longitude. This validates that a string value contains a valid U.S. Social Security Number. This validates that a string value contains a valid IP Adress. This validates that a string value contains a valid v4 IP Adress. This validates that a string value contains a valid v6 IP Adress. This validates that a string value contains a valid CIDR Adress. This validates that a string value contains a valid v4 CIDR Adress. This validates that a string value contains a valid v6 CIDR Adress. This validates that a string value contains a valid resolvable TCP Adress. This validates that a string value contains a valid resolvable v4 TCP Adress. This validates that a string value contains a valid resolvable v6 TCP Adress. This validates that a string value contains a valid resolvable UDP Adress. This validates that a string value contains a valid resolvable v4 UDP Adress. This validates that a string value contains a valid resolvable v6 UDP Adress. This validates that a string value contains a valid resolvable IP Adress. This validates that a string value contains a valid resolvable v4 IP Adress. This validates that a string value contains a valid resolvable v6 IP Adress. This validates that a string value contains a valid Unix Adress. This validates that a string value contains a valid MAC Adress. Note: See Go's ParseMAC for accepted formats and types: NOTE: When returning an error, the tag returned in "FieldError" will be the alias tag unless the dive tag is part of the alias. Everything after the dive tag is not reported as the alias tag. Also, the "ActualTag" in the before case will be the actual tag within the alias that failed. Here is a list of the current built in alias tags: Validator notes: This package panics when bad input is provided, this is by design, bad code like that should not make it to production.
nsync package provides a set of primitives that are not provided by standard Go library. NamedMutex - a map of dynamically created mutexes by a referred name. OnceMutex - very similar to sync.Once, however, it is a mutex not a function wrapper. NamedOnceMutex - a map of dynamically created mutexes that can be acquired only once. However, once mutex unlocked it is removed from the map. So, next attempt to acquire this mutex will succeed. Semaphore - a semaphore primitive that can be acquired limited number of times. TryMutex - A mutex that provide ability to set a timeout to acquire a lock. ControlWaitGroup - a controlled goroutine executor that can limit the number concurrently running goroutines. Can help to solve a resource exhaustion problem. SyncFlag - an atomic flag that is wrapped by mutex. Mutex can be set/unset separately to prevent the flag from being set/unset.
Package kyber provides a toolbox of advanced cryptographic primitives, for applications that need more than straightforward signing and encryption. This top level package defines the interfaces to cryptographic primitives designed to be independent of specific cryptographic algorithms, to facilitate upgrading applications to new cryptographic algorithms or switching to alternative algorithms for experimentation purposes. This toolkits public-key crypto API includes a kyber.Group interface supporting a broad class of group-based public-key primitives including DSA-style integer residue groups and elliptic curve groups. Users of this API can write higher-level crypto algorithms such as zero-knowledge proofs without knowing or caring exactly what kind of group, let alone which precise security parameters or elliptic curves, are being used. The kyber.Group interface supports the standard algebraic operations on group elements and scalars that nontrivial public-key algorithms tend to rely on. The interface uses additive group terminology typical for elliptic curves, such that point addition is homomorphically equivalent to adding their (potentially secret) scalar multipliers. But the API and its operations apply equally well to DSA-style integer groups. As a trivial example, generating a public/private keypair is as simple as: The first statement picks a private key (Scalar) from a the suites's source of cryptographic random or pseudo-random bits, while the second performs elliptic curve scalar multiplication of the curve's standard base point (indicated by the 'nil' argument to Mul) by the scalar private key 'a'. Similarly, computing a Diffie-Hellman shared secret using Alice's private key 'a' and Bob's public key 'B' can be done via: Note that we use 'Mul' rather than 'Exp' here because the library uses the additive-group terminology common for elliptic curve crypto, rather than the multiplicative-group terminology of traditional integer groups - but the two are semantically equivalent and the interface itself works for both elliptic curve and integer groups. Various sub-packages provide several specific implementations of these cryptographic interfaces. In particular, the 'group/mod' sub-package provides implementations of modular integer groups underlying conventional DSA-style algorithms. The `group/nist` package provides NIST-standardized elliptic curves built on the Go crypto library. The 'group/edwards25519' sub-package provides the kyber.Group interface using the popular Ed25519 curve. Other sub-packages build more interesting high-level cryptographic tools atop these primitive interfaces, including: - share: Polynomial commitment and verifiable Shamir secret splitting for implementing verifiable 't-of-n' threshold cryptographic schemes. This can be used to encrypt a message so that any 2 out of 3 receivers must work together to decrypt it, for example. - proof: An implementation of the general Camenisch/Stadler framework for discrete logarithm knowledge proofs. This system supports both interactive and non-interactive proofs of a wide variety of statements such as, "I know the secret x associated with public key X or I know the secret y associated with public key Y", without revealing anything about either secret or even which branch of the "or" clause is true. - sign: The sign directory contains different signature schemes. - sign/anon provides anonymous and pseudonymous public-key encryption and signing, where the sender of a signed message or the receiver of an encrypted message is defined as an explicit anonymity set containing several public keys rather than just one. For example, a member of an organization's board of trustees might prove to be a member of the board without revealing which member she is. - sign/cosi provides collective signature algorithm, where a bunch of signers create a unique, compact and efficiently verifiable signature using the Schnorr signature as a basis. - sign/eddsa provides a kyber-native implementation of the EdDSA signature scheme. - sign/schnorr provides a basic vanilla Schnorr signature scheme implementation. - shuffle: Verifiable cryptographic shuffles of ElGamal ciphertexts, which can be used to implement (for example) voting or auction schemes that keep the sources of individual votes or bids private without anyone having to trust more than one of the shuffler(s) to shuffle votes/bids honestly. For now this library should currently be considered experimental: it will definitely be changing in non-backward-compatible ways, and it will need independent security review before it should be considered ready for use in security-critical applications. However, we intend to bring the library closer to stability and real-world usability as quickly as development resources permit, and as interest and application demand dictates. As should be obvious, this library is intended to be used by developers who are at least moderately knowledgeable about cryptography. If you want a crypto library that makes it easy to implement "basic crypto" functionality correctly - i.e., plain public-key encryption and signing - then [NaCl secretbox](https://godoc.org/golang.org/x/crypto/nacl/secretbox) may be a better choice. This toolkit's purpose is to make it possible - and preferably easy - to do slightly more interesting things that most current crypto libraries don't support effectively. The one existing crypto library that this toolkit is probably most comparable to is the Charm rapid prototyping library for Python (https://charm-crypto.com/category/charm). This library incorporates and/or builds on existing code from a variety of sources, as documented in the relevant sub-packages.
Package requests is a minimal, atomic and expressive way of making HTTP requests. It is inspired partly by the HTTP request libraries in other dynamic languages like Python and Javascript. It is safe for all rodents, not just Gophers. Requests is built as a convenient, expressive API around Go's standard http package. With special Request and Response types to help facilitate and streamline RESTful tasks. To send a common GET request just like you'd do with `http.Get`. To send additional data, such as a query parameter, basic authorization header, or content type, just pass in functional options: The data can be a map or struct (anything JSON-marshalable). You can asynchronously wait on a GET response with `GetAsync`. The response returned has the type *requests.Response which embeds *http.Response type to provide more buffer-like methods such as Len(), String(), Bytes(), and JSON(). These special methods use bytes.Buffer under the hood, thus unread portions of data are returned. Make sure not to read from the response's body beforehand. Requests is an ongoing project. Any contribution is whole-heartedly welcomed. Package requests provide useful and declarative methods for RESTful HTTP requests. Package requests provide useful and declarative methods for RESTful HTTP requests.
Package cpu implements processor feature detection used by the Go standard library.
Package netbug provides an http.Handler for executing the various profilers and debug tools in the Go standard library. netbug provides some advantages over the /net/http/pprof and /runtime/pprof packages: The simplest integration of netbug looks like: You can then access the index page via GET /myroute/ The netbug.RegisterAuthHandler function lets you register the handler on your http.ServeMux and add some simple authentication, in the form of a URL parameter: You can then access the index page via: The package also provides access to the handlers directly, for when you want to, say, wrap them in your own logic. Just be sure that when you use the handlers that netbug provides, you take care to use `http.StripPrefix` to strip the route you registered the handler on. This is because the handlers' logic expect them to be registered on "/". As you would expect, netbug works the same way with the go profiler tool as /net/http/pprof does. To run a 30 second CPU profile on your service for example:
Package log is a generic logging library based on logrus (this may change in the future), that minimize the exposure of src-d projects to logrus or any other logging library, as well defines the standard for configuration and usage of the logging libraries in the organization.
Package zeroconf is a pure Golang library that employs Multicast DNS-SD for browsing and resolving services in your network and registering own services in the local network. It basically implements aspects of the standards RFC 6762 (mDNS) and RFC 6763 (DNS-SD). Though it does not support all requirements yet, the aim is to provide a complient solution in the long-term with the community. By now, it should be compatible to [Avahi](http://avahi.org/) (tested) and Apple's Bonjour (untested). Should work in the most office, home and private environments.
Package ki provides the top-level repository for GoKi Trees: Ki = Tree in Japanese, and "Key" in English -- powerful tree structures supporting scenegraphs, programs, parsing, etc. The sub-packages contain all the relevant code: * ki: is the main Ki interface and Node implementation thereof. * kit: is a type registry that ki uses in various ways and provides useful type-level properties that are used in the GoGi GUI. It also is a powerful 'kit for dealing with Go's reflect system. * ints, floats, dirs, bitflag, atomctr, indent all provide basic Go infrastructure that one could argue should have been in the standard library, but isn't..
FactorLog is a logging infrastructure for Go that provides numerous logging functions for whatever your style may be. It could easily be a replacement for Go's log in the standard library (though it doesn't support functions such as `SetFlags()`). Basic usage: Setting your own format: Setting the verbosity and testing against it: If you care about performance, you can test for verbosity this way: For more usage examples, check the examples/ directory. Format verbs: Example colors (see https://github.com/mgutz/ansi for more examples): All logging functions ending in "ln" are merely convenience functions and won't actually output another newline. This allows the formatters to handle a newline however they like (e.g. if you wanted to make a formatter that would output a streamed format without newlines).
Package tfortools provides a set of functions that are designed to make it easier for developers to add template based scripting to their command line tools. Command line tools written in Go often allow users to specify a template script to tailor the output of the tool to their specific needs. This can be useful both when visually inspecting the data and also when invoking command line tools in scripts. The best example of this is go list which allows users to pass a template script to extract interesting information about Go packages. For example, prints all the imports of the current package. The aim of this package is to make it easier for developers to add template scripting support to their tools and easier for users of these tools to extract the information they need. It does this by augmenting the templating language provided by the standard library package text/template in two ways: 1. It auto generates descriptions of the data structures passed as input to a template script for use in help messages. This ensures that help usage information is always up to date with the source code. 2. It provides a suite of convenience functions to make it easy for script writers to extract the data they need. There are functions for sorting, selecting rows and columns and generating nicely formatted tables. For example, if a program passed a slice of structs containing stock data to a template script, we could use the following script to extract the names of the 3 stocks with the highest trade volume. The output might look something like this: The functions head, sort, tables and col are provided by this package.
Package certman provides live reloading of the certificate and key files used by the standard library http.Server. It defines a type, certMan, with methods watching and getting the files. Only valid certificate and key pairs are loaded and an optional logger can be passed to certman for logging providing it implements the logger interface.
Package hiboot is a web/cli app application framework Hiboot is a cloud native web and cli application framework written in Go. Hiboot integrates the popular libraries but make them simpler, easier to use. It borrowed some of the Spring features like dependency injection, aspect oriented programming, and auto configuration. You can integrate any other libraries easily by auto configuration with dependency injection support. hiboot-data is the typical project that implement customized hiboot starters. see https://godoc.org/github.com/hidevopsio/hiboot-data Overview One of the most significant feature of Hiboot is Dependency Injection. Hiboot implements JSR-330 standard. Let's say that we have two implementations of AuthenticationService, below will explain how does Hiboot work. In Hiboot the injection into fields is triggered by `inject:""` struct tag. when inject tag is present on a field, Hiboot tries to resolve the object to inject by the type of the field. If several implementations of the same service interface are available, you have to disambiguate which implementation you want to be injected. This can be done by naming the field to specific implementation. Although Field Injection is pretty convenient, but the Constructor Injection is the first-class citizen, we usually advise people to use constructor injection as it has below advantages, It's testable, easy to implement unit test. Syntax validation, with syntax validation on most of the IDEs to avoid typo. No need to use a dedicated mechanism to ensure required properties are set. type userController struct { at.RestController basicAuthenticationService AuthenticationService } // Hiboot will inject the implementation of AuthenticationService func newUserController(basicAuthenticationService AuthenticationService) { return &userController{ basicAuthenticationService: basicAuthenticationService, } } func init() { app.Register(newUserController) } Features This section will show you how to create and run a simplest hiboot application. Let’s get started! Get the source code Source Code This is a simple hello world example
Package logging is an alternative to log package in standard library.
Package problems provides an RFC 7807 (https://tools.ietf.org/html/rfc7807) compliant implementation of HTTP problem details. Which are defined as a means to carry machine-readable details of errors in an HTTP response to avoid the need to define new error response formats for HTTP APIs. The problem details specification was designed to allow for schema extensions. Because of this the exposed Problem interface only enforces the required Type and Title fields be set appropriately. Additionally, this library also ships with default http.HandlerFunc's capable of writing problems to http.ResponseWriter's in either of the two standard media formats JSON and XML.
Package uci implements a binding to OpenWrt's UCI (Unified Configuration Interface) files in pure Go. The typical use case is reading and modifying UCI config options: For more details head over to the OpenWrt wiki, or dive into UCI's C source code: The lexer is heavily inspired by Rob Pike's 2011 GTUG Sydney talk "Lexical Scanning in Go" (https://talks.golang.org/2011/lex.slide, https://youtu.be/HxaD_trXwRE), which in turn was a presentation of an early version of Go's text/template parser. It follows, that this library borrows code from Go's standard library (BSD-style licensed). The UCI grammar (for the purpose of this library) is defined as follows: For now, UCI imports/exports (packageDecl production) are not supported yet. The STRING token (value production) is also somewhat vaguely defined, and needs to be aligned with the actual C implementation.
Package ora implements an Oracle database driver. An Oracle database may be accessed through the database/sql package or through the ora package directly. database/sql offers connection pooling, thread safety, a consistent API to multiple database technologies and a common set of Go types. The ora package offers additional features including pointers, slices, nullable types, numerics of various sizes, Oracle-specific types, Go return type configuration, and Oracle abstractions such as environment, server and session. The ora package is written with the Oracle Call Interface (OCI) C-language libraries provided by Oracle. The OCI libraries are a standard for client application communication and driver communication with Oracle databases. The ora package has been verified to work with: Minimum requirements are Go 1.3 with CGO enabled, a GCC C compiler, and Oracle 11g (11.2.0.4.0) or Oracle Instant Client (11.2.0.4.0). Install Oracle or Oracle Instant Client. Copy the [oci8.pc](contrib/oci8.pc) from the `contrib` folder (or the one for your system, maybe tailored to your specific locations) to a folder in `$PKG_CONFIG_PATH` or a system folder, such as The ora package has no external Go dependencies and is available on GitHub and gopkg.in: The ora package supports all built-in Oracle data types. The supported Oracle built-in data types are NUMBER, BINARY_DOUBLE, BINARY_FLOAT, FLOAT, DATE, TIMESTAMP, TIMESTAMP WITH TIME ZONE, TIMESTAMP WITH LOCAL TIME ZONE, INTERVAL YEAR TO MONTH, INTERVAL DAY TO SECOND, CHAR, NCHAR, VARCHAR, VARCHAR2, NVARCHAR2, LONG, CLOB, NCLOB, BLOB, LONG RAW, RAW, ROWID and BFILE. SYS_REFCURSOR is also supported. Oracle does not provide a built-in boolean type. Oracle provides a single-byte character type. A common practice is to define two single-byte characters which represent true and false. The ora package adopts this approach. The oracle package associates a Go bool value to a Go rune and sends and receives the rune to a CHAR(1 BYTE) column or CHAR(1 CHAR) column. The default false rune is zero '0'. The default true rune is one '1'. The bool rune association may be configured or disabled when directly using the ora package but not with the database/sql package. Within a SQL string a placeholder may be specified to indicate where a Go variable is placed. The SQL placeholder is an Oracle identifier, from 1 to 30 characters, prefixed with a colon (:). For example: Placeholders within a SQL statement are bound by position. The actual name is not used by the ora package driver e.g., placeholder names :c1, :1, or :xyz are treated equally. You may access an Oracle database through the database/sql package. The database/sql package offers a consistent API across different databases, connection pooling, thread safety and a set of common Go types. database/sql makes working with Oracle straight-forward. The ora package implements interfaces in the database/sql/driver package enabling database/sql to communicate with an Oracle database. Using database/sql ensures you never have to call the ora package directly. When using database/sql, the mapping between Go types and Oracle types may be changed slightly. The database/sql package has strict expectations on Go return types. The Go-to-Oracle type mapping for database/sql is: The "ora" driver is automatically registered for use with sql.Open, but you can call ora.SetDrvCfg to set the used configuration options including statement configuration and Rset configuration. When configuring the driver for use with database/sql, keep in mind that database/sql has strict Go type-to-Oracle type mapping expectations. The ora package allows programming with pointers, slices, nullable types, numerics of various sizes, Oracle-specific types, Go return type configuration, and Oracle abstractions such as environment, server and session. When working with the ora package directly, the API is slightly different than database/sql. When using the ora package directly, the mapping between Go types and Oracle types may be changed. The Go-to-Oracle type mapping for the ora package is: An example of using the ora package directly: Pointers may be used to capture out-bound values from a SQL statement such as an insert or stored procedure call. For example, a numeric pointer captures an identity value: A string pointer captures an out parameter from a stored procedure: Slices may be used to insert multiple records with a single insert statement: The ora package provides nullable Go types to support DML operations such as insert and select. The nullable Go types provided by the ora package are Int64, Int32, Int16, Int8, Uint64, Uint32, Uint16, Uint8, Float64, Float32, Time, IntervalYM, IntervalDS, String, Bool, Binary and Bfile. For example, you may insert nullable Strings and select nullable Strings: The Stmt.Prep method is variadic accepting zero or more GoColumnType which define a Go return type for a select-list column. For example, a Prep call can be configured to return an int64 and a nullable Int64 from the same column: Go numerics of various sizes are supported in DML operations. The ora package supports int64, int32, int16, int8, uint64, uint32, uint16, uint8, float64 and float32. For example, you may insert a uint16 and select numerics of various sizes: If a non-nullable type is defined for a nullable column returning null, the Go type's zero value is returned. GoColumnTypes defined by the ora package are: When Stmt.Prep doesn't receive a GoColumnType, or receives an incorrect GoColumnType, the default value defined in RsetCfg is used. EnvCfg, SrvCfg, SesCfg, StmtCfg and RsetCfg are the main configuration structs. EnvCfg configures aspects of an Env. SrvCfg configures aspects of a Srv. SesCfg configures aspects of a Ses. StmtCfg configures aspects of a Stmt. RsetCfg configures aspects of Rset. StmtCfg and RsetCfg have the most options to configure. RsetCfg defines the default mapping between an Oracle select-list column and a Go type. StmtCfg may be set in an EnvCfg, SrvCfg, SesCfg and StmtCfg. RsetCfg may be set in a Stmt. EnvCfg.StmtCfg, SrvCfg.StmtCfg, SesCfg.StmtCfg may optionally be specified to configure a statement. If StmtCfg isn't specified default values are applied. EnvCfg.StmtCfg, SrvCfg.StmtCfg, SesCfg.StmtCfg cascade to new descendent structs. When ora.OpenEnv() is called a specified EnvCfg is used or a default EnvCfg is created. Creating a Srv with env.OpenSrv() will use SrvCfg.StmtCfg if it is specified; otherwise, EnvCfg.StmtCfg is copied by value to SrvCfg.StmtCfg. Creating a Ses with srv.OpenSes() will use SesCfg.StmtCfg if it is specified; otherwise, SrvCfg.StmtCfg is copied by value to SesCfg.StmtCfg. Creating a Stmt with ses.Prep() will use SesCfg.StmtCfg if it is specified; otherwise, a new StmtCfg with default values is set on the Stmt. Call Stmt.Cfg() to change a Stmt's configuration. An Env may contain multiple Srv. A Srv may contain multiple Ses. A Ses may contain multiple Stmt. A Stmt may contain multiple Rset. Setting a RsetCfg on a StmtCfg does not cascade through descendent structs. Configuration of Stmt.Cfg takes effect prior to calls to Stmt.Exe and Stmt.Qry; consequently, any updates to Stmt.Cfg after a call to Stmt.Exe or Stmt.Qry are not observed. One configuration scenario may be to set a server's select statements to return nullable Go types by default: Another scenario may be to configure the runes mapped to bool values: Oracle-specific types offered by the ora package are ora.Rset, ora.IntervalYM, ora.IntervalDS, ora.Raw, ora.Lob and ora.Bfile. ora.Rset represents an Oracle SYS_REFCURSOR. ora.IntervalYM represents an Oracle INTERVAL YEAR TO MONTH. ora.IntervalDS represents an Oracle INTERVAL DAY TO SECOND. ora.Raw represents an Oracle RAW or LONG RAW. ora.Lob may represent an Oracle BLOB or Oracle CLOB. And ora.Bfile represents an Oracle BFILE. ROWID columns are returned as strings and don't have a unique Go type. Rset is used to obtain Go values from a SQL select statement. Methods Rset.Next, Rset.NextRow, and Rset.Len are available. Fields Rset.Row, Rset.Err, Rset.Index, and Rset.ColumnNames are also available. The Next method attempts to load data from an Oracle buffer into Row, returning true when successful. When no data is available, or if an error occurs, Next returns false setting Row to nil. Any error in Next is assigned to Err. Calling Next increments Index and method Len returns the total number of rows processed. The NextRow method is convenient for returning a single row. NextRow calls Next and returns Row. ColumnNames returns the names of columns defined by the SQL select statement. Rset has two usages. Rset may be returned from Stmt.Qry when prepared with a SQL select statement: Or, *Rset may be passed to Stmt.Exe when prepared with a stored procedure accepting an OUT SYS_REFCURSOR parameter: Stored procedures with multiple OUT SYS_REFCURSOR parameters enable a single Exe call to obtain multiple Rsets: The types of values assigned to Row may be configured in StmtCfg.Rset. For configuration to take effect, assign StmtCfg.Rset prior to calling Stmt.Qry or Stmt.Exe. Rset prefetching may be controlled by StmtCfg.PrefetchRowCount and StmtCfg.PrefetchMemorySize. PrefetchRowCount works in coordination with PrefetchMemorySize. When PrefetchRowCount is set to zero only PrefetchMemorySize is used; otherwise, the minimum of PrefetchRowCount and PrefetchMemorySize is used. The default uses a PrefetchMemorySize of 134MB. Opening and closing Rsets is managed internally. Rset does not have an Open method or Close method. IntervalYM may be be inserted and selected: IntervalDS may be be inserted and selected: Transactions on an Oracle server are supported. DML statements auto-commit unless a transaction has started: Ses.PrepAndExe, Ses.PrepAndQry, Ses.Ins, Ses.Upd, and Ses.Sel are convenient one-line methods. Ses.PrepAndExe offers a convenient one-line call to Ses.Prep and Stmt.Exe. Ses.PrepAndQry offers a convenient one-line call to Ses.Prep and Stmt.Qry. Ses.Ins composes, prepares and executes a sql INSERT statement. Ses.Ins is useful when you have to create and maintain a simple INSERT statement with a long list of columns. As table columns are added and dropped over the lifetime of a table Ses.Ins is easy to read and revise. Ses.Upd composes, prepares and executes a sql UPDATE statement. Ses.Upd is useful when you have to create and maintain a simple UPDATE statement with a long list of columns. As table columns are added and dropped over the lifetime of a table Ses.Upd is easy to read and revise. Ses.Sel composes, prepares and queries a sql SELECT statement. Ses.Sel is useful when you have to create and maintain a simple SELECT statement with a long list of columns that have non-default GoColumnTypes. As table columns are added and dropped over the lifetime of a table Ses.Sel is easy to read and revise. The Ses.Ping method checks whether the client's connection to an Oracle server is valid. A call to Ping requires an open Ses. Ping will return a nil error when the connection is fine: The Srv.Version method is available to obtain the Oracle server version. A call to Version requires an open Ses: Further code examples are available in the example file, test files and samples folder. The ora package provides a simple ora.Logger interface for logging. Logging is disabled by default. Specify one of three optional built-in logging packages to enable logging; or, use your own logging package. ora.Cfg().Log offers various options to enable or disable logging of specific ora driver methods. For example: To use the standard Go log package: which produces a sample log of: Messages are prefixed with 'ORA I' for information or 'ORA E' for an error. The log package is configured to write to os.Stderr by default. Use the ora/lg.Std type to configure an alternative io.Writer. To use the glog package: which produces a sample log of: To use the log15 package: which produces a sample log of: See https://github.com/rana/ora/tree/master/samples/lg15/main.go for sample code which uses the log15 package. Tests are available and require some setup. Setup varies depending on whether the Oracle server is configured as a container database or non-container database. It's simpler to setup a non-container database. An example for each setup is explained. Non-container test database setup steps: Container test database setup steps: Some helpful SQL maintenance statements: Run the tests. database/sql method Stmt.QueryRow is not supported. Copyright 2015 Rana Ian. All rights reserved. Use of this source code is governed by The MIT License found in the accompanying LICENSE file.
Package jsonpointer implements IETF rfc6901 JSON Pointers are a string syntax for identifying a specific value within a JavaScript Object Notation (JSON) document [RFC4627]. JSON Pointer is intended to be easily expressed in JSON string values as well as Uniform Resource Identifier (URI) [RFC3986] fragment identifiers. this package is intended to work like net/url from the go standard library
Package strcase is a package for converting strings into various word cases (e.g. snake_case, camelCase) Example usage ## Why this package String strcase is pretty straight forward and there are a number of methods to do it. This package is fully featured, more customizable, better tested, and faster than other packages and what you would probably whip up yourself. ### Unicode support We work for with unicode strings and pay very little performance penalty for it as we optimized for the common use case of ASCII only strings. ### Customization You can create a custom caser that changes the behavior to what you want. This customization also reduces the pressure for us to change the default behavior which means that things are more stable for everyone involved. The goal is to make the common path easy and fast, while making the uncommon path possible. ### Initialism support By default, we use the golint intialisms list. You can customize and override the initialisms if you wish to add additional ones, such as "SSL" or "CMS" or domain specific ones to your industry. ### Test coverage We have a wide ranging test suite to make sure that we understand our behavior. Test coverage isn't everything, but we aim for 100% coverage. ### Fast Optimized to reduce memory allocations with Builder. Benchmarked and optimized around common cases. We're on par with the fastest packages (that have less features) and much faster than others. We also benchmarked against code snippets. Using string builders to reduce memory allocation and reordering boolean checks for the common cases have a large performance impact. Hopefully I was fair to each library and happy to rerun benchmarks differently or reword my commentary based on suggestions or updates. ### Zero dependencies That's right - zero. We only import the Go standard library. No hassles with dependencies, licensing, security alerts. ## Why not this package If every nanosecond matters and this is used in a tight loop, use segment.io's libraries (https://github.com/segmentio/go-snakecase and https://github.com/segmentio/go-camelcase). They lack features, but make up for it by being blazing fast. ## Migrating from other packages If you are migrating from from another package, you may find slight differences in output. To reduce the delta, you may find it helpful to use the following custom casers to mimic the behavior of the other package.
Package golden testing with golden files in Go. A golden file is the expected output of test, stored as a separate file rather than as a string literal inside the test code. So when the test is executed, it will read data from the file and compare it to the output produced by the functionality under test. When writing unit tests, there comes a point when you need to check that the complex output of a function matches your expectations. This could be binary data (like an Image, JSON, HTML etc). Golden files are a way of ensuring your function output matches its .golden file. It’s a pattern used in the Go standard library. One of the advantages of the gold files approach is that you can easily update the test data using the command line flag without copying the data into the text variables of go this is very convenient in case of significant changes in the behavior of the system but also requires attention to the changed test data and checking the correctness of the new golden results. A special cli is provided in the package. The special flag `-update` is provided in the package for conveniently updating ethos files, for example, using the following command: Golden files are placed in directory `testdata` this directory is ignored by the standard tools go, and it can accommodate a variety of data used in test or samples.
Package arrow provides an implementation of Apache Arrow. Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and inter-process communication. The fundamental data structure in Arrow is an Array, which holds a sequence of values of the same type. An array consists of memory holding the data and an additional validity bitmap that indicates if the corresponding entry in the array is valid (not null). If the array has no null entries, it is possible to omit this bitmap. This example shows how to create a FixedSizeList array. The resulting array should be: This example shows how one can slice an array. The initial (float64) array is: and the sub-slice is: This example demonstrates creating an array, sourcing the values and null bitmaps directly from byte slices. The null count is set to UnknownNullCount, instructing the array to calculate the null count from the bitmap when NullN is called. This example shows how to create a List array. The resulting array should be: This example demonstrates how to build an array of int64 values using a builder and Append. Whilst convenient for small arrays, This example shows how to create a Struct array. The resulting array should be:
Package errors provides some additional error handling primitives compared to the Go standard library errors package. The Cause, Wrap and Wrapf functions should be compatible with those of https://github.com/pkg/errors but does not provide any call stack information.
DCLua - A light weight Go Lua VM designed for easy embedding. The compiler generates correct code in every case I have tested, but the code quality is sometimes poor. If you want better code quality it is possible to compile scripts with luac and load the binaries... Currently the compiler does not support constant folding, and some special instructions are not used at all (instead preferring simpler sequences of other instructions). For example TESTSET is never generated, TEST is used in all cases (largely because It would greatly complicate the compiler if I tried to use TESTSET where possible). Expressions use a simple "recursive" code generation style, meaning that it wastes registers like crazy in some (rare) cases. Most (if not all) of the API functions may cause a panic, but only if things go REALLY wrong. If a function does not state that it can panic or "raise an error" it will only do so if a critical internal assumption proves to be wrong (AKA there is a bug in the code somewhere). These errors will have a special prefix prepended onto the error message stating that this error indicates an internal VM bug. If you ever see such an error I want to know about it ASAP. That said, if an API function *can* "raise an error" it can and will panic if something goes wrong. This is not a problem inside a native function (as the VM is prepared for this), but if you need to call these functions outside of code to be run by the VM you may want to use Protect or Recover to properly catch these errors. The VM itself does not provide any Lua functions, the standard library is provided entirely by external packages. This means that the standard library never does anything that your own code cannot do (there is no "private API" that is used by the standard library). Anything to do with the OS or file IO is not provided. Such things do not belong in the core libraries of an embedded scripting language (do you really want scripts to be able to read and write random files without restriction?).
Package commp allows calculating a Filecoin Unsealed Commitment (commP/commD) given a bytestream. It is implemented as a standard hash.Hash() interface, with the entire padding and treebuilding algorithm written in golang. The returned digest is a 32-byte raw commitment payload. Use something like https://pkg.go.dev/github.com/filecoin-project/go-fil-commcid#DataCommitmentV1ToCID in order to convert it to a proper cid.Cid. The output of this library is 100% identical to https://github.com/filecoin-project/filecoin-ffi/blob/d82899449741ce19/proofs.go#L177-L196
This package implements a parser for configuration files. This allows easy reading and writing of structured configuration files. Given a sample configuration file: To read this configuration file, do: Note the support for unfolding variables (such as %(base-url)s), which are read from the special (reserved) section name [default]. A new configuration file can also be created with: This results in the file: Note that sections and options are case-insensitive (values are case-sensitive) and are converted to lowercase when saved to a file. The functionality and workflow is loosely based on the configparser.py package of the Python Standard Library.
Package sdk is the official AWS SDK for the Go programming language. The AWS SDK for Go provides APIs and utilities that developers can use to build Go applications that use AWS services, such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3). The SDK removes the complexity of coding directly against a web service interface. It hides a lot of the lower-level plumbing, such as authentication, request retries, and error handling. The SDK also includes helpful utilities on top of the AWS APIs that add additional capabilities and functionality. For example, the Amazon S3 Download and Upload Manager will automatically split up large objects into multiple parts and transfer them concurrently. See the s3manager package documentation for more information. https://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/ Checkout the Getting Started Guide and API Reference Docs detailed the SDK's components and details on each AWS client the SDK supports. The Getting Started Guide provides examples and detailed description of how to get setup with the SDK. https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/welcome.html The API Reference Docs include a detailed breakdown of the SDK's components such as utilities and AWS clients. Use this as a reference of the Go types included with the SDK, such as AWS clients, API operations, and API parameters. https://docs.aws.amazon.com/sdk-for-go/api/ The SDK is composed of two main components, SDK core, and service clients. The SDK core packages are all available under the aws package at the root of the SDK. Each client for a supported AWS service is available within its own package under the service folder at the root of the SDK. aws - SDK core, provides common shared types such as Config, Logger, and utilities to make working with API parameters easier. awserr - Provides the error interface that the SDK will use for all errors that occur in the SDK's processing. This includes service API response errors as well. The Error type is made up of a code and message. Cast the SDK's returned error type to awserr.Error and call the Code method to compare returned error to specific error codes. See the package's documentation for additional values that can be extracted such as RequestId. credentials - Provides the types and built in credentials providers the SDK will use to retrieve AWS credentials to make API requests with. Nested under this folder are also additional credentials providers such as stscreds for assuming IAM roles, and ec2rolecreds for EC2 Instance roles. endpoints - Provides the AWS Regions and Endpoints metadata for the SDK. Use this to lookup AWS service endpoint information such as which services are in a region, and what regions a service is in. Constants are also provided for all region identifiers, e.g UsWest2RegionID for "us-west-2". session - Provides initial default configuration, and load configuration from external sources such as environment and shared credentials file. request - Provides the API request sending, and retry logic for the SDK. This package also includes utilities for defining your own request retryer, and configuring how the SDK processes the request. service - Clients for AWS services. All services supported by the SDK are available under this folder. The SDK includes the Go types and utilities you can use to make requests to AWS service APIs. Within the service folder at the root of the SDK you'll find a package for each AWS service the SDK supports. All service clients follows a common pattern of creation and usage. When creating a client for an AWS service you'll first need to have a Session value constructed. The Session provides shared configuration that can be shared between your service clients. When service clients are created you can pass in additional configuration via the aws.Config type to override configuration provided by in the Session to create service client instances with custom configuration. Once the service's client is created you can use it to make API requests the AWS service. These clients are safe to use concurrently. In the AWS SDK for Go, you can configure settings for service clients, such as the log level and maximum number of retries. Most settings are optional; however, for each service client, you must specify a region and your credentials. The SDK uses these values to send requests to the correct AWS region and sign requests with the correct credentials. You can specify these values as part of a session or as environment variables. See the SDK's configuration guide for more information. https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html See the session package documentation for more information on how to use Session with the SDK. https://docs.aws.amazon.com/sdk-for-go/api/aws/session/ See the Config type in the aws package for more information on configuration options. https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config When using the SDK you'll generally need your AWS credentials to authenticate with AWS services. The SDK supports multiple methods of supporting these credentials. By default the SDK will source credentials automatically from its default credential chain. See the session package for more information on this chain, and how to configure it. The common items in the credential chain are the following: Environment Credentials - Set of environment variables that are useful when sub processes are created for specific roles. Shared Credentials file (~/.aws/credentials) - This file stores your credentials based on a profile name and is useful for local development. EC2 Instance Role Credentials - Use EC2 Instance Role to assign credentials to application running on an EC2 instance. This removes the need to manage credential files in production. Credentials can be configured in code as well by setting the Config's Credentials value to a custom provider or using one of the providers included with the SDK to bypass the default credential chain and use a custom one. This is helpful when you want to instruct the SDK to only use a specific set of credentials or providers. This example creates a credential provider for assuming an IAM role, "myRoleARN" and configures the S3 service client to use that role for API requests. See the credentials package documentation for more information on credential providers included with the SDK, and how to customize the SDK's usage of credentials. https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials The SDK has support for the shared configuration file (~/.aws/config). This support can be enabled by setting the environment variable, "AWS_SDK_LOAD_CONFIG=1", or enabling the feature in code when creating a Session via the Option's SharedConfigState parameter. In addition to the credentials you'll need to specify the region the SDK will use to make AWS API requests to. In the SDK you can specify the region either with an environment variable, or directly in code when a Session or service client is created. The last value specified in code wins if the region is specified multiple ways. To set the region via the environment variable set the "AWS_REGION" to the region you want to the SDK to use. Using this method to set the region will allow you to run your application in multiple regions without needing additional code in the application to select the region. The endpoints package includes constants for all regions the SDK knows. The values are all suffixed with RegionID. These values are helpful, because they reduce the need to type the region string manually. To set the region on a Session use the aws package's Config struct parameter Region to the AWS region you want the service clients created from the session to use. This is helpful when you want to create multiple service clients, and all of the clients make API requests to the same region. See the endpoints package for the AWS Regions and Endpoints metadata. https://docs.aws.amazon.com/sdk-for-go/api/aws/endpoints/ In addition to setting the region when creating a Session you can also set the region on a per service client bases. This overrides the region of a Session. This is helpful when you want to create service clients in specific regions different from the Session's region. See the Config type in the aws package for more information and additional options such as setting the Endpoint, and other service client configuration options. https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config Once the client is created you can make an API request to the service. Each API method takes a input parameter, and returns the service response and an error. The SDK provides methods for making the API call in multiple ways. In this list we'll use the S3 ListObjects API as an example for the different ways of making API requests. ListObjects - Base API operation that will make the API request to the service. ListObjectsRequest - API methods suffixed with Request will construct the API request, but not send it. This is also helpful when you want to get a presigned URL for a request, and share the presigned URL instead of your application making the request directly. ListObjectsPages - Same as the base API operation, but uses a callback to automatically handle pagination of the API's response. ListObjectsWithContext - Same as base API operation, but adds support for the Context pattern. This is helpful for controlling the canceling of in flight requests. See the Go standard library context package for more information. This method also takes request package's Option functional options as the variadic argument for modifying how the request will be made, or extracting information from the raw HTTP response. ListObjectsPagesWithContext - same as ListObjectsPages, but adds support for the Context pattern. Similar to ListObjectsWithContext this method also takes the request package's Option function option types as the variadic argument. In addition to the API operations the SDK also includes several higher level methods that abstract checking for and waiting for an AWS resource to be in a desired state. In this list we'll use WaitUntilBucketExists to demonstrate the different forms of waiters. WaitUntilBucketExists. - Method to make API request to query an AWS service for a resource's state. Will return successfully when that state is accomplished. WaitUntilBucketExistsWithContext - Same as WaitUntilBucketExists, but adds support for the Context pattern. In addition these methods take request package's WaiterOptions to configure the waiter, and how underlying request will be made by the SDK. The API method will document which error codes the service might return for the operation. These errors will also be available as const strings prefixed with "ErrCode" in the service client's package. If there are no errors listed in the API's SDK documentation you'll need to consult the AWS service's API documentation for the errors that could be returned. Pagination helper methods are suffixed with "Pages", and provide the functionality needed to round trip API page requests. Pagination methods take a callback function that will be called for each page of the API's response. Waiter helper methods provide the functionality to wait for an AWS resource state. These methods abstract the logic needed to to check the state of an AWS resource, and wait until that resource is in a desired state. The waiter will block until the resource is in the state that is desired, an error occurs, or the waiter times out. If a resource times out the error code returned will be request.WaiterResourceNotReadyErrorCode. This example shows a complete working Go file which will upload a file to S3 and use the Context pattern to implement timeout logic that will cancel the request if it takes too long. This example highlights how to use sessions, create a service client, make a request, handle the error, and process the response.
Package mwclient provides functionality for interacting with the MediaWiki API. go-mwclient is intended for users who are already familiar with (or are willing to learn) the MediaWiki API. It is intended to make dealing with the API more convenient, but not to hide it. go-mwclient v1 uses version 2 of the MW JSON API. In the example below, basic usage of go-mwclient is shown. Create a new Client object with the New() constructor, and then you are ready to start making requests to the API. If you wish to make requests to multiple MediaWiki sites, you must create a Client for each of them. go-mwclient offers a few methods for making arbitrary requests to the API: Get, GetRaw, Post, and PostRaw (see documentation for the methods for details). They all offer the same basic interface: pass a params.Values map (from the cgt.name/pkg/go-mwclient/params package), receive a response and an error. For convenience, go-mwclient offers several methods for making common requests (login, edit, etc.), but these methods are implemented using the same interface. params.Values params.Values is similar to (and a fork of) the standard library's net/url.Values. The reason why params.Values is used instead is that url.Values is based on a map[string][]string, rather than a map[string]string. This is because url.Values must support multiple keys with the same name. The literal syntax for a map[string][]string is rather cumbersome because the value is a slice rather than just a string, and the MediaWiki API actually does not use multiple keys when multiple values for the same key is required. Instead, one key is used and the values are separated by pipes (|). It is therefore very simple to write multi-value values in params.Values literals while params.Values makes it simple to write multi-value values in literals while avoiding the cumbersome []string literals for the most common case where the is only value. See documentation for the params package for more information. Because of the way type identity works in Go, it is possible for callers to pass a plain map[string]string rather than a params.Values. It is only necessary for users to use params.Values directly if they wish to use params.Values's methods. It makes no difference to go-mwclient. If an API call fails it will return an error. Many things can go wrong during an API call: the network could be down, the API could return an unexpected response (if the API was changed), or perhaps there's an error in your API request. If the error is an API error or warning (and you used the "non-Raw" Get and Post methods), then the error/warning(s) will be parsed and returned in either an APIError or an APIWarnings object, both of which implement the error interface. The "Raw" request methods do not check for API errors or warnings. For more information about API errors and warnings, please see https://www.mediawiki.org/wiki/API:Errors_and_warnings. If maxlag is enabled, it may be that the API has rejected the requests and the amount of retries (3 by default) have been tried unsuccessfully. In that case, the error will be the variable mwclient.ErrAPIBusy. Other methods than the core ones (i.e., other methods than Get and Post) may return other errors.