Package chacha20 implements the ChaCha20 / XChaCha20 stream chipher. Notice that one specific key-nonce combination must be unique for all time. There are three versions of ChaCha20: - ChaCha20 with a 64 bit nonce (en/decrypt up to 2^64 * 64 bytes for one key-nonce combination) - ChaCha20 with a 96 bit nonce (en/decrypt up to 2^32 * 64 bytes (~256 GB) for one key-nonce combination) - XChaCha20 with a 192 bit nonce (en/decrypt up to 2^64 * 64 bytes for one key-nonce combination)
Package graw is a high level, easy to use, Reddit bot library. graw will take a low level handle from the graw/reddit package and manage everything for you. You just specify in a config what events you want to listen to on Reddit, and graw will take care of maintaining the event stream and calling the handler methods of your bot whenever new events occur. Announcing new posts in /r/self is as simple as: graw can handle many event sources on Reddit; see the Config struct for the complete set of offerings. graw has one other function that behaves like Scan(), which is Run(). Scan() is for logged-out bots (what Reddit calls "scripts"). Run() handles logged in bots, which can subscribe to logged-in event sources in the bot's account inbox like mentions and private messages.
Package mp4ff - MP4 media file parser and writer for AVC and HEVC video, AAC audio and stpp/wvtt subtitles. Focused on fragmented files as used for streaming in DASH, MSS and HLS fMP4. The mp4 library has functions for parsing (called Decode) and writing (called Encode). It is focused on fragmented files as used for streaming in DASH, MSS and HLS fMP4. mp4.File is a representation of a "File" which can be more or less complete, but should have some top layer boxes. It can include The typical child boxes are exported so that one can write paths such as to access the (only) trun box of a moof box. Some simple command line tools are available in cmd directory. Example code is available in the examples directory.
Package redisqueue provides a producer and consumer of a queue that uses Redis streams (https://redis.io/topics/streams-intro). The features of this package include: Here's an example of a producer that inserts 1000 messages into a queue: And here's an example of a consumer that reads the messages off of that queue:
Package multistream implements a simple stream router for the multistream-select protocoli. The protocol is defined at https://github.com/multiformats/multistream-select
Package mp4ff - MP4 media file parser and writer for AVC and HEVC video, AAC audio and stpp/wvtt subtitles. Focused on fragmented files as used for streaming in DASH, MSS and HLS fMP4. The mp4 library has functions for parsing (called Decode) and writing (called Encode). It is focused on fragmented files as used for streaming in DASH, MSS and HLS fMP4. mp4.File is a representation of a "File" which can be more or less complete, but should have some top layer boxes. It can include The typical child boxes are exported so that one can write paths such as to access the (only) trun box of a moof box. Some simple command line tools are available in cmd directory. Example code is available in the examples directory.
Package stream provides filters that can be chained together in a manner similar to Unix pipelines. A simple example that prints all go files under the current directory: stream.Run is passed a list of filters that are chained together (stream.Find, stream.Grep, stream.WriteLines are filters). Each filter takes as input a sequence of strings and produces a sequence of strings. The empty sequence is passed as input to the first filter. The output of one filter is fed as input to the next filter. stream.Run is just one way to execute filters. Others are stream.Contents (returns the output of the last filter as a []string), and stream.ForEach (executes a supplied function for every output item). Filter execution can result in errors. These are returned from stream functions normally. For example, the following call will return a non-nil error. Each filter takes as input a sequence of strings (read from a channel) and produces as output a sequence of strings (written to a channel). The stream package provides a bunch of useful filters. Applications can define their own filters easily. For example, here is a filter that repeats every input n times: The output will be: Note that Repeat returns a FilterFunc, a function type that implements the Filter interface. This is a common implementation pattern: many simple filters can be expressed as a single function of type FilterFunc. FilterFunc is an appropriate type to use for most filters like Repeat above. However for some filters, dynamic customization is appropriate. Such filters provide their own implementation of the Filter interface with extra methods. For example, stream.Sort provides extra methods that can be used to control how items are sorted: The interface of this package is inspired by the http://labix.org/pipe package. Users may wish to consider that package in case it fits their needs better.
MIDI package is a midi format parser that parses the content of a midi stream and asyncronously send the data to a provided channel or collect the data into an accessible struct,
Package vssh is a Go library to handle tens of thousands SSH connections and execute the command with higher-level API for building network device / server automation. By calling the run method vssh sends the given command to all available clients or based on your query it runs the command on the specific clients and the results of the ran command can be received in two options, streaming or final result.In streaming you can get line by line from command’s stdout / stderr in real time or in case of non-real time you can get the whole of the lines together. This example demonstrates integration vSSH with AWS EC2
Package p2phttp allows to serve HTTP endpoints and make HTTP requests through LibP2P (https://github.com/libp2p/libp2p) using Go's standard "http" and "net" stacks. Instead of the regular "host:port" addressing, `p2phttp` uses a Peer ID and lets LibP2P take care of the routing, thus taking advantage of features like multi-routes, NAT transversal and stream multiplexing over a single connection. When already running a LibP2P facility, this package allows to expose existing HTTP-based services (like REST APIs) through LibP2P and to use those services with minimal changes to the code-base. For example, a simple http.Server on LibP2P works as: As shown above, a Server only needs a "github.com/libp2p/go-libp2p-gostream" listener. This listener will use a libP2P host to watch for stream tagged with our Protocol. On the other side, a client just needs to be initialized with a custom LibP2P host-based transport to perform requests to such server: In the example above, the client registers a "libp2p" protocol for which the custom transport is used. It can still perform regular "http" requests. The protocol name used is arbitraty and non standard. Note that LibP2P hosts cannot dial to themselves, so there is no possibility of using the same host as server and as client.
Package grpchan provides an abstraction for a gRPC transport, called a Channel. The channel is more general than the concrete *grpc.ClientConn and *grpc.Server types provided by gRPC. It allows gRPC over alternate substrates and includes sub-packages that provide two such alternatives: in-process and HTTP 1.1. The key type in this package is an alternate implementation of grpc.ServiceRegistrar interface that allows you to accumulate service registrations, for use with an implementation other than *grpc.Server. This repo also includes a deprecated protoc plugin. This is no longer needed now that the standard protoc-gen-go-grpc plugin generates code that uses interfaces: grpc.ClientConnInterface and grpc.ServiceRegistrar. In older versions, the generated code only supported concrete types (*grpc.ClientConn and *grpc.Server) so this repo's protoc plugin would generate alternate code that used interfaces (and thus supported other concrete implementations). Continued use of the plugin is only to continue supporting code that uses the functions generated by it. To use the protoc plugin, you need to first build it and make sure its location is in your PATH. When you invoke protoc, include a --grpchan_out parameter that indicates the same output directory as used for your --go_out parameter. Alongside the *.pb.go files generated, the grpchan plugin will also create *.pb.grpchan.go files. In older versions of the Go plugin (when emitting gRPC code), a server registration function for each RPC service defined in the proto source files was generated that looked like so: The grpchan plugin produces a similarly named method that accepts the ServiceRegistry interface: A new transport can then be implemented by just implementing two interfaces: grpc.ClientConnInterface for the client side and grpchan.ServiceRegistry for the server side. The alternate method also works just fine with *grpc.Server as it implements the ServiceRegistry interface. NOTE: If your have code relying on New<ServiceName>ChannelClient methods that earlier versions of this package produced, they can still be generated by passing a "legacy_stubs" option to the plugin. Example: The client-side implementation of a transport is done with just the two methods in grpc.ClientConnInterface: one for unary RPCs and the other for streaming RPCs. Note that when a unary interceptor is invoked for an RPC on a channel that is *not* a *grpc.ClientConn, the parameter of that type will be nil. Not all client call options will make sense for all transports. This repo chooses to ignore call options that do not apply (as opposed to failing the RPC or panicking). However, several call options are likely important to support: those for accessing header and trailer metadata. The peer, per-RPC credentials, and message size limits are other options that are reasonably straight-forward to apply to other transports. But the other options (dealing with on-the-wire encoding, compression, etc) may not be applicable. The server-side implementation of a transport must be able to invoke method and stream handlers for a given service implementation. This is done by implementing the grpc.ServiceRegistrar interface. When a service is registered, a service description is provided that includes access to method and stream handlers. When the transport receives requests for RPC operations, it in turn invokes these handlers. For streaming operations, it must also supply a grpc.ServerStream implementation, for exchanging messages on the stream. Note that the server stream's context will need a custom implementation of the grpc.ServerTransportStream in it, too. Sadly, this interface is just different enough from grpc.ServerStream that they cannot be implemented by the same type. This is particularly necessary for unary calls since this is how a unary handler indicates what headers and trailers to send back to the client.
Package nrstan instruments https://github.com/nats-io/stan.go. This package can be used to simplify instrumenting NATS Streaming subscribers. Currently due to the nature of the NATS Streaming framework we are limited to two integration points: `StartPublishSegment` for publishers, and `SubWrapper` for subscribers. `nrstan.StreamingSubWrapper` can be used to wrap the function for STREAMING stan.Subscribe and stan.QueueSubscribe (https://godoc.org/github.com/nats-io/stan.go#Conn) If the `newrelic.Application` parameter is non-nil, it will create a `newrelic.Transaction` and end the transaction when the passed function is complete. Example: You can use `nrnats.StartPublishSegment` from the `nrnats` package (https://godoc.org/github.com/newrelic/go-agent/v3/integrations/nrnats/#StartPublishSegment) to start an external segment when doing a streaming publish, which must be ended after publishing is complete. Example: Full Publisher/Subscriber example: https://github.com/newrelic/go-agent/blob/master/v3/integrations/nrstan/examples/main.go
muxado implements a general purpose stream-multiplexing protocol. muxado allows clients applications to multiplex any io.ReadWriteCloser (like a net.Conn) into multiple, independent full-duplex streams. muxado is a useful protocol for any two communicating processes. It is an excellent base protocol for implementing lightweight RPC. It eliminates the need for custom async/pipeling code from your peers in order to support multiple simultaneous inflight requests between peers. For the same reason, it also eliminates the need to build connection pools for your clients. It enables servers to initiate streams to clients without building any NAT traversal. muxado can also yield performance improvements (especially latency) for protocols that require rapidly opening many concurrent connections. Here's an example client which responds to simple JSON requests from a server. Maybe the client wants to make a request to the server instead of just responding. This is easy as well: muxado defines the following terms for clarity of the documentation: A "Transport" is an underlying stream (typically TCP) that is multiplexed by sending frames between muxado peers over this transport. A "Stream" is any of the full-duplex byte-streams multiplexed over the transport A "Session" is two peers running the muxado protocol over a single transport muxado's design is influenced heavily by the framing layer of HTTP2 and SPDY. However, instead of being specialized for a higher-level protocol, muxado is designed in a protocol agnostic way with simplicity and speed in mind. More advanced features are left to higher-level libraries and protocols. muxado's API is designed to make it seamless to integrate into existing Go programs. muxado.Session implements the net.Listener interface and muxado.Stream implements net.Conn. muxado ships with two wrappers that add commonly used functionality. The first is a TypedStreamSession which allows a client application to open streams with a type identifier so that the remote peer can identify the protocol that will be communicated on that stream. The second wrapper is a simple Heartbeat which issues a callback to the application informing it of round-trip latency and heartbeat failure.
Package edn implements encoding and decoding of EDN values as defined in https://github.com/edn-format/edn. For a full introduction on how to use go-edn, see https://github.com/go-edn/edn/blob/v1/docs/introduction.md. Fully self-contained examples of go-edn can be found at https://github.com/go-edn/edn/tree/v1/examples. Note that the small examples in this package is not checking errors as persively as you should do when you use this package. This is done because I'd like the examples to be easily readable and understandable. The bigger examples provide proper error handling. EDN, in contrast to JSON, supports arbitrary values as keys. This example shows how one can implement enums and sets, and how to support multiple different forms for a specific value type. The set implemented here supports the notation `:all` for all values. This example shows how to read and write basic EDN tags, and how this can be utilised: In contrast to encoding/json, you can read in data where you only know that the input satisfies some sort of interface, provided the value is tagged. This example shows how one can do streaming with the decoder, and how to properly know when the stream has no elements left.
Package chunk implements streaming block splitters. Splitters read data from a reader and provide byte slices (chunks) The size and contents of these slices depend on the splitting method used.
hllpp implements the HyperLogLog++ cardinality estimator as specified in the HyperLogLog++ paper http://goo.gl/Z5Sqgu. hllpp uses a built-in non-streaming implementation of murmur3 to hash data as you add it to the estimator.
Package gchalk is terminal string styling for go done right, with full Linux, MacOS, and painless Windows 10 support. GChalk is a library heavily inspired by https://github.com/chalk/chalk, the popular Node.js terminal color library, and using golang ports of supports-color (https://github.com/jwalton/go-supportscolor) and ansi-styles (https://github.com/jwalton/gchalk/pkg/ansistyles). A very simple usage example would be: Note that this works on all platforms - there's no need to write to a special stream or use a special print function to get color on Windows 10. Some examples: See the README.md for more details.
Package signalr contains a SignalR client and a SignalR server. Both support the transport types Websockets and Server-Sent Events and the transfer formats Text (JSON) and Binary (MessagePack). The SignalR Protocol is a protocol for two-way RPC over any stream- or message-based transport. Either party in the connection may invoke procedures on the other party, and procedures can return zero or more results or an error. Typically, SignalR connections are HTTP-based, but it is dead simple to implement a signalr.Connection on any transport that supports io.Reader and io.Writer. A Client can be used in client side code to access server methods. From an existing connection, it can be created with NewClient(). A special case is NewHTTPClient(), which creates a Client from a server address and negotiates with the server which kind of connection (Websockets, Server-Sent Events) will be used. The object which will receive server callbacks is passed to NewClient() by using the WithReceiver option. After calling client.Start(), the client is ready to call server methods or to receive callbacks. A Server provides the public methods of a server side class over signalr to the client. Such a server side class is called a hub and must implement HubInterface. It is reasonable to derive your hubs from the Hub struct type, which already implements HubInterface. Servers for arbitrary connection types can be created with NewServer(). To serve a connection, call server.Serve(connection) in a goroutine. Serve ends when the connection is closed or the servers context is canceled. To serve a HTTP connection, use server.MapHTTP(), which connects the server with a path in an http.ServeMux. The server then automatically negotiates which kind of connection (Websockets, Server-Sent Events) will be used. The SignalR protocol constrains the signature of hub or receiver methods that can be used over SignalR. All methods with serializable types as parameters and return types are supported. Methods with multiple return values are not generally supported, but returning one or no value and an optional error is supported. Methods which return a single sending channel (<-chan), and optionally an error, are used to initiate callee side streaming. The caller will receive the contents of the channel as stream. When the returned channel is closed, the stream will be completed. Methods with one or multiple receiving channels (chan<-) as parameters are used as receivers for caller side streaming. The caller invokes this method and pushes one or multiple streams to the callee. The method should end when all channels are closed. A channel is closed by the server when the assigned stream is completed. The methods which return a channel are not supported. In most cases, the caller will be the client and the callee the server. But the vice versa case is also possible.
Package mpegts proves utility functions for mpegts streams
Package irc allows your application to speak the IRC protocol. The Message and Prefix structs provide translation to and from raw IRC messages: Decoder and Encoder can be used to decode and encode messages in a stream: The Conn type combines an Encoder and Decoder for a duplex connection.
Package wire implements the Decred wire protocol. For the complete details of the Decred protocol, see the official wiki entry at https://en.bitcoin.it/wiki/Protocol_specification. The following only serves as a quick overview to provide information on how to use the package. At a high level, this package provides support for marshalling and unmarshalling supported Decred messages to and from the wire. This package does not deal with the specifics of message handling such as what to do when a message is received. This provides the caller with a high level of flexibility. The Decred protocol consists of exchanging messages between peers. Each message is preceded by a header which identifies information about it such as which Decred network it is a part of, its type, how big it is, and a checksum to verify validity. All encoding and decoding of message headers is handled by this package. To accomplish this, there is a generic interface for Decred messages named Message which allows messages of any type to be read, written, or passed around through channels, functions, etc. In addition, concrete implementations of most of the currently supported Decred messages are provided. For these supported messages, all of the details of marshalling and unmarshalling to and from the wire using Decred encoding are handled so the caller doesn't have to concern themselves with the specifics. The following provides a quick summary of how the Decred messages are intended to interact with one another. As stated above, these interactions are not directly handled by this package. For more in-depth details about the appropriate interactions, see the official Decred protocol wiki entry at https://en.bitcoin.it/wiki/Protocol_specification. The initial handshake consists of two peers sending each other a version message (MsgVersion) followed by responding with a verack message (MsgVerAck). Both peers use the information in the version message (MsgVersion) to negotiate things such as protocol version and supported services with each other. Once the initial handshake is complete, the following chart indicates message interactions in no particular order. There are several common parameters that arise when using this package to read and write Decred messages. The following sections provide a quick overview of these parameters so the next sections can build on them. The protocol version should be negotiated with the remote peer at a higher level than this package via the version (MsgVersion) message exchange, however, this package provides the wire.ProtocolVersion constant which indicates the latest protocol version this package supports and is typically the value to use for all outbound connections before a potentially lower protocol version is negotiated. The Decred network is a magic number which is used to identify the start of a message and which Decred network the message applies to. This package provides the following constants: As discussed in the Decred message overview section, this package reads and writes Decred messages using a generic interface named Message. In order to determine the actual concrete type of the message, use a type switch or type assertion. An example of a type switch follows: In order to unmarshall Decred messages from the wire, use the ReadMessage function. It accepts any io.Reader, but typically this will be a net.Conn to a remote node running a Decred peer. Example syntax is: In order to marshall Decred messages to the wire, use the WriteMessage function. It accepts any io.Writer, but typically this will be a net.Conn to a remote node running a Decred peer. Example syntax to request addresses from a remote peer is: Errors returned by this package are either the raw errors provided by underlying calls to read/write from streams such as io.EOF, io.ErrUnexpectedEOF, and io.ErrShortWrite, or of type wire.MessageError. This allows the caller to differentiate between general IO errors and malformed messages through type assertions. This package includes spec changes outlined by the following BIPs:
Package sqlite provides a Go interface to SQLite 3. The semantics of this package are deliberately close to the SQLite3 C API. See the official C API introduction for an overview of the basics. An SQLite connection is represented by a *Conn. Connections cannot be used concurrently. A typical Go program will create a pool of connections (e.g. by using zombiezen.com/go/sqlite/sqlitex.NewPool to create a *zombiezen.com/go/sqlite/sqlitex.Pool) so goroutines can borrow a connection while they need to talk to the database. This package assumes SQLite will be used concurrently by the process through several connections, so the build options for SQLite enable multi-threading and the shared cache. The implementation automatically handles shared cache locking, see the documentation on Stmt.Step for details. The optional SQLite 3 extensions compiled in are: session, FTS5, RTree, JSON1, and GeoPoly. This is not a database/sql driver. For helper functions to make it easier to execute statements, see the zombiezen.com/go/sqlite/sqlitex package. Statements are prepared with the Conn.Prepare and Conn.PrepareTransient methods. When using Conn.Prepare, statements are keyed inside a connection by the original query string used to create them. This means long-running high-performance code paths can write: After all the connections in a pool have been warmed up by passing through one of these Prepare calls, subsequent calls are simply a map lookup that returns an existing statement. SQLite transactions can be managed manually with this package by directly executing BEGIN / COMMIT / ROLLBACK or SAVEPOINT / RELEASE / ROLLBACK statements, but there are also helper functions available in zombiezen.com/go/sqlite/sqlitex: For simple schema migration needs, see the zombiezen.com/go/sqlite/sqlitemigration package. Use Conn.CreateFunction to register Go functions for use as SQL functions. The sqlite package supports the SQLite incremental I/O interface for streaming blob data into and out of the the database without loading the entire blob into a single []byte. (This is important when working either with very large blobs, or more commonly, a large number of moderate-sized blobs concurrently.) See Conn.OpenBlob for more details. Every connection can have a done channel associated with it using the Conn.SetInterrupt method. This is typically the channel returned by a context.Context.Done method. As database connections are long-lived, the Conn.SetInterrupt method can be called multiple times to reset the associated lifetime. Using a Pool to execute SQL in a concurrent HTTP handler. This is the same as the main package example, but uses the SQLite statement API instead of sqlitex.
Package rpc is heavily inspired by Go standard net/rpc package. It aims to do the same thing, except it uses libp2p for communication and provides context support for cancelling operations. A server registers an object, making it visible as a service with the name of the type of the object. After registration, exported methods of the object will be accessible remotely. A server may register multiple objects (services) of different types but it is an error to register multiple objects of the same type. Only methods that satisfy these criteria will be made available for remote access; other methods will be ignored: In effect, the method must look schematically like where T1 and T2 can be marshaled by github.com/ugorji/go/codec. In normal calls, the method's second argument represents the arguments provided by the caller; the third argument represents the result parameters to be returned to the caller. The function error response is passed to the client accordingly. In streaming calls, the method's second and third arguments are argument and replies channels. The method is expected to read from the argument channel until it is closed. The method is expected to send responses on the replies channel and close it when done. Both channels are transparently and asynchronously streamed on the wire between remote hosts. In order to use this package, a ready-to-go libp2p Host must be provided to clients and servers, along with a protocol.ID. rpc will add a stream handler for the given protocol. Contexts are supported and honored when provided. On the server side, methods must take a context. A closure or reset of the libp2p stream will trigger a cancellation of the context received by the functions. On the client side, the user can optionally provide a context. Cancelling the client's context will cancel the operation both on the client and on the server side (by closing the associated stream).
Package minhash implements the bottom-k sketch for streaming set similarity. For more information, This package works best when provided with a strong 64-bit hash function, such as CityHash, Spooky, Murmur3, or SipHash.
Package smux is a multiplexing library for Golang. It relies on an underlying connection to provide reliability and ordering, such as TCP or KCP, and provides stream-oriented multiplexing over a single channel.
Package metrics provides minimalist instrumentation for your applications in the form of counters and gauges. A counter is a monotonically-increasing, unsigned, 64-bit integer used to represent the number of times an event has occurred. By tracking the deltas between measurements of a counter over intervals of time, an aggregation layer can derive rates, acceleration, etc. A gauge returns instantaneous measurements of something using signed, 64-bit integers. This value does not need to be monotonic. A histogram tracks the distribution of a stream of values (e.g. the number of milliseconds it takes to handle requests), adding gauges for the values at meaningful quantiles: 50th, 75th, 90th, 95th, 99th, 99.9th. Measurements from counters and gauges are available as expvars. Your service should return its expvars from an HTTP endpoint (i.e., /debug/vars) as a JSON object.
Package capn is a capnproto library for go see http://kentonv.github.io/capnproto/ capnpc-go provides the compiler backend for capnp after installing to $PATH capnp files can be compiled with capnpc-go requires two annotations for all types. This is the package and import found in go.capnp. Package is needed to know what package to place at the head of the generated file and what go name to use when referring to the type from another package. Import should be the fully qualified import path and is used to generate import statement from other packages and to detect when two types are in the same package. Typically these are added as file annotations. For example: In capnproto, the unit of communication is a message. A message consists of one or more of segments to allow easier allocation, but ideally and typically you just make one segment per message. Logically, a message organized in a tree of objects, with the root always being a struct (as opposed to a list or primitive). Here is an example of writing a new message. We use the demo schema aircraft.capnp from the aircraftlib directory. You may wish to read the schema before reading this example. << Example moved to its own file: See the file, write_test.go >> In summary, when you make a new message, you should first make new segment, and then create the root struct in that segment. Then add your non-child (contained) objects. This is because, as the spec says: All objects are values with pointer semantics that point into the data in a message or segment. Messages can be read/written from a stream uncompressed or using the capnproto compression. In this library a *Segment is taken to refer to both a specific segment as well as the containing message. This is to reduce the number of types generic code needs to deal with and allows objects to be created in the same segment as their outer object (thus reducing the number of far pointers). Most getters/setters in the library don't return an error. Instead a get that fails due to an invalid pointer, out of bounds, etc will return the default value. A invalid set will be noop'ed. If you really need to know whether a set succeeds then errors are provided by the lower level Object methods. Since go doesn't have any templating, lists are created for the basic types and one level of named types. The list of basic types (e.g. List(UInt8), List(Text), etc) are all provided in this library. Lists of user named types are created with the user types (e.g. user struct Foo will create a Foo_List type). capnp schemas that use deeper lists (e.g. List(List(UInt8))) will use PointerList and the user will have to use the Object.ToList* functions to cast to the correct type. For adding documentation comments to the generated code, there's the doc annotation. This annotation adds the comment to a struct, enum or field so that godoc will pick it up. For Example: capnpc-go will generate the following for structs: For each group a typedef is created with a different method set for just the groups fields: That way the following may be used to access a field in a group: Note that Group accessors just cast the type and so have no overhead Named unions are treated as a group with an inner unnamed union. Unnamed unions generate an enum Type_Which and a corresponding Which() function: Which() should be checked before using the getters, and the default case must always be handled. Setters for single values will set the union discriminator as well as set the value. For voids in unions, there is a void setter that just sets the discriminator. For example: For groups in unions, there is a group setter that just sets the discriminator. This must be called before the group getter can be used to set values. For example: capnpc-go generates enum values in all caps. For example in the capnp file: In the generated capnp.go file: In addition an enum.String() function is generated that will convert the constants to a string for debugging or logging purposes. By default, the enum name is used as the tag value, but the tags can be customized with a $Go.tag or $Go.notag annotation. For example: In the generated go file:
Package ogórek(*) is a library for decoding/encoding Python's pickle format. Use Decoder to decode a pickle from input stream, for example: Use Encoder to encode an object as pickle into output stream, for example: The following table summarizes mapping of basic types in between Python and Go: Python classes and instances are mapped to Class and Call, for example: In particular on Go side it is thus by default safe to decode pickles from untrusted sources(^). Over the time the pickle stream format was evolving. The original protocol version 0 is human-readable with versions 1 and 2 extending the protocol in backward-compatible way with binary encodings for efficiency. Protocol version 2 is the highest protocol version that is understood by standard pickle module of Python2. Protocol version 3 added ways to represent Python bytes objects from Python3(~). Protocol version 4 further enhances on version 3 and completely switches to binary-only encoding. Protocol version 5 added support for out-of-band data(%). Please see https://docs.python.org/3/library/pickle.html#data-stream-format for details. On decoding ogórek detects which protocol is being used and automatically handles all necessary details. On encoding, for compatibility with Python2, by default ogórek produces pickles with protocol 2. Bytes thus, by default, will be unpickled as str on Python2 and as bytes on Python3. If an earlier protocol is desired, or on the other hand, if Bytes needs to be encoded efficiently (protocol 2 encoding for bytes is far from optimal), and compatibility with pure Python2 is not an issue, the protocol to use for encoding could be explicitly specified, for example: See EncoderConfig.Protocol for details. Pickle was originally created for serialization in ZODB (http://zodb.org) object database, where on-disk objects can reference each other similarly to how one in-RAM object can have a reference to another in-RAM object. When a pickle with such persistent reference is decoded, ogórek represents the reference with Ref placeholder similarly to Class and Call. However it is possible to hook into decoding and process such references in application specific way, for example loading the referenced object from the database: Similarly, for encoding, an application can hook into serialization process and turn pointers to some in-RAM objects into persistent references. Please see DecoderConfig.PersistentLoad and EncoderConfig.PersistentRef for details. -------- (*) ogórek is Polish for "pickle". (+) for Python2 both str and unicode are decoded into string with Python str being considered as UTF-8 encoded. Correspondingly for protocol ≤ 2 Go string is encoded as UTF-8 encoded Python str, and for protocol ≥ 3 as unicode. (~) bytes can be produced only by Python3 or zodbpickle (https://pypi.org/project/zodbpickle), not by standard Python2. Respectively, for protocol ≤ 2, what ogórek produces is unpickled as bytes by Python3 or zodbpickle, and as str by Python2. (^) contrary to Python implementation, where malicious pickle can cause the decoder to run arbitrary code, including e.g. os.system("rm -rf /"). (%) ogórek currently does not support out-of-band data.
Package mediatailor provides the API client, operations, and parameter types for AWS MediaTailor. Use the AWS Elemental MediaTailor SDKs and CLI to configure scalable ad insertion and linear channels. With MediaTailor, you can assemble existing content into a linear stream and serve targeted ads to viewers while maintaining broadcast quality in over-the-top (OTT) video applications. For information about using the service, including detailed information about the settings covered in this guide, see the AWS Elemental MediaTailor User Guide. Through the SDKs and the CLI you manage AWS Elemental MediaTailor configurations and channels the same as you do through the console. For example, you specify ad insertion behavior and mapping information for the origin server and the ad decision server (ADS).
Package ivs provides the API client, operations, and parameter types for Amazon Interactive Video Service. The Amazon Interactive Video Service (IVS) API is REST compatible, using a standard HTTP API and an Amazon Web Services EventBridge event stream for responses. JSON is used for both requests and responses, including errors. The API is an Amazon Web Services regional service. For a list of supported regions and Amazon IVS HTTPS service endpoints, see the Amazon IVS pagein the Amazon Web Services General Reference. All API request parameters and URLs are case sensitive. For a summary of notable documentation changes in each release, see Document History. Allowed Header Values Accept: application/json Accept-Encoding: gzip, deflate Content-Type: application/json Key Concepts Channel — Stores configuration data related to your live stream. You first create a channel and then use the channel’s stream key to start your live stream. Stream key — An identifier assigned by Amazon IVS when you create a channel, which is then used to authorize streaming. Treat the stream key like a secret, since it allows anyone to stream to the channel. Playback key pair — Video playback may be restricted using playback-authorization tokens, which use public-key encryption. A playback key pair is the public-private pair of keys used to sign and validate the playback-authorization token. Recording configuration — Stores configuration related to recording a live stream and where to store the recorded content. Multiple channels can reference the same recording configuration. Playback restriction policy — Restricts playback by countries and/or origin sites. For more information about your IVS live stream, also see Getting Started with IVS Low-Latency Streaming. A tag is a metadata label that you assign to an Amazon Web Services resource. A tag comprises a key and a value, both set by you. For example, you might set a tag as topic:nature to label a particular video category. See Tagging Amazon Web Services Resources for more information, including restrictions that apply to tags and "Tag naming limits and requirements"; Amazon IVS has no service-specific constraints beyond what is documented there. Tags can help you identify and organize your Amazon Web Services resources. For example, you can use the same tag for different resources to indicate that they are related. You can also use tags to manage access (see Access Tags). The Amazon IVS API has these tag-related endpoints: TagResource, UntagResource, and ListTagsForResource. The following resources support tagging: Channels, Stream Keys, Playback Key Pairs, and Recording Configurations. At most 50 tags can be applied to a resource. Note the differences between these concepts: Authentication is about verifying identity. You need to be authenticated to sign Amazon IVS API requests. Authorization is about granting permissions. Your IAM roles need to have permissions for Amazon IVS API requests. In addition, authorization is needed to view Amazon IVS private channels. (Private channels are channels that are enabled for "playback authorization.") All Amazon IVS API requests must be authenticated with a signature. The Amazon Web Services Command-Line Interface (CLI) and Amazon IVS Player SDKs take care of signing the underlying API calls for you. However, if your application calls the Amazon IVS API directly, it’s your responsibility to sign the requests. You generate a signature using valid Amazon Web Services credentials that have permission to perform the requested action. For example, you must sign PutMetadata requests with a signature generated from a user account that has the ivs:PutMetadata permission. For more information: Authentication and generating signatures — See Authenticating Requests (Amazon Web Services Signature Version 4)in the Amazon Web Services General Reference. Managing Amazon IVS permissions — See Identity and Access Managementon the Security page of the Amazon IVS User Guide. Amazon Resource Names (ARNs) ARNs uniquely identify AWS resources. An ARN is required when you need to specify a resource unambiguously across all of AWS, such as in IAM policies and API calls. For more information, see Amazon Resource Namesin the AWS General Reference. CreateChannel GetChannel BatchGetChannel ListChannels UpdateChannel DeleteChannel CreatePlaybackRestrictionPolicy DeletePlaybackRestrictionPolicy GetPlaybackRestrictionPolicy ListPlaybackRestrictionPolicies UpdatePlaybackRestrictionPolicy For more information, see Setting Up Private Channels in the Amazon IVS User Guide. ImportPlaybackKeyPair GetPlaybackKeyPair ListPlaybackKeyPairs DeletePlaybackKeyPair StartViewerSessionRevocation BatchStartViewerSessionRevocation CreateRecordingConfiguration GetRecordingConfiguration ListRecordingConfigurations DeleteRecordingConfiguration GetStream GetStreamSession ListStreams ListStreamSessions StopStream PutMetadata CreateStreamKey GetStreamKey BatchGetStreamKey ListStreamKeys DeleteStreamKey TagResource UntagResource ListTagsForResource
Package ebml implements the encoder and decoder of Extensible Binary Meta Language (EBML). The package supports Marshal and Unmarshal between tagged struct and EBML binary stream. WebM block data writer is provided as webm sub-package.
Package attributevalue provides marshaling and unmarshaling utilities to convert between Go types and Amazon DynamoDB Streams AttributeValues. These utilities allow you to marshal slices, maps, structs, and scalar values to and from the AttributeValue type. These utilities make it easier to convert between AttributeValue and Go types when working with DynamoDB resources. This package only converts between Go types and DynamoDBStreams AttributeValue. See the feature/dynamodb/attributevalue package for converting to DynamoDB AttributeValue types. The FromDynamoDBMap, FromDynamoDBList, and FromDynamoDB functions provide the conversion utilities to convert a DynamoDB AttributeValue type to a DynamoDBStreams AttributeValue type. Use these utilities when you need to convert the AttributeValue type between the two APIs. To unmarshal an AttributeValue to a Go type you can use the Unmarshal, UnmarshalList, UnmarshalMap, and UnmarshalListOfMaps functions. The List and Map functions are specialized versions of the Unmarshal function for unmarshal slices and maps of Attributevalues. The following example will unmarshal Items result from the DynamoDBStreams GetRecords operation. The items returned will be unmarshaled into the slice of the Records struct. The AttributeValue Marshal and Unmarshal functions support the `dynamodbav` struct tag by default. Additional tags can be enabled with the EncoderOptions and DecoderOptions, TagKey option. See the Marshal and Unmarshal function for information on how struct tags and fields are marshaled and unmarshaled.
Package sodium is a wrapper for https://github.com/jedisct1/libsodium Most of the functions is a method to the "Bytes" type. They are grouped below: Sender sign a message with its SecretKey and the receiver can verify the Signature by sender's PublicKey (Ed25519) (Ed25519ph) An anonymous can encrypt a message with an ephemeral key pair and reveiver's PublicKey. The receiver can decrypt the message with its SecretKey. Only the receiver is authenticated. (X25519-XSalsa20-Poly1305) Authenticated Box can be used to pass encrypt message from a known sender to a known receiver. The sender and the receiver are both authenticated to each other. A one-time shared nonce is also generated and passed to protect the key pairs and messages. (X25519-XSalsa20-Poly1305) Server and Client exchange their public key and calculates a common session key with their own secret key. (rx || tx = BLAKE2B-512(p.n || client_pk || server_pk)) One holder of a secret key authenticates the message with MAC. (HMAC-SHA512256) Use a secret key and a nonce to protect the key, messages could be encrypted into a SecretBox. The encrypted data's intergrity is checked when decryption. (XSalsa20-Poly1305) Use a secret key and a nonce to protect the key, messages could be encrypted. Optional additional data and the message is authenticited with an authentication tag. Both intergrity and authenticity is checked when decryption. The decryption would not be performed unless the authentication tag is verified. AEADCP* (ChaCha20-Poly1305_IETF) AEADXCP* (XChaCha20-Poly1305_IETF) High-level streaming API that use AEAD construct. Using `SecretStreamTag_Sync` to indicate the end of message. And this is useful for application to parse the message earlier. Rekeying is automatic. However using `SecretStreamTag_Rekey` explicitly ask for rekeying. Typical usage is sending chunks with `SecretStreamTag_Message`. XCP (XChaCha20-Poly1305_IETF) Deriving subkeys from a single high-entropy key KDF (BLAKE2B)
Goserial is a simple go package to allow you to read and write from the serial port as a stream of bytes. It aims to have the same API on all platforms, including windows. As an added bonus, the windows package does not use cgo, so you can cross compile for windows from another platform. Unfortunately goinstall does not currently let you cross compile so you will have to do it manually: Currently there is very little in the way of configurability. You can set the baud rate. Then you can Read(), Write(), or Close() the connection. Read() will block until at least one byte is returned. Write is the same. There is currently no exposed way to set the timeouts, though patches are welcome. Currently all ports are opened with 8 data bits, 1 stop bit, no parity, no hardware flow control, and no software flow control. This works fine for many real devices and many faux serial devices including usb-to-serial converters and bluetooth serial ports. You may Read() and Write() simulantiously on the same connection (from different goroutines). Example usage:
Package gostream allows to replace the standard net stack in Go with [LibP2P](https://github.com/libp2p/libp2p) streams. Given a libp2p.Host, gostream provides Dial() and Listen() methods which return implementations of net.Conn and net.Listener. Instead of the regular "host:port" addressing, `gostream` uses a Peer ID, and rather than a raw TCP connection, gostream will use libp2p's net.Stream. This means your connections will take advantage of LibP2P's multi-routes, NAT transversal and stream multiplexing. Note that LibP2P hosts cannot dial to themselves, so there is no possibility of using the same Host as server and as client.
Package mp3 provides decoding of mp3 files into their underlying frames. It is primarily intended for streaming tasks with minimal internal buffering and no requirement to seek. The implementation started as a reworking of github.com/badgerodon/mp3, and has also drawn from Konrad Windszus' excellent article on mp3 frame parsing http://www.codeproject.com/Articles/8295/MPEG-Audio-Frame-Header TODO CRC isn't currently checked.
Package jrpc2 implements a server and a client for the JSON-RPC 2.0 protocol defined by http://www.jsonrpc.org/specification. The *Server type implements a JSON-RPC server. A server communicates with a client over a channel.Channel, and dispatches client requests to user-defined method handlers. Method handlers are functions with this signature: A server finds the handler for a request by looking up its method name in a jrpc2.Assigner. A Handler decodes request parameters using the UnmarshalParams method on the Request: The handler package uses reflection to adapt functions that do not have this type to handlers. For example, given: call handler.New to convert Add to a handler function: The handler package also provides handler.Map, which implements the Assigner interface with a Go map. To advertise this function under the name "Add": Equipped with an Assigner we can now construct a Server: To start the server, we need a channel.Channel. Implementations of the Channel interface handle the framing, transmission, and receipt of JSON messages. The channel package implements some common framing disciplines for byte streams like pipes and sockets. For this example, we'll use a channel that communicates over stdin and stdout, with messages delimited by newlines: Now we can start the server: The Start method does not block. A server runs until its channel closes or it is stopped explicitly by calling srv.Stop(). To wait for the server to finish, call: This reports the error that led to the server exiting. The code for this example is available from tools/examples/adder/adder.go: Interact with the server by sending JSON-RPC requests on stdin, such as for example: The *Client type implements a JSON-RPC client. A client communicates with a server over a channel.Channel, and is safe for concurrent use by multiple goroutines. It supports batched requests and may have arbitrarily many pending requests in flight simultaneously. To create a client we need a channel: To send a single RPC, use the Call method: Call blocks until the response is received. Errors returned by the server have concrete type *jrpc2.Error. To issue a batch of concurrent requests, use the Batch method: Batch blocks until all the responses are received. An error from the Batch call reflects an error in sending the request: The caller must check each response separately for errors from the server. Responses are returned in the same order as the Spec values, save that notifications are omitted. To decode the result from a successful response, use its UnmarshalResult method: To close a client and discard all its pending work, call cli.Close(). A JSON-RPC notification is a one-way request: The client sends the request to the server, but the server does not reply. Use the Notify method of a client to send a notification: A notification is complete once it has been sent. Notifications can also be sent as part of a batch request: On the server, notifications are handled just like other requests, except that the return value is discarded once the handler returns. If a handler does not want to do anything for a notification, it can query the request: The example above shows a server with one method. A handler.Map works for any number of distinctly-named methods: Maps may be combined with the handler.ServiceMap type to export multiple services from the same server: This assigner dispatches "Math.Add" and "Math.Mul" to the arithmetic functions, and "Status.Get" to the getStatus function. A ServiceMap splits the method name on the first period ("."), and you may nest ServiceMaps more deeply if you require a more complex hierarchy. A Server issues concurrent requests to handlers in parallel, up to the limit given by the Concurrency field in ServerOptions. Two requests (either calls or notifications) are concurrent if they arrive as part of the same batch. In addition, two calls are concurrent if the time intervals between the arrival of the request objects and delivery of the response objects overlap. The server may issue concurrent requests to their handlers in any order. Non-concurrent requests are processed in order of arrival. Notifications, in particular, can only be concurrent with other requests in the same batch. This ensures a client that sends a notification can be sure its notification will be fully processed before any subsequent calls are issued to their handlers. These rules imply that the client cannot rely on the execution order of calls that overlap in time: If the caller needs to ensure that call A completes before call B starts, it must wait for A to return before invoking B. Per the JSON-RPC 2.0 spec, method names beginning with "rpc." are reserved by the implementation. By default, a server does not dispatch these methods to its assigner. In this configuration, the server exports a "rpc.serverInfo" method taking no parameters and returning a jrpc2.ServerInfo value. Setting the DisableBuiltin server option to true removes special treatment of "rpc." method names, and disables the rpc.serverInfo handler. When this option is true, method names beginning with "rpc." will be dispatched to the assigner like any other method. The AllowPush server option allows handlers to "push" requests back to the client. This is a non-standard extension of JSON-RPC used by some applications such as the Language Server Protocol (LSP). When this option is enabled, the server's Notify and Callback methods send requests back to the client. Otherwise, those methods will report an error: A method handler may use jrpc2.ServerFromContext to access the server from its context, and then invoke these methods on it. On the client side, the OnNotify and OnCallback options in jrpc2.ClientOptions provide hooks to which any server requests are delivered, if they are set. Since not all clients support server push, handlers should set a timeout when using the server Callback method; otherwise the callback may block forever for a client response that will never arrive. Both the Server and the Client use the standard context package to plumb cancellation and other request-specific metadata in handlers and calls. On the server, the context passed to a handler is automatically cancelled when the server shuts down or when the server's CancelRequest method is invoked for that request. In addition, the NewContext server option can be used to supply default timeouts or other context metadata. On the client, the context passed to the Call and CallResult methods governs the lifetime of the call. If the context ends before the call is complete, the client will terminate the call and report an error. Note that cancellation on the client is not automatically propagated to the server, as JSON-RPC does not define a standard mechanism to do so. One typical approach (used by LSP, for example) is to define a separate method on the server to handle cancellation requests. If an OnCancel hook is set in the ClientOptions, the client calls it when the context for a Call ends before the server has responded. This can be used to forward cancellation to the server separately.
Package sgr providers formatters to write ANSI escape sequences that can format and colorize text. The exact appearance of text, and whether or not various characteristics are even supported, will depend on the terminal capabilities. In particular, writing to a regular file will suppress all ANSI escapes sequences. Package sgr will default to no-color when there is doubt about whether or not the output stream can support ANSI sequences. There are some convenience methods to print common options, such as bold or red text. To get full control over the color and formatting of some text, first create a Style, and then use Formatter.Style. Calls to functions and methods in this package do not directly result in any io. To get the colorized text, the return values of certain methods must be printed using package fmt. For methods simply taking a string, such as Formatter.Bold, the returned value must be printed using either the verb %v or %s. All of the options for %s are supported, including width, precision, and flags. For methods that take a format string plus arguments, such as Formatter.Boldf, the returned value must still be printed using either %v or %s. However, no options are supported. To control options such as width and precision, use the format string passed to Formatter.Boldf Selection of the correct formatter depends on several environment variables. Primarily, package sgr checks the value of TERM to see if any color space attributes are present. ANSI compatibility is always assumed, but the color depth will be adjusted. However, if TERM is equal to 'dumb', then all formatting will be suppressed. Some terminals use COLORTERM to advertise 24-bit capability. If COLORTERM is equal to 'truecolor' or '24bit', then the formatter will support a 24-bit color depth. Finally, if the environment variable NO_COLOR is present, no matter its value, the color depth will be 1-bit. This does not disable all formatting, so text attributes such as bold and italic are still possible, but all colors will be suppressed.
Package multibuf implements buffer optimized for streaming large chunks of data, multiple reads and optional partial buffering to disk.
Package circonusgometrics provides instrumentation for your applications in the form of counters, gauges and histograms and allows you to publish them to Circonus A counter is a monotonically-increasing, unsigned, 64-bit integer used to represent the number of times an event has occurred. By tracking the deltas between measurements of a counter over intervals of time, an aggregation layer can derive rates, acceleration, etc. A gauge returns instantaneous measurements of something using signed, 64-bit integers. This value does not need to be monotonic. A histogram tracks the distribution of a stream of values (e.g. the number of seconds it takes to handle requests). Circonus can calculate complex analytics on these. A period push to a Circonus httptrap is confgurable.
snappystream wraps snappy-go and supplies a Reader and Writer for the snappy framed stream format: