Package stats is a extension for the goldmark (http://github.com/yuin/goldmark). This extension adds text statistic function.
Package rtcp implements encoding and decoding of RTCP packets according to RFCs 3550 and 5506. RTCP is a sister protocol of the Real-time Transport Protocol (RTP). Its basic functionality and packet structure is defined in RFC 3550. RTCP provides out-of-band statistics and control information for an RTP session. It partners with RTP in the delivery and packaging of multimedia data, but does not transport any media data itself. The primary function of RTCP is to provide feedback on the quality of service (QoS) in media distribution by periodically sending statistics information such as transmitted octet and packet counts, packet loss, packet delay variation, and round-trip delay time to participants in a streaming multimedia session. An application may use this information to control quality of service parameters, perhaps by limiting flow, or using a different codec. Decoding RTCP packets: Encoding RTCP packets:
Package iostat presents I/O and CPU statistics.
Package elasticache provides the client and types for making API requests to Amazon ElastiCache. Amazon ElastiCache is a web service that makes it easier to set up, operate, and scale a distributed cache in the cloud. With ElastiCache, customers get all of the benefits of a high-performance, in-memory cache with less of the administrative burden involved in launching and managing a distributed cache. The service makes setup, scaling, and cluster failure handling much simpler than in a self-managed cache deployment. In addition, through integration with Amazon CloudWatch, customers get enhanced visibility into the key performance statistics associated with their cache and can receive alarms if a part of their cache runs hot. See https://docs.aws.amazon.com/goto/WebAPI/elasticache-2015-02-02 for more information on this service. See elasticache package documentation for more information. https://docs.aws.amazon.com/sdk-for-go/api/service/elasticache/ To Amazon ElastiCache with the SDK use the New function to create a new service client. With that client you can make API requests to the service. These clients are safe to use concurrently. See the SDK's documentation for more information on how to use the SDK. https://docs.aws.amazon.com/sdk-for-go/api/ See aws.Config documentation for more information on configuring SDK clients. https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config See the Amazon ElastiCache client ElastiCache for more information on creating client for this service. https://docs.aws.amazon.com/sdk-for-go/api/service/elasticache/#New
Load configuration file. misc helpers HTTP API to control probe service for representing collected data. HTTP client with timeouts. Client from standard library lacks support of timeouts then we use this one. Code taken from https://gist.github.com/dmichael/5710968 and also explained here http://stackoverflow.com/questions/16895294/how-to-set-timeout-for-http-get-requests-in-golang Stream parsers and keepers. OBSOLETED by webui-report.go Web reports generator Load configuration file. The code keeps streams statistics and program internal statistics. Statistics output to files and to JSON HTTP API. OBSOLETED by templates/*.tmpl Templates for webui Web UI. Reports generator Integration with Zabbix monitoring tool
Package Rye is a simple library to support http services. Rye provides a middleware handler which can be used to chain http handlers together while providing simple statsd metrics for use with a monitoring solution such as DataDog or other logging aggregators. Rye also provides some additional middleware handlers that are entirely optional but easily consumed using Rye. In order to use rye, you should vendor it and the statsd client within your project. Begin by importing the required libraries: Create a statsd client (if desired) and create a rye Config in order to pass in optional dependencies: Create a middleware handler. The purpose of the Handler is to keep Config and to provide an interface for chaining http handlers. Build your http handlers using the Handler type from the **rye** package. Here are some example (custom) handlers: Finally, to setup your handlers in your API Rye comes with built-in configurable `statsd` statistics that you could record to your favorite monitoring system. To configure that, you'll need to set up a `Statter` based on the `github.com/cactus/go-statsd-client` and set it in your instantiation of `MWHandler` through the `rye.Config`. When a middleware is called, it's timing is recorded and a counter is recorded associated directly with the http status code returned during the call. Additionally, an `errors` counter is also sent to the statter which allows you to count any errors that occur with a code equaling or above 500. Example: If you have a middleware handler you've created with a method named `loginHandler`, successful calls to that will be recorded to `handlers.loginHandler.2xx`. Additionally you'll receive stats such as `handlers.loginHandler.400` or `handlers.loginHandler.500`. You also will receive an increase in the `errors` count. If you're sending your logs into a system such as DataDog, be aware that your stats from Rye can have prefixes such as `statsd.my-service.my-k8s-cluster.handlers.loginHandler.2xx` or even `statsd.my-service.my-k8s-cluster.errors`. Just keep in mind your stats could end up in the destination sink system with prefixes. With Golang 1.7, a new feature has been added that supports a request specific context. This is a great feature that Rye supports out-of-the-box. The tricky part of this is how the context is modified on the request. In Golang, the Context is always available on a Request through `http.Request.Context()`. Great! However, if you want to add key/value pairs to the context, you will have to add the context to the request before it gets passed to the next Middleware. To support this, the `rye.Response` has a property called `Context`. This property takes a properly created context (pulled from the `request.Context()` function. When you return a `rye.Response` which has `Context`, the **rye** library will craft a new Request and make sure that the next middleware receives that request. Here's the details of creating a middleware with a proper `Context`. You must first pull from the current request `Context`. In the example below, you see `ctx := r.Context()`. That pulls the current context. Then, you create a NEW context with your additional context key/value. Finally, you return `&rye.Response{Context:ctx}` Now in a later middleware, you can easily retrieve the value you set! For another simple example, look in the JWT middleware - it adds the JWT into the context for use by other middlewares. It uses the `CONTEXT_JWT` key to push the JWT token into the `Context`. Rye comes with various pre-built middleware handlers. Pre-built middlewares source (and docs) can be found in the package dir following the pattern `middleware_*.go`. To use them, specify the constructor of the middleware as one of the middleware handlers when you define your routes: OR The JWT Middleware pushes the JWT token onto the Context for use by other middlewares in the chain. This is a convenience that allows any part of your middleware chain quick access to the JWT. Example usage might include a middleware that needs access to your user id or email address stored in the JWT. To access this `Context` variable, the code is very simple:
Package Rye is a simple library to support http services. Rye provides a middleware handler which can be used to chain http handlers together while providing simple statsd metrics for use with a monitoring solution such as DataDog or other logging aggregators. Rye also provides some additional middleware handlers that are entirely optional but easily consumed using Rye. In order to use rye, you should vendor it and the statsd client within your project. Begin by importing the required libraries: Create a statsd client (if desired) and create a rye Config in order to pass in optional dependencies: Create a middleware handler. The purpose of the Handler is to keep Config and to provide an interface for chaining http handlers. Build your http handlers using the Handler type from the **rye** package. Here are some example (custom) handlers: Finally, to setup your handlers in your API Rye comes with built-in configurable `statsd` statistics that you could record to your favorite monitoring system. To configure that, you'll need to set up a `Statter` based on the `github.com/cactus/go-statsd-client` and set it in your instantiation of `MWHandler` through the `rye.Config`. When a middleware is called, it's timing is recorded and a counter is recorded associated directly with the http status code returned during the call. Additionally, an `errors` counter is also sent to the statter which allows you to count any errors that occur with a code equaling or above 500. Example: If you have a middleware handler you've created with a method named `loginHandler`, successful calls to that will be recorded to `handlers.loginHandler.2xx`. Additionally you'll receive stats such as `handlers.loginHandler.400` or `handlers.loginHandler.500`. You also will receive an increase in the `errors` count. If you're sending your logs into a system such as DataDog, be aware that your stats from Rye can have prefixes such as `statsd.my-service.my-k8s-cluster.handlers.loginHandler.2xx` or even `statsd.my-service.my-k8s-cluster.errors`. Just keep in mind your stats could end up in the destination sink system with prefixes. With Golang 1.7, a new feature has been added that supports a request specific context. This is a great feature that Rye supports out-of-the-box. The tricky part of this is how the context is modified on the request. In Golang, the Context is always available on a Request through `http.Request.Context()`. Great! However, if you want to add key/value pairs to the context, you will have to add the context to the request before it gets passed to the next Middleware. To support this, the `rye.Response` has a property called `Context`. This property takes a properly created context (pulled from the `request.Context()` function. When you return a `rye.Response` which has `Context`, the **rye** library will craft a new Request and make sure that the next middleware receives that request. Here's the details of creating a middleware with a proper `Context`. You must first pull from the current request `Context`. In the example below, you see `ctx := r.Context()`. That pulls the current context. Then, you create a NEW context with your additional context key/value. Finally, you return `&rye.Response{Context:ctx}` Now in a later middleware, you can easily retrieve the value you set! For another simple example, look in the JWT middleware - it adds the JWT into the context for use by other middlewares. It uses the `CONTEXT_JWT` key to push the JWT token into the `Context`. Rye comes with various pre-built middleware handlers. Pre-built middlewares source (and docs) can be found in the package dir following the pattern `middleware_*.go`. To use them, specify the constructor of the middleware as one of the middleware handlers when you define your routes: OR The JWT Middleware pushes the JWT token onto the Context for use by other middlewares in the chain. This is a convenience that allows any part of your middleware chain quick access to the JWT. Example usage might include a middleware that needs access to your user id or email address stored in the JWT. To access this `Context` variable, the code is very simple:
Command goat provides an implementation of a BitTorrent tracker, written in Go. goat can be built using Go 1.1+. It can be downloaded, built, and installed, simply by running: In addition, goat depends on a MySQL server for data storage. After creating a database and user for goat, its database schema may be imported from the SQL files located in 'res/'. goat will not run unless MySQL is installed, and a database and user are properly configured for its use. Optionally, goat can be built to use ql (https://github.com/cznic/ql) as its storage backend. This is done by supplying the 'ql' tag in the go get command: A blank ql database file is located under 'res/ql/goat.db', and will be copied to '~/.config/goat/goat.db' on UNIX systems. goat is now able to use ql as its storage backend, for those who do not wish to use an external, MySQL backend. goat is capable of listening for torrent traffic in three modes: HTTP, HTTPS, and UDP. HTTP/HTTPS are the recommended methods, and are required in order for goat to serve its API, and to allow use of private tracker passkeys. HTTP is considered the standard mode of operation for goat. HTTP allows gathering a great number of metrics, use of passkeys, use of a client whitelist, and access to goat's RESTful API, when configured. For most trackers, this will be the only listener which is necessary in order for goat to function properly. The HTTPS listener provides a method to encrypt traffic to the tracker, but must be used with caution. Unless the SSL certificate in use is signed by a proper certificate authority, it will distress most clients, and they may outright refuse to announce to it. If you are in possession of a certificate signed by a certificate authority, this mode may be more ideal, as it provides added security for your clients. The UDP listener is the most unusual method of the three, and should only be used for public trackers. The BitTorrent UDP tracker protocol specifies a very specific packet format, meaning that additional information or parameters cannot be packed into a UDP datagram in a standard way. The UDP tracker may be the fastest and least bandwidth-intensive, but as stated, should only be used for public trackers. A new feature goat added to goat in order to allow better interoperability with many languages is a RESTful API, which is served using the HTTP or HTTPS listeners. This API enables easy retrieval of tracker statistics, while allowing goat to run as a completely independent process. It should be noted that the API is only enabled when configured, and when a HTTP or HTTPS listener is enabled. Without a transport mechanism, the API will be inaccessible. The API features several modes of authentication, including HTTP Basic for login and HMAC-SHA1 other calls. Upon logging into the API using HTTP Basic with a username and password pair, an API public key and secret will be generated. The public key is used as the username for HTTP Basic authentication, and the secret key is used to calculate a HMAC-SHA1 signature for the password. As part of API signature generation, a random nonce value must be generated and added to the request. It is added to the password portion of the HTTP Basic request, and also to the string which is used to create the signature. Nonce values must be changed on every request, or the request will fail. The current pseudocode format of the HMAC-SHA1 signature is as follows: The proper format for a HTTP Basic request is as follows: When the public key, nonce, and API signature are sent via HTTP Basic, the server will verify the signature. Successful authentication will allow access to the API. This list contains all API calls currently recognized by goat. Each call must be authenticated using the aforementioned methods. Request an API public key and secret key for this user. The public key, user ID, and secret key are used to authenticate further API calls. The expire time indicates when this key is set to expire. Further API calls will extend the expiration time. Retrieve a list of all files tracked by goat. Some extended attributes are not added to reduce strain on database, and to provide a more general overview. Retrieve extended attributes about a specific file with matching ID. This provides counts for number of completions, seeders, leechers, and a list of fileUser relationships associated with a given file. Retrieve a variety of metrics about the current status of goat, including its PID, hostname, memory usage, number of HTTP/UDP hits, etc. Create a user with the specified username, password, and torrent limit. Reterieve a list of all users registered to goat, including their ID, torrent limit, and username. Retrieve information about a single user with matching ID, including their ID, torrent limit, and username. goat is configured using a JSON file, which will be created under '~/.config/goat/config.json' on UNIX systems. Here is an example configuration, describing the settings available to the user.
Package peer provides a common base for creating and managing Bitcoin network peers. This package builds upon the wire package, which provides the fundamental primitives necessary to speak the bitcoin wire protocol, in order to simplify the process of creating fully functional peers. In essence, it provides a common base for creating concurrent safe fully validating nodes, Simplified Payment Verification (SPV) nodes, proxies, etc. A quick overview of the major features peer provides are as follows: Provides a basic concurrent safe bitcoin peer for handling bitcoin communications via the peer-to-peer protocol Full duplex reading and writing of bitcoin protocol messages Automatic handling of the initial handshake process including protocol version negotiation Asynchronous message queuing of outbound messages with optional channel for notification when the message is actually sent Flexible peer configuration Caller is responsible for creating outgoing connections and listening for incoming connections so they have flexibility to establish connections asthey see fit (proxies, etc) User agent name and version Bitcoin network Service support signalling (full nodes, bloom filters, etc) Maximum supported protocol version Ability to register callbacks for handling bitcoin protocol messages Inventory message batching and send trickling with known inventory detection and avoidance Automatic periodic keep-alive pinging and pong responses Random Nonce generation and self connection detection Proper handling of bloom filter related commands when the caller does not specify the related flag to signal support Disconnects the peer when the protocol version is high enough Does not invoke the related callbacks for older protocol versions Snapshottable peer statistics such as the total number of bytes read and written, the remote address, user agent, and negotiated protocol version Helper functions pushing addresses, getblocks, getheaders, and reject messages These could all be sent manually via the standard message output function, but the helpers provide additional nice functionality such as duplicate filtering and address randomization Ability to wait for shutdown/disconnect Comprehensive test coverage All peer configuration is handled with the Config struct. This allows the caller to specify things such as the user agent name and version, the bitcoin network to use, which services it supports, and callbacks to invoke when bitcoin messages are received. See the documentation for each field of the Config struct for more details. A peer can either be inbound or outbound. The caller is responsible for establishing the connection to remote peers and listening for incoming peers. This provides high flexibility for things such as connecting via proxies, acting as a proxy, creating bridge peers, choosing whether to listen for inbound peers, etc. NewOutboundPeer and NewInboundPeer functions must be followed by calling Connect with a net.Conn instance to the peer. This will start all async I/O goroutines and initiate the protocol negotiation process. Once finished with the peer call Disconnect to disconnect from the peer and clean up all resources. WaitForDisconnect can be used to block until peer disconnection and resource cleanup has completed. In order to do anything useful with a peer, it is necessary to react to bitcoin messages. This is accomplished by creating an instance of the MessageListeners struct with the callbacks to be invoke specified and setting the Listeners field of the Config struct specified when creating a peer to it. For convenience, a callback hook for all of the currently supported bitcoin messages is exposed which receives the peer instance and the concrete message type. In addition, a hook for OnRead is provided so even custom messages types for which this package does not directly provide a hook, as long as they implement the wire.Message interface, can be used. Finally, the OnWrite hook is provided, which in conjunction with OnRead, can be used to track server-wide byte counts. It is often useful to use closures which encapsulate state when specifying the callback handlers. This provides a clean method for accessing that state when callbacks are invoked. The QueueMessage function provides the fundamental means to send messages to the remote peer. As the name implies, this employs a non-blocking queue. A done channel which will be notified when the message is actually sent can optionally be specified. There are certain message types which are better sent using other functions which provide additional functionality. Of special interest are inventory messages. Rather than manually sending MsgInv messages via Queuemessage, the inventory vectors should be queued using the QueueInventory function. It employs batching and trickling along with intelligent known remote peer inventory detection and avoidance through the use of a most-recently used algorithm. In addition to the bare QueueMessage function previously described, the PushAddrMsg, PushGetBlocksMsg, PushGetHeadersMsg, and PushRejectMsg functions are provided as a convenience. While it is of course possible to create and send these message manually via QueueMessage, these helper functions provided additional useful functionality that is typically desired. For example, the PushAddrMsg function automatically limits the addresses to the maximum number allowed by the message and randomizes the chosen addresses when there are too many. This allows the caller to simply provide a slice of known addresses, such as that returned by the addrmgr package, without having to worry about the details. Next, the PushGetBlocksMsg and PushGetHeadersMsg functions will construct proper messages using a block locator and ignore back to back duplicate requests. Finally, the PushRejectMsg function can be used to easily create and send an appropriate reject message based on the provided parameters as well as optionally provides a flag to cause it to block until the message is actually sent. A snapshot of the current peer statistics can be obtained with the StatsSnapshot function. This includes statistics such as the total number of bytes read and written, the remote address, user agent, and negotiated protocol version. This package provides extensive logging capabilities through the UseLogger function which allows a btclog.Logger to be specified. For example, logging at the debug level provides summaries of every message sent and received, and logging at the trace level provides full dumps of parsed messages as well as the raw message bytes using a format similar to hexdump -C. This package supports all BIPS supported by the wire package. (https://godoc.org/github.com/p9c/pod/wire#hdr-Bitcoin_Improvement_Proposals) This example demonstrates the basic process for initializing and creating an outbound peer. Peers negotiate by exchanging version and verack messages. For demonstration, a simple handler for version message is attached to the peer.
Package dockerstats provides the ability to get currently running Docker container statistics, including memory and CPU usage. To get the statistics of running Docker containers, you can use the `Current()` function: Alternatively, you can use the `NewMonitor()` function to receive a constant stream of Docker container stats, available on the Monitor's `Stream` channel:
Package pf (Packet Filter) The FreeBSD operating system has multiple packet filter build-in. One of the packet filters was ported from OpenBSD and is called pf (packetfilter). Packet filtering restricts the types of packets that pass through network interfaces entering or leaving the host based on filter rules as described in. The packet filter can also replace addresses and ports of packets. Replacing source addresses and ports of outgoing packets is called NAT (Network Address Translation) and is used to connect an internal network (usually reserved address space) to an external one (the Internet) by making all connections to external hosts appear to come from the gateway. Replacing destination addresses and ports of incoming packets is used to redirect connections to different hosts and/or ports. A combination of both translations, bidirectional NAT, is also supported. This go module enables easy access to the packet filter inside the kernel. The FreeBSD kernel module responsible for implementing pf is called pf.ko. Since the kernel interface is different between the operating systems this version currently only works with FreeBSD. The packet filter creates the pseudo-device node /dev/pf, it allows userland processes to control the behavior of the packet filter through an ioctl(2) interface. There are commands to enable and disable the filter, load rulesets, add and remove individual rules or state table entries, and retrieve statistics. The most commonly used functions are covered by this library. Manipulations like loading a ruleset that involve more than a single ioctl(2) call require a so-called ticket, which prevents the occurrence of multiple concurrent manipulations. Tickets are modeled as transaction objects inside the library. Working with pf directly on a remote connection can cause you to loose the connection in case of a programming error. Make sure you have a second way to access the system e.g. a serial console.
Package Rye is a simple library to support http services. Rye provides a middleware handler which can be used to chain http handlers together while providing simple statsd metrics for use with a monitoring solution such as DataDog or other logging aggregators. Rye also provides some additional middleware handlers that are entirely optional but easily consumed using Rye. In order to use rye, you should vendor it and the statsd client within your project. Begin by importing the required libraries: Create a statsd client (if desired) and create a rye Config in order to pass in optional dependencies: Create a middleware handler. The purpose of the Handler is to keep Config and to provide an interface for chaining http handlers. Build your http handlers using the Handler type from the **rye** package. Here are some example (custom) handlers: Finally, to setup your handlers in your API Rye comes with built-in configurable `statsd` statistics that you could record to your favorite monitoring system. To configure that, you'll need to set up a `Statter` based on the `github.com/cactus/go-statsd-client` and set it in your instantiation of `MWHandler` through the `rye.Config`. When a middleware is called, it's timing is recorded and a counter is recorded associated directly with the http status code returned during the call. Additionally, an `errors` counter is also sent to the statter which allows you to count any errors that occur with a code equaling or above 500. Example: If you have a middleware handler you've created with a method named `loginHandler`, successful calls to that will be recorded to `handlers.loginHandler.2xx`. Additionally you'll receive stats such as `handlers.loginHandler.400` or `handlers.loginHandler.500`. You also will receive an increase in the `errors` count. If you're sending your logs into a system such as DataDog, be aware that your stats from Rye can have prefixes such as `statsd.my-service.my-k8s-cluster.handlers.loginHandler.2xx` or even `statsd.my-service.my-k8s-cluster.errors`. Just keep in mind your stats could end up in the destination sink system with prefixes. With Golang 1.7, a new feature has been added that supports a request specific context. This is a great feature that Rye supports out-of-the-box. The tricky part of this is how the context is modified on the request. In Golang, the Context is always available on a Request through `http.Request.Context()`. Great! However, if you want to add key/value pairs to the context, you will have to add the context to the request before it gets passed to the next Middleware. To support this, the `rye.Response` has a property called `Context`. This property takes a properly created context (pulled from the `request.Context()` function. When you return a `rye.Response` which has `Context`, the **rye** library will craft a new Request and make sure that the next middleware receives that request. Here's the details of creating a middleware with a proper `Context`. You must first pull from the current request `Context`. In the example below, you see `ctx := r.Context()`. That pulls the current context. Then, you create a NEW context with your additional context key/value. Finally, you return `&rye.Response{Context:ctx}` Now in a later middleware, you can easily retrieve the value you set! For another simple example, look in the JWT middleware - it adds the JWT into the context for use by other middlewares. It uses the `CONTEXT_JWT` key to push the JWT token into the `Context`. Rye comes with various pre-built middleware handlers. Pre-built middlewares source (and docs) can be found in the package dir following the pattern `middleware_*.go`. To use them, specify the constructor of the middleware as one of the middleware handlers when you define your routes: OR The JWT Middleware pushes the JWT token onto the Context for use by other middlewares in the chain. This is a convenience that allows any part of your middleware chain quick access to the JWT. Example usage might include a middleware that needs access to your user id or email address stored in the JWT. To access this `Context` variable, the code is very simple: