Package tarmac is a client package for WASM functions running within a Tarmac server. This package provides user-friendly functions that wrap the Web Assembly Procedure Call (waPC) based functions of Tarmac. Guest WASM functions running inside Tarmac can use this library to call back the Tarmac host and perform host-level actions such as storing data within the database, logging specific data, or looking up configurations.
Package pq is a pure Go Postgres driver for the database/sql package. In most cases clients will use the database/sql package instead of using this package directly. For example: You can also connect to a database using a URL. For example: Similarly to libpq, when establishing a connection using pq you are expected to supply a connection string containing zero or more parameters. A subset of the connection parameters supported by libpq are also supported by pq. Additionally, pq also lets you specify run-time parameters (such as search_path or work_mem) directly in the connection string. This is different from libpq, which does not allow run-time parameters in the connection string, instead requiring you to supply them in the options parameter. For compatibility with libpq, the following special connection parameters are supported: Valid values for sslmode are: See http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING for more information about connection string parameters. Use single quotes for values that contain whitespace: A backslash will escape the next character in values: Note that the connection parameter client_encoding (which sets the text encoding for the connection) may be set but must be "UTF8", matching with the same rules as Postgres. It is an error to provide any other value. In addition to the parameters listed above, any run-time parameter that can be set at backend start time can be set in the connection string. For more information, see http://www.postgresql.org/docs/current/static/runtime-config.html. Most environment variables as specified at http://www.postgresql.org/docs/current/static/libpq-envars.html supported by libpq are also supported by pq. If any of the environment variables not supported by pq are set, pq will panic during connection establishment. Environment variables have a lower precedence than explicitly provided connection parameters. database/sql does not dictate any specific format for parameter markers in query strings, and pq uses the Postgres-native ordinal markers, as shown above. The same marker can be reused for the same parameter: pq does not support the LastInsertId() method of the Result type in database/sql. To return the identifier of an INSERT (or UPDATE or DELETE), use the Postgres RETURNING clause with a standard Query or QueryRow call: For more details on RETURNING, see the Postgres documentation: For additional instructions on querying see the documentation for the database/sql package. pq may return errors of type *pq.Error which can be interrogated for error details: See the pq.Error type for details. You can perform bulk imports by preparing a statement returned by pq.CopyIn (or pq.CopyInSchema) in an explicit transaction (sql.Tx). The returned statement handle can then be repeatedly "executed" to copy data into the target table. After all data has been processed you should call Exec() once with no arguments to flush all buffered data. Any call to Exec() might return an error which should be handled appropriately, but because of the internal buffering an error returned by Exec() might not be related to the data passed in the call that failed. CopyIn uses COPY FROM internally. It is not possible to COPY outside of an explicit transaction in pq. Usage example: PostgreSQL supports a simple publish/subscribe model over database connections. See http://www.postgresql.org/docs/current/static/sql-notify.html for more information about the general mechanism. To start listening for notifications, you first have to open a new connection to the database by calling NewListener. This connection can not be used for anything other than LISTEN / NOTIFY. Calling Listen will open a "notification channel"; once a notification channel is open, a notification generated on that channel will effect a send on the Listener.Notify channel. A notification channel will remain open until Unlisten is called, though connection loss might result in some notifications being lost. To solve this problem, Listener sends a nil pointer over the Notify channel any time the connection is re-established following a connection loss. The application can get information about the state of the underlying connection by setting an event callback in the call to NewListener. A single Listener can safely be used from concurrent goroutines, which means that there is often no need to create more than one Listener in your application. However, a Listener is always connected to a single database, so you will need to create a new Listener instance for every database you want to receive notifications in. The channel name in both Listen and Unlisten is case sensitive, and can contain any characters legal in an identifier (see http://www.postgresql.org/docs/current/static/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS for more information). Note that the channel name will be truncated to 63 bytes by the PostgreSQL server. You can find a complete, working example of Listener usage at http://godoc.org/github.com/flynn/pq/listen_example.
Tlogdb is a trivial transparent log client and server. It is meant as more a starting point to be customized than a tool to be used directly. A transparent log is a tamper-proof, append-only, immutable log of data records. That is, if the server were to violate the “append-only, immutable” properties, that tampering would be detected by the client. For more about transparent logs, see https://research.swtch.com/tlog. To create a new log (new server state): The newlog command creates a new database in file (default tlog.db) containing an empty log and a newly generated public/private key pair for the server using the given name. The newlog command prints the newly generated public key. To see it again: To add a record named name to the log: To serve the authenticated log data: The default server address is localhost:6655. The client maintains a cache database both for performance (avoiding duplicate downloads) and for storing the server's public key and the most recently seen log head. To create a new client cache: The newcache command creates a new database in file (default tlogclient.db) and stores the given public key for later use. The key should be the output of the tlogdb's server commands newlog or publickey, described above. To look up a record in the log: The default server address is again localhost:6655. The protocol between client and server is the same as used in the Go module checksum database, documented at https://golang.org/design/25530-sumdb#checksum-database. There are three endpoints: /latest serves a signed tree head; /lookup/NAME looks up the given name, and /tile/* serves log tiles. Putting the various commands together in a Unix shell:
Tlogdb is a trivial transparent log client and server. It is meant as more a starting point to be customized than a tool to be used directly. A transparent log is a tamper-proof, append-only, immutable log of data records. That is, if the server were to violate the “append-only, immutable” properties, that tampering would be detected by the client. For more about transparent logs, see https://research.swtch.com/tlog. To create a new log (new server state): The newlog command creates a new database in file (default tlog.db) containing an empty log and a newly generated public/private key pair for the server using the given name. The newlog command prints the newly generated public key. To see it again: To add a record named name to the log: To serve the authenticated log data: The default server address is localhost:6655. The client maintains a cache database both for performance (avoiding duplicate downloads) and for storing the server's public key and the most recently seen log head. To create a new client cache: The newcache command creates a new database in file (default tlogclient.db) and stores the given public key for later use. The key should be the output of the tlogdb's server commands newlog or publickey, described above. To look up a record in the log: The default server address is again localhost:6655. The protocol between client and server is the same as used in the Go module checksum database, documented at https://golang.org/design/25530-sumdb#checksum-database. There are three endpoints: /latest serves a signed tree head; /lookup/NAME looks up the given name, and /tile/* serves log tiles. Putting the various commands together in a Unix shell:
Command goat provides an implementation of a BitTorrent tracker, written in Go. goat can be built using Go 1.1+. It can be downloaded, built, and installed, simply by running: In addition, goat depends on a MySQL server for data storage. After creating a database and user for goat, its database schema may be imported from the SQL files located in 'res/'. goat will not run unless MySQL is installed, and a database and user are properly configured for its use. Optionally, goat can be built to use ql (https://github.com/cznic/ql) as its storage backend. This is done by supplying the 'ql' tag in the go get command: A blank ql database file is located under 'res/ql/goat.db', and will be copied to '~/.config/goat/goat.db' on UNIX systems. goat is now able to use ql as its storage backend, for those who do not wish to use an external, MySQL backend. goat is capable of listening for torrent traffic in three modes: HTTP, HTTPS, and UDP. HTTP/HTTPS are the recommended methods, and are required in order for goat to serve its API, and to allow use of private tracker passkeys. HTTP is considered the standard mode of operation for goat. HTTP allows gathering a great number of metrics, use of passkeys, use of a client whitelist, and access to goat's RESTful API, when configured. For most trackers, this will be the only listener which is necessary in order for goat to function properly. The HTTPS listener provides a method to encrypt traffic to the tracker, but must be used with caution. Unless the SSL certificate in use is signed by a proper certificate authority, it will distress most clients, and they may outright refuse to announce to it. If you are in possession of a certificate signed by a certificate authority, this mode may be more ideal, as it provides added security for your clients. The UDP listener is the most unusual method of the three, and should only be used for public trackers. The BitTorrent UDP tracker protocol specifies a very specific packet format, meaning that additional information or parameters cannot be packed into a UDP datagram in a standard way. The UDP tracker may be the fastest and least bandwidth-intensive, but as stated, should only be used for public trackers. A new feature goat added to goat in order to allow better interoperability with many languages is a RESTful API, which is served using the HTTP or HTTPS listeners. This API enables easy retrieval of tracker statistics, while allowing goat to run as a completely independent process. It should be noted that the API is only enabled when configured, and when a HTTP or HTTPS listener is enabled. Without a transport mechanism, the API will be inaccessible. The API features several modes of authentication, including HTTP Basic for login and HMAC-SHA1 other calls. Upon logging into the API using HTTP Basic with a username and password pair, an API public key and secret will be generated. The public key is used as the username for HTTP Basic authentication, and the secret key is used to calculate a HMAC-SHA1 signature for the password. As part of API signature generation, a random nonce value must be generated and added to the request. It is added to the password portion of the HTTP Basic request, and also to the string which is used to create the signature. Nonce values must be changed on every request, or the request will fail. The current pseudocode format of the HMAC-SHA1 signature is as follows: The proper format for a HTTP Basic request is as follows: When the public key, nonce, and API signature are sent via HTTP Basic, the server will verify the signature. Successful authentication will allow access to the API. This list contains all API calls currently recognized by goat. Each call must be authenticated using the aforementioned methods. Request an API public key and secret key for this user. The public key, user ID, and secret key are used to authenticate further API calls. The expire time indicates when this key is set to expire. Further API calls will extend the expiration time. Retrieve a list of all files tracked by goat. Some extended attributes are not added to reduce strain on database, and to provide a more general overview. Retrieve extended attributes about a specific file with matching ID. This provides counts for number of completions, seeders, leechers, and a list of fileUser relationships associated with a given file. Retrieve a variety of metrics about the current status of goat, including its PID, hostname, memory usage, number of HTTP/UDP hits, etc. Create a user with the specified username, password, and torrent limit. Reterieve a list of all users registered to goat, including their ID, torrent limit, and username. Retrieve information about a single user with matching ID, including their ID, torrent limit, and username. goat is configured using a JSON file, which will be created under '~/.config/goat/config.json' on UNIX systems. Here is an example configuration, describing the settings available to the user.
Package goBolt implements drivers for the Neo4J Bolt Protocol Versions 1-4. There are some limitations to the types of collections the internalDriver supports. Specifically, maps should always be of type map[string]interface{} and lists should always be of type []interface{}. It doesn't seem that the Bolt protocol supports uint64 either, so the biggest number it can send right now is the int64 max. The URL format is: `bolt://(user):(password)@(host):(port)` Schema must be `bolt`. User and password is only necessary if you are authenticating. TLS is supported by using query parameters on the connection string, like so: `bolt://host:port?tls=true&tls_no_verify=false` The supported query params are: * timeout - the number of seconds to set the connection timeout to. Defaults to 60 seconds. * tls - Set to 'true' or '1' if you want to use TLS encryption * tls_no_verify - Set to 'true' or '1' if you want to accept any server certificate (for testing, not secure) * tls_ca_cert_file - path to a custom ca cert for a self-signed TLS cert * tls_cert_file - path to a cert file for this client (need to verify this is processed by Neo4j) * tls_key_file - path to a key file for this client (need to verify this is processed by Neo4j) Errors returned from the API support wrapping, so if you receive an error from the library, it might be wrapping other errors. You can get the innermost error by using the `InnerMost` method. Failure messages from Neo4J are reported, along with their metadata, as an error. In order to get the failure message metadata from a wrapped error, you can do so by calling `err.(*errors.Error).InnerMost().(messages.FailureMessage).Metadata` If there is an error with the database connection, you should get a sql/internalDriver ErrBadConn as per the best practice recommendations of the Golang SQL Driver. However, this error may be wrapped, so you might have to call `InnerMost` to get it, as specified above.
Package isokey allows you to make and verify API keys without a database connection via HMAC signatures. The keys are scalable and persistent. All information is stored in the key, and with the client.
Package sdk is a client package for WASM functions running within a Tarmac server. This package provides user-friendly functions that wrap the Web Assembly Procedure Call (waPC) based functions of Tarmac. Guest WASM functions running inside Tarmac can use this library to call back the Tarmac host and perform host-level actions such as storing data within the database, logging specific data, or looking up configurations.
Package radix implements an asynchronous Redis client. Client is a structure for accessing a Redis database. After establishing a connection with NewClient, commands can be executed with Client.Command. Client.Command returns a Reply with different methods for accessing the retrieved values. Client.MultiCommand can be used for sending multiple commands in a single request and Client.Transaction offers a simple way for executing atomic requests. Client.Subscription returns a Subscription that can be used for listening published messages.
Package pgconn is a low-level PostgreSQL database driver. pgconn provides lower level access to a PostgreSQL connection than a database/sql or pgx connection. It operates at nearly the same level is the C library libpq. Use Connect to establish a connection. It accepts a connection string in URL or DSN and will read the environment for libpq style environment variables. ExecParams and ExecPrepared execute a single query. They return readers that iterate over each row. The Read method reads all rows into memory. Exec and ExecBatch can execute multiple queries in a single round trip. They return readers that iterate over each query result. The ReadAll method reads all query results into memory. All potentially blocking operations take a context.Context. If a context is canceled while the method is in progress the method immediately returns. In most circumstances, this will close the underlying connection. The CancelRequest method may be used to request the PostgreSQL server cancel an in-progress query without forcing the client to abort.
Origins is an open source bi-temporal database for storing and retrieving facts about the state of things. It supports "time-travel" queries, aggregate views, and change detection. The primary interface is the CLI which can be installed by running: This package defines some of the primitive structures and algorithms for manipulating, reading, and writing facts and is used to build higher level client APIs. Fact sorting is done using the Timsort algorithm which is hybrid algorithm of merge sort and insertion sort. This is chosen because facts are generally partially sorted by entity since facts are derived from higher level objects. For comparison, comparators for the default Quicksort algorithm are implemented for benchmarking purposes. Wikipedia: https://en.wikipedia.org/wiki/Timsort Comparison to quicksort: http://stackoverflow.com/a/19587279/407954
The Escher HTTP request signing framework is intended to provide a secure way for clients to sign HTTP requests, and servers to check the integrity of these messages. The goal of the protocol is to introduce an authentication solution for REST API services, that is more secure than the currently available protocols. RFC 2617 (HTTP Authentication) defines Basic and Digest Access Authentication. They’re widely used, but Basic Access Authentication doesn’t encrypt the secret and doesn’t add integrity checks to the requests. Digest Access Authentication sends the secret encrypted, but the algorithm with creating a checksum with a nonce and using md5 should not be considered highly secure these days, and as with Basic Access Authentication, there’s no way to check the integrity of the message. RFC 6749 (OAuth 2.0 Authorization) enables a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf. This is not helpful for a machine-to-machine communication situation, like a REST API authentication, because typically there’s no third-party user involved. Additionally, after a token is obtained from the authorization endpoint, it is used with no encryption, and doesn’t provide integration checking, or prevent repeating messages. OAuth 2.0 is a stateful protocol which needs a database to store the tokens for client sessions. Amazon and other service providers created protocols addressing these issues, however there is no public standard with open source implementations available from them. As Escher is based on a publicly documented, widely, in-the-wild used protocol, the specification does not include novelty techniques. 2. Signing an HTTP Request Escher defines a stateless signature generation mechanism. The signature is calculated from the key parts of the HTTP request, a service identifier string called credential scope, and a client key and client secret. The signature generation steps are: canonicalizing the request, creating a string to calculate the signature, and adding the signature to the original request. Escher supports two hash algorithms: SHA256 and SHA512 designed by the NSA (U.S. National Security Agency). 2.1. Canonicalizing the Request In order to calculate a checksum from the key HTTP request parts, the HTTP request method, the request URI, the query parts, the headers, and the request body have to be canonicalized. The output of the canonicalization step will be a string including the request parts separated by LF (line feed, “n”) characters. The string will be used to calculate a checksum for the request. 2.1.1. The HTTP method The HTTP method defined by RFC2616 (Hypertext Transfer Protocol) is case sensitive, and must be available in upper case, no transformation has to be applied: POST 2.1.2. The Path The path is the absolute path of the URL. Starts with a slash (/) character, and does not include the query part (and the question mark). Escher follows the rules defined by RFC3986 (Uniform Resource Identifier) to normalize the path. Basically it means: Convert relative paths to absolute, remove redundant path components. URI-encode each path components: the “reserved characters” defined by RFC3986 (Uniform Resource Identifier) have to be kept as they are (no encoding applied) all other characters have to be percent encoded, including SPACE (to %20, instead of +) non-ASCII, UTF-8 characters should be percent encoded to 2 or more pieces (á to %C3%A1) percent encoded hexadecimal numbers have to be upper cased (eg: a%c2%b1b to a%C2%B1b) Normalize empty paths to /. For example: 2.1.3. The Query String RFC3986 (Uniform Resource Identifier) should provide guidance for canonicalization of the query string, but here’s the complete list of the rules to be applied: URI-encode each query parameter names and values the “reserved characters” defined by RFC3986 (Uniform Resource Identifier) have to be kept as they are (no encoding applied) all other characters have to be percent encoded, including SPACE (to %20, instead of +) non-ASCII, UTF-8 characters should be percent encoded to 2 or more pieces (á to %C3%A1) percent encoded hexadecimal numbers have to be upper cased (eg: a%c2%b1b to a%C2%B1b) Normalize empty query strings to empty string. Sort query parameters by the encoded parameter names (ASCII order). Do not shorten parameter values if their parameter name is the same (key=B&key=A is a valid output), the order of parameters in a URL may be significant (this is not defined by the HTTP standard). Separate parameter names and values by = signs, include = for empty values, too Separate parameters by & For example: To canonicalize the headers, the following rules have to be followed: Lower case the header names. Separate header names and values by a :, with no spaces. Sort header names to alphabetical order (ASCII). Group headers with the same names into a header, and separate their values by commas, without sorting. Trim header values, keep all the spaces between quote characters ("). For example: 2.1.5. Signed Headers The list of headers to include when calculating the signature. Lower cased value of header names, separated by ;, like this: date;host 2.1.6. Body Checksum A checksum for the request body, aka the payload has to be calculated. Escher supports SHA-256 and SHA-512 algorithms for checksum calculation. If the request contains no body, an empty string has to be used as the input for the hash algorithm. The selected algorithm will be added later to the authorization header, so the server will be able to use the same algorithm for validation. The checksum of the body has to be presented as a lower cased hexadecimal string, for example: 2.1.7. Concatenating the Canonicalized Parts All the steps above produce a row of data, except the headers canonicalization, as it creates one row per headers. These have to be concatenated with LF (line feed, “n”) characters into a string. An example: 2.2. Creating the Signature The next step is creating another string which will be directly used to calculate the signature. 2.2.1. Algorithm ID The algorithm ID comes from the algo_prefix (default value is ESR) and the algorithm used to calculate checksums during the signing process. The string algo_prefix, “HMAC”, and the algorithm name should be concatenated with dashes, like this: 2.2.2. Long Date The long date is the request date in the ISO 8601 basic format, like YYYYMMDD + T + HHMMSS + Z. Note that the basic format uses no punctuation. Example is: This date has to be added later, too, as a date header (default header name is X-Escher-Date). 2.2.3. Date and Credential Scope Next information is the short date, and the credential scope concatenated with a / character. The short date is the request date’s date part, an ISO 8601 basic formatted representation, the credential scope is defined by the service. Example: This will be added later, too, as part of the authorization header (default header name is X-Escher-Auth). 2.2.4. Checksum of the Canonicalized Request Take the output of step 2.1.7., and create a checksum from the canonicalized checksum string. This checksum has to be represented as a lower cased hexadecimal string, too. Something like this will be an output: 2.2.5. Concatenating the Signing String Concatenate the outputs of steps 2.2. with LF characters. Example output: 2.2.6. The Signing Key The signing key is based on the algo_prefix, the client secret, the parts of the credential scope, and the request date. Take the algo_prefix, concatenate the client secret to it. First apply the HMAC algorithm to the request date, then apply the actual value on each of the credential scope parts (splitted at /). The end result is a binary signing key. Pseudo code: 2.2.7. Create the Signature The signature is created from the output of steps 2.2.5. (Signing String) and 2.2.6. (Signing Key). With the selected algorithm, create a checksum. It has to be represented as a lower cased hexadecimal string. Something like this will be an output: 2.3. Adding the Signature to the Request The final step of the Escher signing process is adding the Signature to the request. Escher adds a new header to the request, by default, the header name is X-Escher-Auth. The header value will include the algorithm ID (see 2.2.1.), the client key, the short date and the credential scope (see 2.2.3.), the signed headers string (see 2.1.5.) and finally the signature (see 2.2.7.). The values of this inputs have to be concatenated like this: 3. Presigning a URL The URL presigning process is very similar to the request signing procedure. But for a URL, there are no headers, no request body, so the calculation of the Signature is different. Also, the Signature cannot be added to the headers, but is included as query parameters. A significant difference is that the presigning allows defining an expiration time. By default, it is 86400 secs, 24 hours. The current time and the expiration time will be included in the URL, and the server has to check if the URL is expired. 3.1. Canonicalizing the URL to Presign The canonicalization for URL presigning is the same process as for HTTP requests, in this section we will cover the differences only. 3.1.1. The HTTP method The HTTP method for presigned URLs is fixed to: For example: 3.1.3. The Query String The query is coming from the URL, but the algorithm, credentials, date, expiration time, and signed headers have to be added to the query parts. 3.1.4. The Headers A URL has no headers, Escher creates the Host header based on the URL’s domain information, and adds it to the canonicalized request. For example: 3.1.5. Signed Headers It will be host, as that’s the only header included. Example:
Package hord provides a simple and extensible interface for interacting with various database systems in a uniform way. Hord is designed to be a database-agnostic library that provides a common interface for interacting with different database systems. It allows developers to write code that is decoupled from the underlying database technology, making it easier to switch between databases or support multiple databases in the same application. To use Hord, import it as follows: To create a database client, you need to import and use the appropriate driver package along with the `hord` package. For example, to use the Redis driver: Each driver provides its own `Dial` function to establish a connection to the database. Refer to the specific driver documentation for more details. Once you have a database client, you can use it to perform various database operations. The API is consistent across different drivers. Refer to the `hord.Database` interface documentation for a complete list of available methods. Hord provides common error types and constants for consistent error handling across drivers. Refer to the `hord` package documentation for more information on error handling. Contributions to Hord are welcome! If you want to add support for a new database driver or improve the existing codebase, please refer to the contribution guidelines in the project's repository.
Package httplog logs http requests and responses. It’s highly configurable, e.g. in production, log all response and requests, but don’t log the body or headers, in your dev environment log everything and so on. httplog also has different ways to log depending on your preference — structured logging via JSON, relational database logging or just plain standard library logging. httplog has logic to turn on/off logging based on options you can either pass in to the middleware handler or from a JSON input file included with the library. httplog offers three middleware choices, each of which adhere to fairly common middleware patterns: a simple HandlerFunc (`LogHandlerFunc`), a function (`LogHandler`) that takes a handler and returns a handler (aka Constructor) (`func (http.Handler) http.Handler`) often used with alice (https://github.com/justinas/alice) and finally a function (`LogAdapter`) that returns an Adapter type. An `httplog.Adapt` function and `httplog.Adapter` type are provided. Beyond logging request and response elements, httplog creates a unique id for each incoming request (using xid (https://github.com/rs/xid)) and sets it (and a few other key request elements) into the request context. You can access these context items using provided helper functions, including a function that returns an audit struct you can add to response payloads that provide clients with helpful information for support. !!!!WARNING!!!! - This package works, but is something I wrote a long time ago and really needs to be updated. I logged Issue #8 to some day address this.
Package clustersql is an SQL "meta"-Driver - A clustering, implementation- agnostic wrapper for any backend implementing "database/sql/driver". It does (latency-based) load-balancing and error-recovery over the registered set of nodes. It is assumed that database-state is transparently replicated over all nodes by some database-side clustering solution. This driver ONLY handles the client side of such a cluster. This package simply multiplexes the driver.Open() function of sql/driver to every attached node. The function is called on each node, returning the first successfully opened connection. (Any connections opening subsequently will be closed.) If opening does not succeed for any node, the latest error gets returned. Any other errors will be masked by default. However, any given latest error for any attached node will remain exposed through expvar, as well as some basic counters and timestamps. To make use of this kind of clustering, use this package with any backend driver implementing "database/sql/driver" like so: There is currently no way around instanciating the backend driver explicitly You can perform backend-driver specific settings such as Create a new clustering driver with the backend driver Add nodes, including driver-specific name format, in this case Go-MySQL DSN. Here, we add three nodes belonging to a galera (https://mariadb.com/kb/en/mariadb/documentation/replication-cluster-multi-master/galera/) cluster Make the clusterDriver available to the go sql interface under an arbitrary name Open the registered clusterDriver with an arbitrary DSN string (not used) Continue to use the sql interface as documented at http://golang.org/pkg/database/sql/ Before using this in production, you should configure your cluster details in config.toml and run Note however, that non-failure of the above is no guarantee for a correctly set-up cluster. Finally, you SHOULD set db.MaxIdleConns and db.MaxOpenConns to a non-zero value. Although the sql driver usually does a good job of doing its own pooling, file descriptors can leak in corner cases (of which this library might constitue an example).
Package otgorm allows for the wrapping of GORM calls to databases with OpenTelemetry tracing spans. You only need to create your GORM db client and pass that into otgorm.WithContext along with context.Context(). If there is a parent span referenced within the context the GORM call will be a child span.
Package pgconn is a low-level PostgreSQL database driver. pgconn provides lower level access to a PostgreSQL connection than a database/sql or pgx connection. It operates at nearly the same level is the C library libpq. Use Connect to establish a connection. It accepts a connection string in URL or DSN and will read the environment for libpq style environment variables. ExecParams and ExecPrepared execute a single query. They return readers that iterate over each row. The Read method reads all rows into memory. Exec and ExecBatch can execute multiple queries in a single round trip. They return readers that iterate over each query result. The ReadAll method reads all query results into memory. All potentially blocking operations take a context.Context. If a context is canceled while the method is in progress the method immediately returns. In most circumstances, this will close the underlying connection. The CancelRequest method may be used to request the PostgreSQL server cancel an in-progress query without forcing the client to abort.
Package pq is a pure Go Postgres driver for the database/sql package. In most cases clients will use the database/sql package instead of using this package directly. For example: You can also connect to a database using a URL. For example: Similarly to libpq, when establishing a connection using pq you are expected to supply a connection string containing zero or more parameters. A subset of the connection parameters supported by libpq are also supported by pq. Additionally, pq also lets you specify run-time parameters (such as search_path or work_mem) directly in the connection string. This is different from libpq, which does not allow run-time parameters in the connection string, instead requiring you to supply them in the options parameter. For compatibility with libpq, the following special connection parameters are supported: Valid values for sslmode are: See http://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING for more information about connection string parameters. Use single quotes for values that contain whitespace: A backslash will escape the next character in values: Note that the connection parameter client_encoding (which sets the text encoding for the connection) may be set but must be "UTF8", matching with the same rules as Postgres. It is an error to provide any other value. In addition to the parameters listed above, any run-time parameter that can be set at backend start time can be set in the connection string. For more information, see http://www.postgresql.org/docs/current/static/runtime-config.html. Most environment variables as specified at http://www.postgresql.org/docs/current/static/libpq-envars.html supported by libpq are also supported by pq. If any of the environment variables not supported by pq are set, pq will panic during connection establishment. Environment variables have a lower precedence than explicitly provided connection parameters. The pgpass mechanism as described in http://www.postgresql.org/docs/current/static/libpq-pgpass.html is supported, but on Windows PGPASSFILE must be specified explicitly. database/sql does not dictate any specific format for parameter markers in query strings, and pq uses the Postgres-native ordinal markers, as shown above. The same marker can be reused for the same parameter: pq does not support the LastInsertId() method of the Result type in database/sql. To return the identifier of an INSERT (or UPDATE or DELETE), use the Postgres RETURNING clause with a standard Query or QueryRow call: For more details on RETURNING, see the Postgres documentation: For additional instructions on querying see the documentation for the database/sql package. Parameters pass through driver.DefaultParameterConverter before they are handled by this package. When the binary_parameters connection option is enabled, []byte values are sent directly to the backend as data in binary format. This package returns the following types for values from the PostgreSQL backend: All other types are returned directly from the backend as []byte values in text format. pq may return errors of type *pq.Error which can be interrogated for error details: See the pq.Error type for details. You can perform bulk imports by preparing a statement returned by pq.CopyIn (or pq.CopyInSchema) in an explicit transaction (sql.Tx). The returned statement handle can then be repeatedly "executed" to copy data into the target table. After all data has been processed you should call Exec() once with no arguments to flush all buffered data. Any call to Exec() might return an error which should be handled appropriately, but because of the internal buffering an error returned by Exec() might not be related to the data passed in the call that failed. CopyIn uses COPY FROM internally. It is not possible to COPY outside of an explicit transaction in pq. Usage example: PostgreSQL supports a simple publish/subscribe model over database connections. See http://www.postgresql.org/docs/current/static/sql-notify.html for more information about the general mechanism. To start listening for notifications, you first have to open a new connection to the database by calling NewListener. This connection can not be used for anything other than LISTEN / NOTIFY. Calling Listen will open a "notification channel"; once a notification channel is open, a notification generated on that channel will effect a send on the Listener.Notify channel. A notification channel will remain open until Unlisten is called, though connection loss might result in some notifications being lost. To solve this problem, Listener sends a nil pointer over the Notify channel any time the connection is re-established following a connection loss. The application can get information about the state of the underlying connection by setting an event callback in the call to NewListener. A single Listener can safely be used from concurrent goroutines, which means that there is often no need to create more than one Listener in your application. However, a Listener is always connected to a single database, so you will need to create a new Listener instance for every database you want to receive notifications in. The channel name in both Listen and Unlisten is case sensitive, and can contain any characters legal in an identifier (see http://www.postgresql.org/docs/current/static/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS for more information). Note that the channel name will be truncated to 63 bytes by the PostgreSQL server. You can find a complete, working example of Listener usage at http://godoc.org/github.com/lib/pq/example/listen.
Package lq implements a spatial database which stores objects each of which is associated with a 2D point (a location in a 2D space). The points serve as the "search key" for the associated object. It is intended to efficiently answer "circle inclusion" queries, also known as "range queries": basically questions like: Which objects are within a radius R of the location L? In this context, "efficiently" means significantly faster than the naive, brute force O(n) testing of all known points. Additionally it is assumed that the objects move along unpredictable paths, so that extensive preprocessing (for example, constructing a Delaunay triangulation of the point set) may not be practical. The implementation is a "bin lattice": a 2D rectangular array of brick-shaped (rectangles) regions of space. Each region is represented by a pointer to a (possibly empty) doubly-linked list of objects. All of these sub-bricks are the same size. All bricks are aligned with the global coordinate axes. Terminology used here: the region of space associated with a bin is called a sub-brick. The collection of all sub-bricks is called the super-brick. The super-brick should be specified to surround the region of space in which (almost) all the key-points will exist. If key-points move outside the super-brick everything will continue to work, but without the speed advantage provided by the spatial subdivision. For more details about how to specify the super-brick's position, size and subdivisions see NewDB below. Overview of usage: an application using this facility to perform locality queries over objects of type myStruct would first create a database with: Then, call Attach for each objects to attach to the database. Attach returns a 'proxy' object, which is a link between the user object and its representation in the locality database. When a client object moves, the application calls Update with the new location. Update is a method of the lq.Proxy object, that's why the the proxy object is generally kept within the user object, though it can be managed separately: To perform a query, DB.ForEachWithinRadius is passed a user function which will be called for all client objects in the locality. See Func below for more detail. The DB.FindNearestInRadius function can be used to find a single nearest neighbor using the database. Note that "locality query" is also known as neighborhood query, neighborhood search, near neighbor search, and range query. Author: Aurélien Rainone Based on original work of: Craig Reynolds
Package migration automatically handles versioning of a database schema by applying a series of migrations supplied by the client. It uses features only from the database/sql package, so it tries to be driver independent. However, to track the version of the database, it is necessary to execute some SQL. I've made an effort to keep those queries simple, but if they don't work with your database, you may override them. This package works by applying a series of migrations to a database. Once a migration is created, it should never be changed. Every time a database is opened with this package, all necessary migrations are executed in a single transaction. If any part of the process fails, an error is returned and the transaction is rolled back so that the database is left untouched. (Note that for this to be useful, you'll need to use a database that supports rolling back changes to your schema. Notably, MySQL does not support this, although SQLite and PostgreSQL do.) The version of a database is defined as the number of migrations applied to it.
Package opaque implements OPAQUE, an asymmetric password-authenticated key exchange protocol that is secure against pre-computation attacks. It enables a client to authenticate to a server without ever revealing its password to the server. Protocol details can be found on the IETF RFC page (https://datatracker.ietf.org/doc/draft-irtf-cfrg-opaque) and on the GitHub specification repository (https://github.com/cfrg/draft-irtf-cfrg-opaque). Example_Configuration shows how to instantiate a configuration, which is used to initialize clients and servers from. Configurations MUST remain the same for a given client between sessions, or the client won't be able to execute the protocol. Configurations can be serialized and deserialized, if you need to save, hardcode, or transmit it. Example_Deserialization demonstrates a couple of ways to deserialize OPAQUE protocol messages. Message interpretation depends on the configuration context it's exchanged in. Hence, we need the corresponding configuration. We can then directly deserialize messages from a Configuration or pass them to Client or Server instances which can do it as well. You must know in advance what message you are expecting, and call the appropriate deserialization function. Example_FakeResponse shows how to counter some client enumeration attacks by faking an existing client entry. Precompute the fake client record, and return it when no valid record was found. Use this with the server's LoginInit function whenever a client wants to retrieve an envelope but a client entry does not exist. Failing to do so results in an attacker being able to enumerate users. Example_LoginKeyExchange demonstrates in a single function the interactions between a client and a server for the login phase. This is of course a proof-of-concept demonstration, as client and server execute separately. Example_Registration demonstrates in a single function the interactions between a client and a server for the registration phase. This is of course a proof-of-concept demonstration, as client and server execute separately. The server outputs a ClientRecord and the credential identifier. The latter is a unique identifier for a given client (e.g. database entry ID), and that must absolutely stay the same for the whole client existence and never be reused. Example_ServerSetup shows how to set up the long term values for the OPAQUE server. - The secret OPRF seed can be unique for each client or the same for all, but must be the same for a given client between registration and all login sessions. - The AKE key pair can also be the same for all clients or unique, but must be the same for a given client between registration and all login sessions.
Package ldredis provides a Redis-backed persistent data store for the LaunchDarkly Go SDK. For more details about how and why you can use a persistent data store, see: https://docs.launchdarkly.com/v2.0/docs/using-a-persistent-feature-store To use the Redis data store with the LaunchDarkly client: The default Redis pool configuration uses an address of localhost:6379, a maximum of 16 concurrent connections, and blocking connection requests. You may customize the configuration by using the methods of the StoreBuilder returned by DataStore: Note that CacheSeconds() is not a method of StoreBuilder, but rather a method of ldcomponents.PersistentDataStore(), because the caching behavior is provided by the SDK for all database integrations. For advanced customization of the underlying Redigo client, use StoreBuilder methods such as StoreBuilder.DialOptions and StoreBuilder.Pool. Note that some Redis client features can also be specified as part of the URL: Redigo supports the redis:// syntax (https://www.iana.org/assignments/uri-schemes/prov/redis), which can include a password and a database number, as well as rediss:// (https://www.iana.org/assignments/uri-schemes/prov/rediss), which enables TLS. If you are also using Redis for other purposes, the data store can coexist with other data as long as you are not using the same keys. By default, the keys used by the data store will always start with "launchdarkly:"; you can change this to another prefix if desired.
Package couch implements a client for a CouchDB database. Version 0.1 focuses on basic operations, proper conflict management, error handling and replication. Not part of this version are attachment handling, general statistics and optimizations, change detection and creating views. Most of the features are accessible using the generic Do() function, though. Getting started: Every document in CouchDB has to be identifiable by a document id and a revision id. Two types already implement this interface called Identifiable: Doc and DynamicDoc. Doc can be used as an anonymous field in your own struct. DynamicDoc is a type alias for map[string]interface{}, use it when your documents have no implicit schema at all. To make code examples easier to follow, there will be no explicit error handling in these examples even though it's fully supported throughout the API. Insert() will create a new document if it doesn't have an id yet: After the operation the final id and revision id will be written back to p. That's why you can now just edit p and call Insert() again which will save the same document under a new revision. After this edit, p will contain the latest revision id. Note that it is possible that this second edit fails because someone else edited and saved the same document in the meantime. You will be notified of this in form of an error and you should then first retrieve the latest document revision to see the changes of this lost update: CouchDB doesn't edit documents in-place but adds a complete revision for each edit. That's why you will be correctly informed of any lost update. Because CouchDB supports multi-master replication of databases, it is possible that conflicts like the one described above can't be avoided. CouchDB is not going to interrupt replication because of a lost update. Let's say you have two instances running, maybe a central one and a mobile one and both are kept in sync by replication. Now let's assume you edit a document on your mobile DB and someone else edits the same document on the central DB. After you've come online again, you use bi-directional replication to sync the databases. CouchDB will now create a branch structure for your document, similar to version control systems. Your document has two conflicting revisions and in this case they can't necessarily be resolved automatically. This client helps you with a number of methods to resolve such an issue quickly. Read more about the conflict model http://docs.couchdb.org/en/latest/replication/conflicts.html Continuing with above example, replicate the database: Now, on the other database, edit the document (note that it has the same id there): Now edit the document on the first database. Retrieve it first to make sure it has the correct revision id: Now replicate anotherDB back to our first database: Now we have two conflicting versions of a document. Only you as the editor can decide whether "LatestAnna" or "AnotherAnna" is correct. To detect this conflict there are a number of methods. First, you can just ask a document: You probably want to have a look at the revisions in your preferred format, use Revisions() to unmarshal the revision data into a slice of a custom data type: Pick one of the revisions or create a new document to solve the conflict: That's it. You can detect conflicts like these throughout your database using: Errors returned by CouchDB will be converted into a Go error. Its regular Error() method will then return a combination of the shortform (e.g. bad_request) as well as the longer and more specific description. To be able to identify a specific error within your application, use ErrorType() to get the shortform only.
Package helper: common project provides commonly used helper utility functions, custom utility types, and third party package wrappers. common project helps code reuse, and faster composition of logic without having to delve into commonly recurring code logic and related testings. common project source directories and brief description: + /ascii = helper types and/or functions related to ascii manipulations. + /crypto = helper types and/or functions related to encryption, decryption, hashing, such as rsa, aes, sha, tls etc. + /csv = helper types and/or functions related to csv file manipulations. + /rest = helper types and/or functions related to http rest api GET, POST, PUT, DELETE actions invoked from client side. + /tcp = helper types providing wrapped tcp client and tcp server logic. - /wrapper = wrappers provides a simpler usage path to third party packages, as well as adding additional enhancements. /helper-conv.go = helpers for data conversion operations. /helper-db.go = helpers for database data type operations. /helper-emv.go = helpers for emv chip card related operations. /helper-io.go = helpers for io related operations. /helper-net.go = helpers for network related operations. /helper-num.go = helpers for numeric related operations. /helper-other.go = helpers for misc. uncategorized operations. /helper-reflect.go = helpers for reflection based operations. /helper-regex.go = helpers for regular express related operations. /helper-str.go = helpers for string operations. /helper-struct.go = helpers for struct related operations. /helper-time.go = helpers for time related operations. /helper-uuid.go = helpers for generating globally unique ids.
go-rexster-client is a Rexster graph database client for Go. See https://github.com/tinkerpop/rexster/wiki for more information about Rexster. It implements a subset of the Rexster REST API: https://github.com/tinkerpop/rexster/wiki/Basic-REST-API. To use the *Batch functions, you must have the batch kibble installed. See https://github.com/tinkerpop/rexster/tree/master/rexster-kibbles/batch-kibble for more information. In the Rexster source dir, this means copying batch-kibble-2.4.0-SNAPSHOT.jar to ./rexster-server/target/rexster-server-2.4.0-SNAPSHOT-standalone/lib/.
Package freeGeoIP or go-freeGeoIP is a Golang client for Free IP Geolocation information API with inbuilt cache support to increase the 15k per hour rate limit of the application https://freegeoip.app/ By default, the client will cache the IP Geolocation information for 24 hours, but the expiry can be set manually. If you want set the information cache with no expiration time set the expiry function to nil. You can use the package using the following command: freegeoip.app provides a free IP geolocation API for software developers. It uses a database of IP addresses that are associated to cities along with other relevant information like time zone, latitude and longitude. You're allowed up to 15,000 queries per hour by default. Once this limit is reached, all of your requests will result in HTTP 403, forbidden, until your quota is cleared. The HTTP API takes GET requests in the following schema: Supported formats are: csv, xml, json and jsonp. If no IP or hostname is provided, then your own IP is looked up. Contributors are more than welcome and much appreciated. Please feel free to open a PR to improve anything you don't like, or would like to add. Please make your changes in a specific branch and request to pull into master! If you can please make sure all the changes work properly and does not affect the existing functioning. No PR is too small! Even the smallest effort is countable. This project is licensed under the MIT license.(https://github.com/Shivam010/go-freeGeoIP/blob/master/LICENSE)
Package lddynamodb provides a DynamoDB-backed persistent data store for the LaunchDarkly Go SDK. For more details about how and why you can use a persistent data store, see: https://docs.launchdarkly.com/sdk/features/storing-data/dynamodb#go To use the DynamoDB data store with the LaunchDarkly client: By default, the data store uses a basic DynamoDB client configuration that is equivalent to doing this: This default configuration will only work if your AWS credentials and region are available from AWS environment variables and/or configuration files. If you want to set those programmatically or modify any other configuration settings, you can use the methods of the lddynamodb.DataStoreBuilder returned by lddynamodb.DataStore(). For example: Note that CacheSeconds() is not a method of lddynamodb.DataStoreBuilder, but rather a method of ldcomponents.PersistentDataStore(), because the caching behavior is provided by the SDK for all database integrations. If you are also using DynamoDB for other purposes, the data store can coexist with other data in the same table as long as you use the Prefix option to make each application use different keys. However, it is advisable to configure separate tables in DynamoDB, for better control over permissions and throughput.
Package metrics is a telemetry client designed for Uber's software networking team. It prioritizes performance on the hot path and integration with both push- and pull-based collection systems. Like Prometheus and Tally, it supports metrics tagged with arbitrary key-value pairs. Like Prometheus, but unlike Tally, metric names should be relatively long and descriptive - generally speaking, metrics from the same process shouldn't share names. (See the documentation for the Root struct below for a longer explanation of the uniqueness rules.) For example, prefer "grpc_successes_by_procedure" over "successes", since "successes" is common and vague. Where relevant, metric names should indicate their unit of measurement (e.g., "grpc_success_latency_ms"). Counters represent monotonically increasing values, like a car's odometer. Gauges represent point-in-time readings, like a car's speedometer. Both counters and gauges expose not only write operations (set, add, increment, etc.), but also atomic reads. This makes them easy to integrate directly into your business logic: you can use them anywhere you'd otherwise use a 64-bit atomic integer. This package doesn't support analogs of Tally's timer or Prometheus's summary, because they can't be accurately aggregated at query time. Instead, it approximates distributions of values with histograms. These require more up-front work to set up, but are typically more accurate and flexible when queried. See https://prometheus.io/docs/practices/histograms/ for a more detailed discussion of the trade-offs involved. Plain counters, gauges, and histograms have a fixed set of tags. However, it's common to encounter situations where a subset of a metric's tags vary constantly. For example, you might want to track the latency of your database queries by table: you know the database cluster, application name, and hostname at process startup, but you need to specify the table name with each query. To model these situations, this package uses vectors. Each vector is a local cache of metrics, so accessing them is quite fast. Within a vector, all metrics share a common set of constant tags and a list of variable tags. In our database query example, the constant tags are cluster, application, and hostname, and the only variable tag is table name. Usage examples are included in the documentation for each vector type. This package integrates with StatsD- and M3-based collection systems by periodically pushing differential updates. (Users can integrate with other push-based systems by implementing the push.Target interface.) It integrates with pull-based collectors by exposing an HTTP handler that supports Prometheus's text and protocol buffer exposition formats. Examples of both push and pull integration are included in the documentation for the root struct's Push and ServeHTTP methods. If you're unfamiliar with Tally and Prometheus, you may want to consult their documentation: