Immutable

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. The tree is not safe for concurrent use, and must be guarded by a Mutex or RWLock as appropriate - the exception is immutable trees returned by MutableTree.GetImmutable() which are safe for concurrent use as long as the version is not deleted via DeleteVersion(). Basic usage of MutableTree: Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. Basic usage of MutableTree. Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package freeze enables the "freezing" of data, similar to JavaScript's Object.freeze(). A frozen object cannot be modified; attempting to do so will result in an unrecoverable panic. Freezing is useful for providing soft guarantees of immutability. That is: the compiler can't prevent you from mutating an frozen object, but the runtime can. One of the unfortunate aspects of Go is its limited support for constants: structs, slices, and even arrays cannot be declared as consts. This becomes a problem when you want to pass a slice around to many consumers without worrying about them modifying it. With freeze, you can guard against these unwanted or intended behaviors. To accomplish this, the mprotect syscall is used. Sadly, this necessitates allocating new memory via mmap and copying the data into it. This performance penalty should not be prohibitive, but it's something to be aware of. In case it wasn't clear from the previous paragraph, this package is not intended to be used in production. A well-designed API is a much saner solution than freezing your data structures. I would even caution against using freeze in your automated testing, due to its platform-specific nature. freeze is best used for "one-off" debugging. Something like this: 1. Observe bug 2. Suspect that shared mutable data is the culprit 3. Call freeze.Object on the data after it is created 4. Run program again; it crashes 5. Inspect stack trace to identify where the data was modified 6. Fix bug 7. Remove call to freeze.Object Again: do not use freeze in production. It's a cool proof-of-concept, and it can be useful for debugging, but that's about it. Let me put it another way: freeze imports four packages: reflect, runtime, unsafe, and syscall (actually golang.org/x/sys/unix). Does that sound like a package you want to depend on? Okay, back to the real documention: Functions are provided for freezing the three "pointer types:" Pointer, Slice, and Map. Each function returns a copy of their input that is backed by protected memory. In addition, Object is provided for freezing recursively. Given a slice of pointers, Object will prevent modifications to both the pointer data and the slice data, while Slice merely does the latter. To freeze an object: Note that since foo does not contain any pointers, calling Pointer(f) would have the same effect here. It is recommended that, where convenient, you reassign the return value to its original variable, as with append. Otherwise, you will retain both the mutable original and the frozen copy. Likewise, to freeze a slice: Interfaces can also be frozen, since internally they are just pointers to objects. The effect of this is that the interface's pure methods can still be called, but impure methods cannot. Unfortunately the impurity of a given method is defined by the implementation, not the interface. Even a String method could conceivably modify some internal state. Furthermore, the caveat about unexported struct fields (see below) applies here, so many exported objects cannot be completely frozen. This package depends heavily on the internal representations of the slice and map types. These objects are not likely to change, but if they do, this package will break. In general, you can't call Object on the same object twice. This is because Object will attempt to rewrite the object's internal pointers -- which is a memory modification. Calling Pointer or Slice twice should be fine. Object cannot descend into unexported struct fields. It can still freeze the field itself, but if the field contains a pointer, the data it points to will not be frozen. Appending to a frozen slice will trigger a panic iff len(slice) < cap(slice). This is because appending to a full slice will allocate new memory. Unix is the only supported platform. Windows support is not planned, because it doesn't support a syscall analogous to mprotect.

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. Basic usage of MutableTree. Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Fully persistent data structures. A persistent data structure is a data structure that always preserves the previous version of itself when it is modified. Such data structures are effectively immutable, as their operations do not update the structure in-place, but instead always yield a new structure. Persistent data structures typically share structure among themselves. This allows operations to avoid copying the entire data structure.

This package is the root package of the govmomi library. The library is structured as follows: The minimal usable functionality is available through the vim25 package. It contains subpackages that contain generated types, managed objects, and all available methods. The vim25 package is entirely independent of the other packages in the govmomi tree -- it has no dependencies on its peers. The vim25 package itself contains a client structure that is passed around throughout the entire library. It abstracts a session and its immutable state. See the vim25 package for more information. The session package contains an abstraction for the session manager that allows a user to login and logout. It also provides access to the current session (i.e. to determine if the user is in fact logged in) The object package contains wrappers for a selection of managed objects. The constructors of these objects all take a *vim25.Client, which they pass along to derived objects, if applicable. The govc package contains the govc CLI. The code in this tree is not intended to be used as a library. Any functionality that govc contains that _could_ be used as a library function but isn't, _should_ live in a root level package. Other packages, such as "event", "guest", or "license", provide wrappers for the respective subsystems. They are typically not needed in normal workflows so are kept outside the object package.

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. The tree is not safe for concurrent use, and must be guarded by a Mutex or RWLock as appropriate - the exception is immutable trees returned by MutableTree.GetImmutable() which are safe for concurrent use as long as the version is not deleted via DeleteVersion(). Basic usage of MutableTree: Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. The tree is not safe for concurrent use, and must be guarded by a Mutex or RWLock as appropriate - the exception is immutable trees returned by MutableTree.GetImmutable() which are safe for concurrent use as long as the version is not deleted via DeleteVersion(). Basic usage of MutableTree: Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. The tree is not safe for concurrent use, and must be guarded by a Mutex or RWLock as appropriate - the exception is immutable trees returned by MutableTree.GetImmutable() which are safe for concurrent use as long as the version is not deleted via DeleteVersion(). Basic usage of MutableTree: Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package builder provides a method for writing fluent immutable builders.

Package gographt provides interfaces and data structures that describe discrete graphs. It is inspired by the excellent JGraphT library (http://jgrapht.org/) and makes use of some applicable parts of it. Note that there are licensing constraints on JGraphT that may apply to this package as well; you may assume some risk by using it. See https://github.com/jgrapht/jgrapht/wiki/Relicensing for more information. This package supports simple graphs, multigraphs, and pseudographs in both directed and undirected variants. Each of these variants may be weighted or unweighted. A unique feature of this package is support for deterministic iteration; that is, each graph can retain information about the order of vertices and edges added to it, and given an immutable copy, iterate in a consistent order (not necessarily that of insertion, though). Most algorithms implemented also support deterministic iteration. In general, public graph interfaces are immutable and the types returned by constructor methods are mutable. Algorithms that accept graphs will always use the immutable interfaces and clone the graph if needed to perform computations.

Package lfmap provides generic concurrent lock-free map implementation, using immutable map and atomic swaps.

Package occ provides generic optimistic concurrency wrapper for values which can be represented by a pointer and accessed or replaced with a new copy concurrently. It is useful for implementation of lock-free concurrency on top of immutable data structures.

Package datastore provides a client for Google Cloud Datastore. See https://godoc.org/cloud.google.com/go for authentication, timeouts, connection pooling and similar aspects of this package. Entities are the unit of storage and are associated with a key. A key consists of an optional parent key, a string application ID, a string kind (also known as an entity type), and either a StringID or an IntID. A StringID is also known as an entity name or key name. It is valid to create a key with a zero StringID and a zero IntID; this is called an incomplete key, and does not refer to any saved entity. Putting an entity into the datastore under an incomplete key will cause a unique key to be generated for that entity, with a non-zero IntID. An entity's contents are a mapping from case-sensitive field names to values. Valid value types are: Slices of structs are valid, as are structs that contain slices. The Get and Put functions load and save an entity's contents. An entity's contents are typically represented by a struct pointer. Example code: GetMulti, PutMulti and DeleteMulti are batch versions of the Get, Put and Delete functions. They take a []*Key instead of a *Key, and may return a datastore.MultiError when encountering partial failure. Mutate generalizes PutMulti and DeleteMulti to a sequence of any Datastore mutations. It takes a series of mutations created with NewInsert, NewUpdate, NewUpsert and NewDelete and applies them atomically. An entity's contents can be represented by a variety of types. These are typically struct pointers, but can also be any type that implements the PropertyLoadSaver interface. If using a struct pointer, you do not have to explicitly implement the PropertyLoadSaver interface; the datastore will automatically convert via reflection. If a struct pointer does implement PropertyLoadSaver then those methods will be used in preference to the default behavior for struct pointers. Struct pointers are more strongly typed and are easier to use; PropertyLoadSavers are more flexible. The actual types passed do not have to match between Get and Put calls or even across different calls to datastore. It is valid to put a *PropertyList and get that same entity as a *myStruct, or put a *myStruct0 and get a *myStruct1. Conceptually, any entity is saved as a sequence of properties, and is loaded into the destination value on a property-by-property basis. When loading into a struct pointer, an entity that cannot be completely represented (such as a missing field) will result in an ErrFieldMismatch error but it is up to the caller whether this error is fatal, recoverable or ignorable. By default, for struct pointers, all properties are potentially indexed, and the property name is the same as the field name (and hence must start with an upper case letter). Fields may have a `datastore:"name,options"` tag. The tag name is the property name, which must be one or more valid Go identifiers joined by ".", but may start with a lower case letter. An empty tag name means to just use the field name. A "-" tag name means that the datastore will ignore that field. The only valid options are "omitempty", "noindex" and "flatten". If the options include "omitempty" and the value of the field is a zero value, then the field will be omitted on Save. Zero values are best defined in the golang spec (https://golang.org/ref/spec#The_zero_value). Struct field values will never be empty, except for nil pointers. If options include "noindex" then the field will not be indexed. All fields are indexed by default. Strings or byte slices longer than 1500 bytes cannot be indexed; fields used to store long strings and byte slices must be tagged with "noindex" or they will cause Put operations to fail. For a nested struct field, the options may also include "flatten". This indicates that the immediate fields and any nested substruct fields of the nested struct should be flattened. See below for examples. To use multiple options together, separate them by a comma. The order does not matter. If the options is "" then the comma may be omitted. Example code: A field of slice type corresponds to a Datastore array property, except for []byte, which corresponds to a Datastore blob. Zero-length slice fields are not saved. Slice fields of length 1 or greater are saved as Datastore arrays. When a zero-length Datastore array is loaded into a slice field, the slice field remains unchanged. If a non-array value is loaded into a slice field, the result will be a slice with one element, containing the value. Loading a Datastore Null into a basic type (int, float, etc.) results in a zero value. Loading a Null into a slice of basic type results in a slice of size 1 containing the zero value. Loading a Null into a pointer field results in nil. Loading a Null into a field of struct type is an error. A struct field can be a pointer to a signed integer, floating-point number, string or bool. Putting a non-nil pointer will store its dereferenced value. Putting a nil pointer will store a Datastore Null property, unless the field is marked omitempty, in which case no property will be stored. Loading a Null into a pointer field sets the pointer to nil. Loading any other value allocates new storage with the value, and sets the field to point to it. If the struct contains a *datastore.Key field tagged with the name "__key__", its value will be ignored on Put. When reading the Entity back into the Go struct, the field will be populated with the *datastore.Key value used to query for the Entity. Example code: If the struct pointed to contains other structs, then the nested or embedded structs are themselves saved as Entity values. For example, given these definitions: then an Outer would have one property, Inner, encoded as an Entity value. If an outer struct is tagged "noindex" then all of its implicit flattened fields are effectively "noindex". If the Inner struct contains a *Key field with the name "__key__", like so: then the value of K will be used as the Key for Inner, represented as an Entity value in datastore. If any nested struct fields should be flattened, instead of encoded as Entity values, the nested struct field should be tagged with the "flatten" option. For example, given the following: an Outer's properties would be equivalent to those of: Note that the "flatten" option cannot be used for Entity value fields. The server will reject any dotted field names for an Entity value. An entity's contents can also be represented by any type that implements the PropertyLoadSaver interface. This type may be a struct pointer, but it does not have to be. The datastore package will call Load when getting the entity's contents, and Save when putting the entity's contents. Possible uses include deriving non-stored fields, verifying fields, or indexing a field only if its value is positive. Example code: The *PropertyList type implements PropertyLoadSaver, and can therefore hold an arbitrary entity's contents. If a type implements the PropertyLoadSaver interface, it may also want to implement the KeyLoader interface. The KeyLoader interface exists to allow implementations of PropertyLoadSaver to also load an Entity's Key into the Go type. This type may be a struct pointer, but it does not have to be. The datastore package will call LoadKey when getting the entity's contents, after calling Load. Example code: To load a Key into a struct which does not implement the PropertyLoadSaver interface, see the "Key Field" section above. Queries retrieve entities based on their properties or key's ancestry. Running a query yields an iterator of results: either keys or (key, entity) pairs. Queries are re-usable and it is safe to call Query.Run from concurrent goroutines. Iterators are not safe for concurrent use. Queries are immutable, and are either created by calling NewQuery, or derived from an existing query by calling a method like Filter or Order that returns a new query value. A query is typically constructed by calling NewQuery followed by a chain of zero or more such methods. These methods are: Example code: Client.RunInTransaction runs a function in a transaction. Example code: Pass the ReadOnly option to RunInTransaction if your transaction is used only for Get, GetMulti or queries. Read-only transactions are more efficient. This package supports the Cloud Datastore emulator, which is useful for testing and development. Environment variables are used to indicate that datastore traffic should be directed to the emulator instead of the production Datastore service. To install and set up the emulator and its environment variables, see the documentation at https://cloud.google.com/datastore/docs/tools/datastore-emulator.

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. The tree is not safe for concurrent use, and must be guarded by a Mutex or RWLock as appropriate - the exception is immutable trees returned by MutableTree.GetImmutable() which are safe for concurrent use as long as the version is not deleted via DeleteVersion(). Basic usage of MutableTree: Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package pcre provides access to the Perl Compatible Regular Expresion library, PCRE. It implements two main types, Regexp and Matcher. Regexp objects store a compiled regular expression. They consist of two immutable parts: pcre and pcre_extra. Compile()/MustCompile() initialize pcre. Calling Study() on a compiled Regexp initializes pcre_extra. Compilation of regular expressions using Compile or MustCompile is slightly expensive, so these objects should be kept and reused, instead of compiling them from scratch for each matching attempt. CompileJIT and MustCompileJIT are way more expensive, because they run Study() after compiling a Regexp, but they tend to give much better perfomance: http://sljit.sourceforge.net/regex_perf.html Matcher objects keeps the results of a match against a []byte or string subject. The Group and GroupString functions provide access to capture groups; both versions work no matter if the subject was a []byte or string, but the version with the matching type is slightly more efficient. Matcher objects contain some temporary space and refer the original subject. They are mutable and can be reused (using Match, MatchString, Reset or ResetString). For details on the regular expression language implemented by this package and the flags defined below, see the PCRE documentation. http://www.pcre.org/pcre.txt

Package datastore provides a client for Google Cloud Datastore. See https://godoc.org/cloud.google.com/go for authentication, timeouts, connection pooling and similar aspects of this package. Entities are the unit of storage and are associated with a key. A key consists of an optional parent key, a string application ID, a string kind (also known as an entity type), and either a StringID or an IntID. A StringID is also known as an entity name or key name. It is valid to create a key with a zero StringID and a zero IntID; this is called an incomplete key, and does not refer to any saved entity. Putting an entity into the datastore under an incomplete key will cause a unique key to be generated for that entity, with a non-zero IntID. An entity's contents are a mapping from case-sensitive field names to values. Valid value types are: Slices of structs are valid, as are structs that contain slices. The Get and Put functions load and save an entity's contents. An entity's contents are typically represented by a struct pointer. Example code: GetMulti, PutMulti and DeleteMulti are batch versions of the Get, Put and Delete functions. They take a []*Key instead of a *Key, and may return a datastore.MultiError when encountering partial failure. Mutate generalizes PutMulti and DeleteMulti to a sequence of any Datastore mutations. It takes a series of mutations created with NewInsert, NewUpdate, NewUpsert and NewDelete and applies them atomically. An entity's contents can be represented by a variety of types. These are typically struct pointers, but can also be any type that implements the PropertyLoadSaver interface. If using a struct pointer, you do not have to explicitly implement the PropertyLoadSaver interface; the datastore will automatically convert via reflection. If a struct pointer does implement PropertyLoadSaver then those methods will be used in preference to the default behavior for struct pointers. Struct pointers are more strongly typed and are easier to use; PropertyLoadSavers are more flexible. The actual types passed do not have to match between Get and Put calls or even across different calls to datastore. It is valid to put a *PropertyList and get that same entity as a *myStruct, or put a *myStruct0 and get a *myStruct1. Conceptually, any entity is saved as a sequence of properties, and is loaded into the destination value on a property-by-property basis. When loading into a struct pointer, an entity that cannot be completely represented (such as a missing field) will result in an ErrFieldMismatch error but it is up to the caller whether this error is fatal, recoverable or ignorable. By default, for struct pointers, all properties are potentially indexed, and the property name is the same as the field name (and hence must start with an upper case letter). Fields may have a `datastore:"name,options"` tag. The tag name is the property name, which must be one or more valid Go identifiers joined by ".", but may start with a lower case letter. An empty tag name means to just use the field name. A "-" tag name means that the datastore will ignore that field. The only valid options are "omitempty", "noindex" and "flatten". If the options include "omitempty" and the value of the field is a zero value, then the field will be omitted on Save. Zero values are best defined in the golang spec (https://golang.org/ref/spec#The_zero_value). Struct field values will never be empty, except for nil pointers. If options include "noindex" then the field will not be indexed. All fields are indexed by default. Strings or byte slices longer than 1500 bytes cannot be indexed; fields used to store long strings and byte slices must be tagged with "noindex" or they will cause Put operations to fail. For a nested struct field, the options may also include "flatten". This indicates that the immediate fields and any nested substruct fields of the nested struct should be flattened. See below for examples. To use multiple options together, separate them by a comma. The order does not matter. If the options is "" then the comma may be omitted. Example code: A field of slice type corresponds to a Datastore array property, except for []byte, which corresponds to a Datastore blob. Zero-length slice fields are not saved. Slice fields of length 1 or greater are saved as Datastore arrays. When a zero-length Datastore array is loaded into a slice field, the slice field remains unchanged. If a non-array value is loaded into a slice field, the result will be a slice with one element, containing the value. Loading a Datastore Null into a basic type (int, float, etc.) results in a zero value. Loading a Null into a slice of basic type results in a slice of size 1 containing the zero value. Loading a Null into a pointer field results in nil. Loading a Null into a field of struct type is an error. A struct field can be a pointer to a signed integer, floating-point number, string or bool. Putting a non-nil pointer will store its dereferenced value. Putting a nil pointer will store a Datastore Null property, unless the field is marked omitempty, in which case no property will be stored. Loading a Null into a pointer field sets the pointer to nil. Loading any other value allocates new storage with the value, and sets the field to point to it. If the struct contains a *datastore.Key field tagged with the name "__key__", its value will be ignored on Put. When reading the Entity back into the Go struct, the field will be populated with the *datastore.Key value used to query for the Entity. Example code: If the struct pointed to contains other structs, then the nested or embedded structs are themselves saved as Entity values. For example, given these definitions: then an Outer would have one property, Inner, encoded as an Entity value. If an outer struct is tagged "noindex" then all of its implicit flattened fields are effectively "noindex". If the Inner struct contains a *Key field with the name "__key__", like so: then the value of K will be used as the Key for Inner, represented as an Entity value in datastore. If any nested struct fields should be flattened, instead of encoded as Entity values, the nested struct field should be tagged with the "flatten" option. For example, given the following: an Outer's properties would be equivalent to those of: Note that the "flatten" option cannot be used for Entity value fields. The server will reject any dotted field names for an Entity value. An entity's contents can also be represented by any type that implements the PropertyLoadSaver interface. This type may be a struct pointer, but it does not have to be. The datastore package will call Load when getting the entity's contents, and Save when putting the entity's contents. Possible uses include deriving non-stored fields, verifying fields, or indexing a field only if its value is positive. Example code: The *PropertyList type implements PropertyLoadSaver, and can therefore hold an arbitrary entity's contents. If a type implements the PropertyLoadSaver interface, it may also want to implement the KeyLoader interface. The KeyLoader interface exists to allow implementations of PropertyLoadSaver to also load an Entity's Key into the Go type. This type may be a struct pointer, but it does not have to be. The datastore package will call LoadKey when getting the entity's contents, after calling Load. Example code: To load a Key into a struct which does not implement the PropertyLoadSaver interface, see the "Key Field" section above. Queries retrieve entities based on their properties or key's ancestry. Running a query yields an iterator of results: either keys or (key, entity) pairs. Queries are re-usable and it is safe to call Query.Run from concurrent goroutines. Iterators are not safe for concurrent use. Queries are immutable, and are either created by calling NewQuery, or derived from an existing query by calling a method like Filter or Order that returns a new query value. A query is typically constructed by calling NewQuery followed by a chain of zero or more such methods. These methods are: Example code: Client.RunInTransaction runs a function in a transaction. Example code: Pass the ReadOnly option to RunInTransaction if your transaction is used only for Get, GetMulti or queries. Read-only transactions are more efficient. This package supports the Cloud Datastore emulator, which is useful for testing and development. Environment variables are used to indicate that datastore traffic should be directed to the emulator instead of the production Datastore service. To install and set up the emulator and its environment variables, see the documentation at https://cloud.google.com/datastore/docs/tools/datastore-emulator.

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. Basic usage of MutableTree. Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package mast provides an immutable, versioned, diffable map implementation of a Merkle Search Tree (MST). Masts can be huge (not limited to memory). Masts can be stored in anything, like a filesystem, KV store, or blob store. Masts are designed to be safely concurrently accessed by multiple threads/hosts. And like Merkle DAGs, Masts are designed to be easy to cache and synchronize. - Efficient storage of multiple versions of materialized views - Diffing of versions integrates CDC/streaming - Efficient copy-on-write alternative to Go builtin map Mast is an implementation of the structure described in the awesome paper, "Merkle Search Trees: Efficient State-Based CRDTs in Open Networks", by Alex Auvolat and François Taïani, 2019 (https://hal.inria.fr/hal-02303490/document). MSTs are similar to persistent B-Trees, except node splits are chosen deterministically based on key and tree height, rather than node size, so MSTs eliminate the dependence on insertion/deletion order that B-Trees have for equivalence, making MSTs an interesting choice for conflict-free replicated data types (CRDTs). (MSTs are also simpler to implement than B-Trees, needing less complicated rebalancing, and no rotations.) MSTs are like other Merkle structures in that two instances can easily be compared to confirm equality or find differences, since equal node hashes indicate equal contents. A Mast can be Clone()d for sharing between threads. Cloning creates a new version that can evolve independently, yet shares all the unmodified subtrees with its parent, and as such are relatively cheap to create. The immutable data types in Clojure, Haskell, ML and other functional languages really do make it easier to "reason about" systems; easier to test, provide a foundation to build more quickly on. https://github.com/bodil/im-rs, "Blazing fast immutable collection datatypes for Rust", by Bodil Stokke, is an exemplar: the diff algorithm and use of property testing, are instructive, and Chunks and PoolRefs fill gaps in understanding of Rust's ownership model for library writers coming from GC'd languages.

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. Basic usage of MutableTree. Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have:

Package iavl implements a versioned, snapshottable (immutable) AVL+ tree for persisting key-value pairs. Basic usage of MutableTree. Proof of existence: Proof of absence: Now we delete an old version: Can't create a proof of absence for a version we no longer have: