gorgonia.org/gorgonia
Package gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. Do differentiation with them just as easily. Autodiff showcases automatic differentiation Basic example of representing mathematical equations as graphs. In this example, we want to represent the following equation Gorgonia provides an API that is fairly idiomatic - most of the functions in in the API return (T, error). This is useful for many cases, such as an interactive shell for deep learning. However, it must also be acknowledged that this makes composing functions together a bit cumbersome. To that end, Gorgonia provides two alternative methods. First, the `Lift` based functions; Second the `Must` function Linear Regression Example The formula for a straight line is We want to find an `m` and a `c` that fits the equation well. We'll do it in both float32 and float64 to showcase the extensibility of Gorgonia This example showcases the reasons for the more confusing functions. This example showcases dealing with errors. This is part 2 of the raison d'être of the more complicated functions - dealing with errors SymbolicDiff showcases symbolic differentiation
Readme
Gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. If this sounds like Theano or TensorFlow, it's because the idea is quite similar. Specifically, the library is pretty low-level, like Theano, but has higher goals like Tensorflow.
Gorgonia:
The primary goal for Gorgonia is to be a highly performant machine learning/graph computation-based library that can scale across multiple machines. It should bring the appeal of Go (simple compilation and deployment process) to the ML world. It's a long way from there currently, however, the baby steps are already there.
The secondary goal for Gorgonia is to provide a platform for the exploration of non-standard deep-learning and neural network-related things. This includes things like neo-hebbian learning, corner-cutting algorithms, evolutionary algorithms, and the like.
The main reason to use Gorgonia is developer comfort. If you're using a Go stack extensively, now you have access to the ability to create production-ready machine learning systems in an environment that you are already familiar with and comfortable with.
ML/AI at large is usually split into two stages: the experimental stage where one builds various models, tests, and retests; and the deployed state where a model after being tested and played with, is deployed. This necessitates different roles like data scientist and data engineer.
Typically the two phases have different tools: Python (PyTorch, etc) is commonly used for the experimental stage, and then the model is rewritten in some more performant language like C++ (using dlib, mlpack etc). Of course, nowadays the gap is closing and people frequently share the tools between them. Tensorflow is one such tool that bridges the gap.
Gorgonia aims to do the same but for the Go environment. Gorgonia is currently fairly performant - its speeds are comparable to PyTorch's and Tensorflow's CPU implementations. GPU implementations are a bit finicky to compare due to the heavy CGO tax, but rest assured that this is an area of active improvement.
The package is go-gettable: go get -u gorgonia.org/gorgonia.
Gorgonia is compatible with Go modules.
Up-to-date documentation, references, and tutorials are present on the official Gorgonia website at https://gorgonia.org.
Gorgonia's project has a Slack channel on gopherslack, as well as a Twitter account. Official updates and announcements will be posted to those two sites.
Gorgonia works by creating a computation graph and then executing it. Think of it as a programming language, but is limited to mathematical functions, and has no branching capability (no if/then or loops). In fact, this is the dominant paradigm that the user should be used to thinking about. The computation graph is an AST.
Microsoft's CNTK, with its BrainScript, is perhaps the best at exemplifying the idea that building a computation graph and running the computation graphs are different things and that the user should be in different modes of thought when going about them.
Whilst Gorgonia's implementation doesn't enforce the separation of thought as far as CNTK's BrainScript does, the syntax does help a little bit.
Here's an example - say you want to define a math expression z = x + y. Here's how you'd do it:
package gorgonia_test
import (
"fmt"
"log"
. "gorgonia.org/gorgonia"
)
// Basic example of representing mathematical equations as graphs.
//
// In this example, we want to represent the following equation
// z = x + y
func Example_basic() {
g := NewGraph()
var x, y, z *Node
var err error
// define the expression
x = NewScalar(g, Float64, WithName("x"))
y = NewScalar(g, Float64, WithName("y"))
if z, err = Add(x, y); err != nil {
log.Fatal(err)
}
// create a VM to run the program on
machine := NewTapeMachine(g)
defer machine.Close()
// set initial values then run
Let(x, 2.0)
Let(y, 2.5)
if err = machine.RunAll(); err != nil {
log.Fatal(err)
}
fmt.Printf("%v", z.Value())
// Output: 4.5
}
You might note that it's a little more verbose than other packages of similar nature. For example, instead of compiling to a callable function, Gorgonia specifically compiles into a *program which requires a *TapeMachine to run. It also requires manual a Let(...) call.
The author would like to contend that this is a Good Thing - to shift one's thinking to machine-based thinking. It helps a lot in figuring out where things might go wrong.
Additionally, there is no support for branching - that is to say, there are no conditionals (if/else) or loops. The aim is not to build a Turing-complete computer.
More examples are present in the example subfolder of the project, and step-by-step tutorials are present on the main website
Gorgonia comes with CUDA support out of the box. Please see the reference documentation about how cuda works on the Gorgonia.org website, or jump to the tutorial.
We use semver 2.0.0 for our versioning. Before 1.0, Gorgonia's APIs are expected to change quite a bit. API is defined by the exported functions, variables, and methods. For the developers' sanity, there are minor differences to SemVer that we will apply before version 1.0. They are enumerated below:
Gorgonia's API is as of right now, not considered stable. It will be stable from version 1.0 forward.
Gorgonia supports 2 versions below the Master branch of Go. This means Gorgonia will support the current released version of Go, and up to 4 previous versions - providing something doesn't break. Where possible a shim will be provided (for things like new sort APIs or math/bits which came out in Go 1.9).
The current version of Go is 1.13.1. The earliest version Gorgonia supports is Go 1.11.x but Gonum supports only 1.12+. Therefore, the minimum Go version to run the master branch is Go > 1.12.
Gorgonia runs on :
If you have tested Gorgonia on other platforms, please update this list.
Gorgonia uses some pure assembler instructions to accelerate some mathematical operations. Unfortunately, only amd64 is supported.
Obviously, since you are most probably reading this on Github, Github will form the major part of the workflow for contributing to this package.
See also: CONTRIBUTING.md
All contributions are welcome. However, there is a new class of contributors, called Significant Contributors.
A Significant Contributor has shown a deep understanding of how the library works and/or its environs. Here are examples of what constitutes a Significant Contribution:
The significant Contributors list will be updated once a month (if anyone even uses Gorgonia that is).
The best way of support right now is to open a ticket on Github.
runtime.GC() calls in the tests?The answer to this is simple - the design of the package uses CUDA in a particular way: specifically, a CUDA device and context are tied to a VM, instead of at the package level. This means for every VM created, a different CUDA context is created per device per VM. This way all the operations will play nicely with other applications that may be using CUDA (this needs to be stress-tested, however).
The CUDA contexts are only destroyed when the VM gets garbage collected (with the help of a finalizer function). In the tests, about 100 VMs get created, and garbage collection for the most part can be considered random. This leads to cases where the GPU runs out of memory as there are too many contexts being used.
Therefore at the end of any tests that may use GPU, a runtime.GC() call is made to force garbage collection, freeing GPU memories.
In production, one is unlikely to start that many VMs, therefore it's not a problem. If there is, open a ticket on GitHub, and we'll look into adding a Finish() method for the VMs.
Gorgonia is licensed under a variant of Apache 2.0. It's the same as the Apache 2.0 Licence, except not being able to commercially profit directly from the package unless you're a Significant Contributor (for example, providing commercial support for the package). It's perfectly fine to profit directly from a derivative of Gorgonia (for example, if you use Gorgonia as a library in your product)
Everyone is still allowed to use Gorgonia for commercial purposes (for example: using it in software for your business).
There are very few dependencies that Gorgonia uses - and they're all pretty stable, so as of now there isn't a need for vendoring tools. These are the list of external packages that Gorgonia calls, ranked in order of reliance that this package has (sub-packages are omitted):
| Package | Used For | Vitality | Notes | Licence |
|---|---|---|---|---|
| gonum/graph | Sorting *ExprGraph | Vital. Removal means Gorgonia will not work | Development of Gorgonia is committed to keeping up with the most updated version | gonum license (MIT/BSD-like) |
| gonum/blas | Tensor subpackage linear algebra operations | Vital. Removal means Gorgonial will not work | Development of Gorgonia is committed to keeping up with the most updated version | gonum license (MIT/BSD-like) |
| cu | CUDA drivers | Needed for CUDA operations | Same maintainer as Gorgonia | MIT/BSD-like |
| math32 | float32 operations | Can be replaced by float32(math.XXX(float64(x))) | Same maintainer as Gorgonia, same API as the built-in math package | MIT/BSD-like |
| hm | Type system for Gorgonia | Gorgonia's graphs are pretty tightly coupled with the type system | Same maintainer as Gorgonia | MIT/BSD-like |
| vecf64 | optimized []float64 operations | Can be generated in the tensor/genlib package. However, plenty of optimizations have been made/will be made | Same maintainer as Gorgonia | MIT/BSD-like |
| vecf32 | optimized []float32 operations | Can be generated in the tensor/genlib package. However, plenty of optimizations have been made/will be made | Same maintainer as Gorgonia | MIT/BSD-like |
| set | Various set operations | Can be easily replaced | Stable API for the past 1 year | set licence (MIT/BSD-like) |
| gographviz | Used for printing graphs | Graph printing is only vital to debugging. Gorgonia can survive without, but with a major (but arguably nonvital) feature loss | Last update 12th April 2017 | gographviz license (Apache 2.0) |
| rng | Used to implement helper functions to generate initial weights | Can be replaced fairly easily. Gorgonia can do without the convenience functions too | rng license (Apache 2.0) | |
| errors | Error wrapping | Gorgonia won't die without it. In fact Gorgonia has also used goerrors/errors in the past. | Stable API for the past 6 months | errors licence (MIT/BSD-like) |
| gonum/mat | Compatibility between Tensor and Gonum's Matrix | Development of Gorgonia is committed to keeping up with the most updated version | gonum license (MIT/BSD-like) | |
| testify/assert | Testing | Can do without but will be a massive pain in the ass to test | testify license (MIT/BSD-like) |
These are the packages and libraries that inspired and were adapted from in the process of writing Gorgonia (the Go packages that were used were already declared above):
| Source | How it's Used | Licence |
|---|---|---|
| Numpy | Inspired large portions. Directly adapted algorithms for a few methods (explicitly labeled in the docs) | MIT/BSD-like. Numpy Licence |
| Theano | Inspired large portions. (Unsure: number of directly adapted algorithms) | MIT/BSD-like Theano's license |
| Caffe | im2col and col2im directly taken from Caffe. Convolution algorithms inspired by the original Caffee methods | Caffe Licence |
FAQs
Package gorgonia is a library that helps facilitate machine learning in Go. Write and evaluate mathematical equations involving multidimensional arrays easily. Do differentiation with them just as easily. Autodiff showcases automatic differentiation Basic example of representing mathematical equations as graphs. In this example, we want to represent the following equation Gorgonia provides an API that is fairly idiomatic - most of the functions in in the API return (T, error). This is useful for many cases, such as an interactive shell for deep learning. However, it must also be acknowledged that this makes composing functions together a bit cumbersome. To that end, Gorgonia provides two alternative methods. First, the `Lift` based functions; Second the `Must` function Linear Regression Example The formula for a straight line is We want to find an `m` and a `c` that fits the equation well. We'll do it in both float32 and float64 to showcase the extensibility of Gorgonia This example showcases the reasons for the more confusing functions. This example showcases dealing with errors. This is part 2 of the raison d'être of the more complicated functions - dealing with errors SymbolicDiff showcases symbolic differentiation
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Cyber extortion in the US and Canada hit record levels in 2023, with ransomware attacks surging and median ransom demands skyrocketing, though fewer companies are choosing to pay ransoms.
Security News
Ecma TC39 is meeting this week and has moved key ECMAScript proposals forward, advancing Deferred Import Evaluation, Error.isError(), RegExp Escaping, and Promise.try to the next stages.
Security News
Researchers have demonstrated that teams of LLM agents can exploit zero-day vulnerabilities with a 53% success rate, and the costs of using AI to do so are rapidly becoming more affordable than hiring a human penetration tester.