github.com/samonzeweb/profilinggo
This documentation gives an overview of possibilities offered by go tooling to measure performances or collect runtime information. It is not a detailed tutorial about benchmarking, profiling or tracing.
This documentation could also act as a reminder.
In most cases, you can try by yourself by running provided example source code. It is easy to experiment and play with these tools, as a kind of live demo or for a workshop.
Main subjects are :
Profiling and tracing could apply to benchmarks.
The code is provided as a Go module. There is no need to set the GOPATH environment variable.
You need a Go version compatible with Go modules. Provided code uses Go 1.12.
Go tooling does not measure CPU usage, it measures elapsed time. It's important not to confuse both, because if the code isn't CPU bound, they are not correlated.
For example if you add time.Sleep(time.Millisecond) into the code it will change the output. The CPU isn't involved, but the result changes.
Benchmarking is done through the Go testing tools. It's rather simple and well documented.
The primary result of a benchmark is, per tested operation :
Each benchmark could also be a starting point for profiling or tracing operations.
See the code of fibonacci/fibonacci_test.go for a minimal example.
Run the tests :
go test ./fibonaccigo test ./fibonacci -bench .go test ./fibonacci -bench . -benchmemThe argument following -bench is a regular expression. All benchmark functions whose names match are executed. The . in the previous examples isn't the current directory but a pattern matching all tests. To run a specific benchmark, use the regexp : -bench Suite (means everything containing Suite).
Useful tip : see ResetTimer() to ignore test setup in measures, see also StopTimer() and StartTimer(): https://golang.org/pkg/testing/#B.ResetTimer
It's possible to compare benchmarks with an external tool :
go install golang.org/x/tools/cmd/benchcmp@latest
go test ./fibonacci -bench . -benchmem > old.txt
(do some changes in the code)
go test ./fibonacci -bench . -benchmem > new.txt
~/go/bin/benchcmp old.txt new.txt
Profiling data are sampled and aggregated ones, not detailed traces. CPU profile measures elapsed time, and memory profile measures heap allocations (the stack is ignored).
While CPU benchmarks show how long an operation take (global view), profiling show function's execution duration (detailed view). You get the same global/detailed view with memory consumption.
Get profiling data from the benchmarks:
-cpuprofile=cpu.out-benchmem -memprofile=mem.outAn example with both :
go test ./fibonacci \
-bench BenchmarkSuite \
-benchmem \
-cpuprofile=cpu.out \
-memprofile=mem.out
CPU and memory profiling data from benchmarks are always stored in two separate files and will be analysed separately.
There are two way to exploit profiling data with standard go tools :
go tool pprof cpu.outgo tool pprof -http=localhost:8080 cpu.outThe View menu :
Graph and flamegraph are rather similar, but there is a major difference : flamegraph shows sampled call stack with time/memory data (it's a tree), while graph could have multiple path converging to the same function (it's not a tree). The time/memory data associated to functions having multiple paths converging to is an aggregation. Both are useful, just choose the good one for your case.
Viewing the data in the browser requires Graphviz. It can be installed using many package manager, ou downloaded from its official site : https://graphviz.org.
Use pprof.StartCPUProfile(), pprof.StopCPUProfile() and pprof.WriteHeapProfile(). See the pprof package documentation for more information.
A simple example :
cd showfib
go build
./showfib 40 2>cpu.out
go tool pprof -http=localhost:8080 cpu.out
If showfib is too fast no data will be available. In this case increase the value until it take few seconds to run.
This part isn't the most useful, but GC traces could easily spot a too high GC pressure.
Using an environment variable with any program (or test) : GODEBUG=gctrace=1
Or using code :
"runtime/trace"
trace.Start(os.Stderr)
defer trace.Stop()
Example with webfib :
cd webfib
go build
GODEBUG=gctrace=1 ./webfib
(other terminal)
go install github.com/rakyll/hey@latest
~/go/bin/hey -n 10000 http://localhost:8000/?n=30
Traces are event collected during the program execution. They give a chronological view of program's execution with detailed information about heap, GC, goroutines, core usage, ...
Generate traces with a test, and visualize data :
go test ./fibonacci \
-bench BenchmarkSuite \
-trace=trace.out
go tool trace trace.out
There are many detailed information... the blog post Go execution tracer is a good quick tour of the trace tool GUI.
Tip : from the View trace part hit ? to show a help.
Like tracing with test, but you have to add code to collect traces into a file, and then use go tool trace.
"runtime/trace"
trace.Start(os.Stderr)
defer trace.Stop()
(no example, see trace package for more information)
Here Go tooling starts to really shine ! Go allows any program running a http server to be analysed during execution, even in production. And it's really easy.
Data are gathered only on demand, it doesn't use resources when it is not used.
Importing the net/http/pprof standard package add handler to DefaultServeMux. If there is already a web server using it, that's all.
import _ "net/http/pprof"
// Add this only if needed
go func() {
log.Println(http.ListenAndServe("localhost:8000", nil))
}()
The webfib exemple use it. Live data are available here : http://localhost:8000/debug/pprof/ , but is is it's not very user friendly.
Handlers provided by net/http/pprof should only be accessible to trusted client. It is not something you want to be available directly through internet, or internally by untrusted third party.
The package net/http/pprof register handlers to DefaultServeMux, use a separate server (create a dedicated Mux) for your http server. Each one using a different port, different security rules could be applied.
It's possible to profile program using net/http/pprof with go tool pprof.
We'll use 3 terminals to :
cd webfib
go build
./webfib
go tool pprof -http=localhost:8080 http://localhost:8000/
~/go/bin/hey -n 2000 -c 200 http://localhost:8000/?n=30
The go tool ... command line collect data, then open the browser. Be patient...
Trace data have to be collected manually, and feed into go tool trace.
We'll use 3 terminals to :
cd webfib
go build
./webfib
curl -o trace.out http://localhost:8000/debug/pprof/trace?seconds=15
~/go/bin/hey -n 2000 -c 200 http://localhost:8000/?n=30
Now analyse data with Chrome : go tool trace trace.out
User-defined traces were introduced with Go 1.11. It's not a new tool, but rather a way to add events to traces within your code.
What's in the box :
Task struct : to trace high-level operations.Region struct : to trace lower-level operations.Log function : to add log information into traces.The anowebfib (like another webfib) use all mentionned points. Each HTTP request are identified using a Task, each call to fibonacci package with Region, and n are logged.
Example, with 3 terminals :
cd anowebfib
go build
./anowebfib
curl -o trace.out http://localhost:8000/debug/pprof/trace?seconds=20
~/go/bin/hey -n 3000 -c 200 http://localhost:8000/unique?n=30
~/go/bin/hey -n 3000 -c 200 http://localhost:8000/multiple?n=30
As usual : go tool trace trace.out
See :
Packages :
Others :
FAQs
Unknown package
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.