github.com/tonistiigi/go-csvvalue

go-csvvalue

go-csvvalue provides an efficient parser for a single-line CSV value.

It is more efficient than the standard library encoding/csv package for parsing many small values. The main problem with stdlib implementation is that it calls bufio.NewReader internally, allocating 4KB of memory on each invocation. For multi-line CSV parsing, the standard library is still recommended. If you wish to optimize memory usage for encoding/csv, call csv.NewReader with an instance of *bufio.Reader that already has a 4KB buffer allocated and then reuse that buffer for all reads.

For further memory optimization, an existing string slice can be optionally passed to be reused for returning the parsed fields.

For backwards compatibility with stdlib record parser, the input may contain a trailing newline character.

Benchmark

goos: linux
goarch: amd64
pkg: github.com/tonistiigi/go-csvvalue
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkFields/stdlib/withcache-4         	 1109917	      1103 ns/op	    4520 B/op	      14 allocs/op
BenchmarkFields/stdlib/nocache-4           	 1082838	      1125 ns/op	    4520 B/op	      14 allocs/op
BenchmarkFields/csvvalue/withcache-4       	28554976	        42.12 ns/op	       0 B/op	       0 allocs/op
BenchmarkFields/csvvalue/nocache-4         	13666134	        83.77 ns/op	      48 B/op	       1 allocs/op

goos: darwin
goarch: arm64
pkg: github.com/tonistiigi/go-csvvalue
BenchmarkFields/stdlib/nocache-10                1679923               784.9 ns/op          4520 B/op         14 allocs/op
BenchmarkFields/stdlib/withcache-10              1641891               826.9 ns/op          4520 B/op         14 allocs/op
BenchmarkFields/csvvalue/withcache-10           34399642                33.93 ns/op            0 B/op          0 allocs/op
BenchmarkFields/csvvalue/nocache-10             17441373                67.21 ns/op           48 B/op          1 allocs/op
PASS

Credits

This package is mostly based on encoding/csv implementation and also uses that package for compatibility testing.