csv_parser
Csv parser can read from csv file and parse records to golang structs.
Csv parser won't read all records with a single method call, instead it reads from a channel, therefore you can limit the memory usage at a low level.
Getting start
Installation
go get github.com/why2go/csv_parser
Supported field types
- primitive types: bool, int, int8, int16, int32, int64, uint, uint8, uint16, uint32, uint64, float32, float64, string.
- slice: whose element's type is primitive type.
- map: whose key's type is string and value's type is primitive type.
Struct fields don't match the above type will be ignored.
Supported Tags
type Demo struct {
Name string `csv:"name,required,default=unknown" json:"name"`
Attri map[string]int16 `csv:"attri" json:"attri"`
Msg []string `csv:"msg" json:"msg"`
}
As shown above, you can specify tags for csv parser when define a struct.
Csv tag options are sepereated by comma. The first tag part will be treated as the header name, which will be used to match the csv file content. required
option means the csv file headers must contain this header name, otherwise an error will be returned. default
option can be used to assign a default value to the field when no value is provieded in the file. If the value text provided in the file can't be transformed to the struct field type, and you choose to ignore the parse error when you create the parser instance with WithIgnoreFieldParseError
option, default value will be assigned to the struct field as well.
Default option is not applicable for Slice
and Map
type.
Example
package main
import (
"bytes"
"context"
"encoding/csv"
"encoding/json"
"fmt"
"github.com/why2go/csv_parser"
)
func main() {
data := `name,{{attri:age}},{{attri:height}},[[msg]],[[msg]]
Alice,20,,"Hi, I'm Alice.",Nice to meet you!
Bob,21,175,"Hi, I'm Bob.",Nice to meet you!
Candy,,189,"Hi, I'm Candy.",Nice to meet you!
,23,172,"Hi, I'm David.",
`
type Demo struct {
Name string `csv:"name,required,default=unknown" json:"name"`
Attri map[string]int16 `csv:"attri" json:"attri"`
Msg []string `csv:"msg" json:"msg"`
}
r := csv.NewReader(bytes.NewBufferString(data))
parser, err := csv_parser.NewCsvParser[Demo](r)
if err != nil {
panic(err)
}
defer parser.Close()
for dataWrapper := range parser.DataChan(context.Background()) {
if err != nil {
panic(err)
}
b, err := json.Marshal(dataWrapper.Data)
if err != nil {
panic(err)
}
fmt.Printf("demo: %s\n", string(b))
}
fmt.Printf("done\n")
}