github.com/supersonictw/elastic-butterfly

Butterfly

[Violet] + [Butterfly] = {StarStart!}

The World-Wide-Web crawler for Apache Solr

Installation

Get the execute file official build via GitHub Releases.

No install required, but you can put it on the directory your terminal $PATH variable points for convenient.

Eventually, set up your config file (default name: .butterfly_config.json), and move it to the directory where you want to execute butterfly command.

Usage

Warning

The config file must be existed under the directory you execute butterfly command.

Execute

./butterfly <URI>

The URI is a variable, that the website URL you want to crawl on start.

Developmet Environment

Requirement

Butterfly requires Go Language Compiler >= 1.13

Please check your go version or install the latest version.

• GNU Linux / MacOSX / Unix Like

  Execute this command

  sh bootstrap.sh

• Win32 Platform

  We have no plan to support Microsoft Windows, but you still can get the go packages Butterfly required.

Compilation

Just do it.

go build

The "butterfly" will be the execute file you build on your own self.

(c) 2020 Star Inc.