fluent-plugin-sqlquery-ssh

===========================

Fluentd Input plugin to execute mysql query and fetch rows across multiple dbs based on an initial shard query returning array of host/db to hit

Installation

install with gem or fluent-gem command as:

# for fluentd
$ fluentd-gem install fluent-plugin-sqlquery-ssh

# for td-agent
$ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-sqlquery-ssh

Configuration

Config Sample

<source>
  type sqlquery_ssh
  shard_map_query SELECT database_host, group_concat(database_name) AS database_name FROM cust_customer WHERE status = 'customer' AND collect_stats=1 GROUP BY database_host order by database_host DESC;
  query SELECT database() as customer from invoice
  host system-reporting.db.prd.us-east-silo.ls  #only use for reporting slaves
  ssh_gateway mgmt.us-east-silo.lightspeedapp.com
  skip_shards [""] #currently broken. use shard query to reduce set.
  ssh_username #user name with jumphost access
  db_username #reporting db username (do not use this on prod other than reporting)
  db_password #the db password for the user above.
  nest_result no #whether to nest the result set
  nest_key data #key to nest
  row_count no #add row count to the return
  row_count_key row_count
  tag debug #the matching tag
</source>

<match debug*>
  type stdout
</match>