insights

Insights

As our education marketplace grew, we faced a dilemma: we had little visibility into our data.

Questions such as "who is our best customer?", "which target countries are gaining in sales?" or "what are our most popular categories for paid leads?" required tinkering in the Rails console or writing custom stats pages. We lost a lot of developer time over the years.

So we started looking for the best Business Intelligence solution out there.

Unfortunately, they all had their problems. Some required knowledge of SQL. Some made multi-table data exploration a pain. Most cost an arm, a leg and an iPhone. Per month.

The best tool we found was Looker, but at a monthly price equal to our burn rate, it wouldn't work.

So I decided to re-implement the essential parts of Looker as an open source alternative.

Insights is a desktop or self-hosted "SQL-not-required" data analytics and business intelligence tool. Featuring linkable URLs, easy data exploration, automatic joins, graphs, exports, facets (pivots), saveable views, pretty colors and a ridiculously permissive license (MIT).

It's a work in progress and you're brave for checking it out! Cheers!

Play with the demo here.

Installing

To install, make sure you have Node 7.6+ installed and then run:

npm install -g insights@latest

insights          # run the electron version
insights --server # run the server

How does it work

Similar to Looker and their LookML, insights lets you to define your data model in a file called insights.yml.

If this file is not found, we try to autodetect your database schema and connections.

You can use the insights_export gem to generate this file from your Rails Models. Adapters for other frameworks coming soon.

You keep this file with your code and update it whenever something changes. You edit it to add custom fields (e.g. full_name: first_name || ' ' || last_name), hide existing fields (e.g. encrypted_password) or hide entire models.

When your database changes, run rake insights:export and the file is updated automatically.

One entry in this file looks like this:

Order:
  enabled: true  # set to false to hide
  model: Order
  table_name: orders
  primary_key: id
  columns:
    id:
      type: number
      index: primary_key
    total_price:
      type: number
    hidden_field: false # this stays hidden
    currency:
      type: string
    # ...
  custom:
    total_price_in_eur:
      sql: "$$.total_price * $$.currency_to_eur"
      type: number
  links:
    order_lines:
      model: OrderLine
      model_key: order_id
      my_key: id
    user:
      model: User
      model_key: id
      my_key: user_id

You give this insights.yml file and a database connection to insights and start exploring.

The point is this: your developers update the .yml file. Your CEO and CFO browse the interface. If they ask for a report, you send them a link to the right view and they can explore further.

How to use

Play with the demo here and try to answer the following questions:

1. Which product has been bought the most? (solution)
2. Sales by country by month (solution) - export it as a PDF as well!
3. Sales by delivery status (solution)
4. Where are your users from (solution)
5. ... by month? (solution)
6. ... only ones with confirmed orders? (solution)

Hint: to count rows, select the id field and then count from the table header.

Coming soon

• Dashboards
• Better graph controls
• Graphs that don't require a time column
• View generated SQL
• Moderate React/Kea frontend code refactoring
• Polishing