🚀 Big News: Socket Acquires Coana to Bring Reachability Analysis to Every Appsec Team.Learn more
Socket
Book a DemoInstallSign in
Socket

graphcast

Package Overview
Dependencies
Maintainers
1
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

graphcast

A framework for transforming tabular (CSV, SQL) and hierarchical data (JSON, XML) into property graphs and ingesting them into graph databases (ArangoDB, Neo4j)

0.14.2
PyPI
Maintainers
1

GraphCast suthing logo

A framework for transforming tabular (CSV, SQL) and hierarchical data (JSON, XML) into property graphs and ingesting them into graph databases (ArangoDB, Neo4j).

Python License: BSL pre-commit PyPI version DOI

Core Concepts

Property Graphs

GraphCast works with property graphs, which consist of:

  • Vertices: Nodes with properties and optional unique identifiers
  • Edges: Relationships between vertices with their own properties
  • Properties: Both vertices and edges may have properties

Schema

The Schema defines how your data should be transformed into a graph and contains:

  • Vertex Definitions: Specify vertex types, their properties, and unique identifiers
  • Edge Definitions: Define relationships between vertices and their properties
  • Resource Mapping: describe how data sources map to vertices and edges
  • Transforms: Modify data during the casting process

Resources

Resources are your data sources that can be:

  • Table-like: CSV files, database tables
  • JSON-like: JSON files, nested data structures

Features

  • Graph Transformation Meta-language: A powerful declarative language to describe how your data becomes a property graph:
    • Define vertex and edge structures
    • Set compound indexes for vertices and edges
    • Use blank vertices for complex relationships
    • Specify edge constraints and properties
    • Apply advanced filtering and transformations
  • Parallel processing: Use as many cores as you have
  • Database support: Ingest into ArangoDB and Neo4j using the same API (database agnostic)

Documentation

Full documentation is available at: growgraph.github.io/graphcast

Installation

pip install graphcast

Usage Examples

Simple ingest

from suthing import ConfigFactory, FileHandle

from graphcast import Schema, Caster, Patterns


schema = Schema.from_dict(FileHandle.load("schema.yaml"))

conn_conf = ConfigFactory.create_config({
        "protocol": "http",
        "hostname": "localhost",
        "port": 8535,
        "username": "root",
        "password": "123",
        "database": "_system",
}
)

patterns = Patterns.from_dict(
    {
        "patterns": {
            "work": {"regex": "\Sjson$"},
        }
    }
)

schema.fetch_resource()

caster = Caster(
    schema,
)

caster.ingest_files(
    path="./data",
    conn_conf=conn_conf,
    patterns=patterns,
)

Development

To install requirements

git clone git@github.com:growgraph/graphcast.git && cd graphcast
uv sync --dev

Tests

Test databases

Spin up Arango from arango docker folder by

docker-compose --env-file .env up arango

and Neo4j from neo4j docker folder by

docker-compose --env-file .env up neo4j

To run unit tests

pytest test

Requirements

  • Python 3.11+
  • python-arango

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

FAQs

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts