@keyvaluesystems/nestjs-dataloader

NestJS-DataLoader

This package helps to easily implement Dataloaders in a NestJS project that uses TypeORM. Instead of writing separate classes for each Dataloader use case, you can leverage the functionalities of this package to do the same in few lines of code!!

Installation

Using npm:

npm install @keyvaluesystems/nestjs-dataloader

For typeorm version below 3.0 (@nestjs/typeorm version below 9.0), please use the version 2.0.0 :

npm i @keyvaluesystems/nestjs-dataloader@2

Setup

New versions of TypeORM (3.0 and above) use DataSource instead of Connection, so most of the APIs have been changed and the old APIs have become deprecated.

To be able to use TypeORM entities in transactions, you must first add a DataSource using the addDataSource function:

import { DataSource } from 'typeorm';
import { addDataSource } from '@keyvaluesystems/nestjs-dataloader';
...
const dataSource = new DataSource({
	type: 'ConnectionType i.e mysql, postgres etc',
    host: 'HostName',
    port: PORT_No,
    username: 'DBUsername',
    password: 'DBPassword'
});
...

addDataSource(dataSource);

Example for Nest.js:

// database.module.ts

import { Module } from '@nestjs/common';
import { TypeOrmModule } from '@nestjs/typeorm';
import { DataSource } from 'typeorm';
import { addDataSource } from '@keyvaluesystems/nestjs-dataloader';

@Module({
	imports: [
	   TypeOrmModule.forRootAsync({
	     useFactory() {
	       return {
	         type: 'ConnectionType i.e mysql, postgres etc',
	         host: 'HostName',
	         port: PORT_No,
	         username: 'DBUsername',
	         password: 'DBPassword',
	         synchronize: false,
	         logging: false,
	       };
	     },
	     async dataSourceFactory(options) {
	       if (!options) {
	         throw new Error('Invalid options passed');
	       }

	       return addDataSource(new DataSource(options));
	     },
	   }),

	   ...
	 ],
	 providers: [...],
	 exports: [...],
})
class AppModule {}

dataSoureFactory is a part of TypeOrmModuleAsyncOptions and needs to be specified in the database.module.ts file to connect to the given package.

Example Scenario:

Consider the following entities:

@Entity()
class User {
  @PrimaryGeneratedColumn('uuid')
  public id!: string;

  @Column()
  public email!: string;

  @Column({ nullable: true })
  public password?: string;

  @Column()
  public name!: string;

  @OneToMany(() => Address,(address) => address.user)
  public addresses!: Address[];

  @ManyToOne(()=>Country,(country)=> country.users)
  public country: Country;

  @Column('uuid')
  public countryId: string;

  @OneToOne(() => Profile, (profile) => profile.user)
  profile?: Profile;

  @Column('uuid')
  profileId?: string;

  @ManyToMany(() => Product, (product) => product.favoriteUsers, {
    onDelete: 'NO ACTION',
    onUpdate: 'NO ACTION',
  })
  favoriteProducts?: Product[];
}

@Entity()
export class Profile{
  @PrimaryGeneratedColumn('uuid')
  id!: string;

  @Column()
  age!: number;

  @Column()
  gender!: Gender;

  @Column()
  phone!: string;

  @OneToOne(() => User, (user) => user.profile)
  user!: User;

}

@Entity()
class Country{
  @PrimaryGeneratedColumn('uuid')
  public id!: string;

  @Column()
  public name!: string;

  @OneToMany(()=>User,(user)=>user.country)
  public users?:User[];
}

@Entity()
class Address {
  @PrimaryGeneratedColumn('uuid')
  public id!: string;

  @Column({ nullable: true })
  public addressLine!: string;

  @Column({ nullable: true })
  public pincode?: string;

  @ManyToOne(() => User,(user) => user.addresses)
  public user!: User;

  @Column('uuid')
  public userId!: string;

  @CreateDateColumn()
  public createdAt!: Date;

  @Column({ default: false })
  public isPrimary!: boolean;
}

@Entity()
export class Product {
  @PrimaryGeneratedColumn()
  id: number;

  @Column()
  name: string;

  @ManyToMany(() => User, (user) => user.favoriteProducts, {
    onDelete: 'NO ACTION',
    onUpdate: 'NO ACTION',
    cascade: true,
  })
  @JoinTable({
    name: 'user_favorite_products_products',
    joinColumn: {
      name: 'user_id',
      referencedColumnName: 'id',
    },
    inverseJoinColumn: {
      name: 'product_id',
      referencedColumnName: 'id',
    },
  })
  favoriteUsers?: User[];
}

Lets consider the corresponding GraphQL types:

type User {
  id: ID
  name: String
  age: Int
  cart: Cart
  addresses: [Address]
  country: Country
  profile: Profile
}

type Country {
  id: ID
  name: String
}

type Address {
  id: ID
  addressLine: String
  pincode: String
}

type Profile {
  id: ID
  age: Int
  gender: String
  phone: String
}

Usage

This package provides a Parameter Decorator named InjectLoader which is intended to be used in the parameter list of a ResovleField.
InjectLoader can be used in either a simple form or a more flexible form.

Use Cases:

ManyToOne
User's Country :- You can easily resolve a user's country by

//user.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
...

@ResolveField()
async country(
  @Parent() user: User,
  @InjectLoader(Country)loader: DataLoader<string, Country>,
) {
  return await loader.load(user.countryId);
}

Use InjectLoader as a param decorator and pass the entity class to be resolved(Country in this case).
This will inject a dataloader of type DataLoader<string,Country> to the parameter loader.
You can then resovle the value(Country) by loading key user.countryId to the dataloader.

OneToMany
User's Addresses :-
You can easily resolve a user's addresses by

//user.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
...

@ResolveField()
async addresses(
  @Parent() user: User,
  @InjectLoader({
    fromEntity: Address,
    resolveType: 'many',
    fieldName: 'userId',
  })loader: DataLoader<string, Address[]>,
) {
  return await loader.load(user.id);
}

Here we pass an input object to the InjectLoader param-decorator, as described below:

  {
    fromEntity: Address,
    resolveType: 'many',
    fieldName: 'userId',
  }

fromEntity is the entity class to be resolved(Address in this case).
resolveType defines whethes each key should resolve to a single value or an array of values.( Here the user entity has multiple addresses so passing 'many').
fieldName is the name of the relation field.('userId' of the Address entity in this case).
This will inject a dataloader of type DataLoader<string,Address[]>.
The library will resovle the value(Address) by querying for the key in the field 'userId' of Address entity.

If you want to sort the Addresses you can give additional options as:

  {
    fromEntity: Address,
    resolveType: 'many',
    fieldName: 'userId',
    resolveInput:{
      order: {
        createdAt: 'ASC'
      }
    }
  }

resolveInput can take a FindManyOptions<Address> object. (FindManyOptions<T> is defined in TypeORM). This works in conjunction with the other inputs given.

If you want to list user along with the primary address only, your User graphql type will be like this:

type User {
  id: ID
  name: String
  age: Int
  cart: Cart
  addresses: [Address]
  primaryAddress: Address
  country: Country
}

And your corresponding resolve field for primaryAddress will be like this:

//user.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
...

@ResolveField()
async primaryAddress(
  @Parent() user: User,
  @InjectLoader({
    fromEntity: Address,
    fieldName: 'userId',
    resolveInput: {
      where: {
        isPrimary: true
      }
    }
  })loader: DataLoader<string, Address>,
) {
  return await loader.load(user.id);
}

This will return the first address of the user which have isPrimary set to true.

You can also implement this using custom finder function as follows:

//user.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
import { findPrimaryAddress } from 'address.service.ts';
...

@ResolveField()
async primaryAddress(
  @Parent() user: User,
  @InjectLoader({
    fromEntity: Address,
    fieldName: 'userId',
    resolveInput: findPrimaryAddress
  })loader: DataLoader<string, Address>,
) {
  return await loader.load(user.id);
}

The custom finder function will be as follows:

//address.service.ts
async findPrimaryAddress(userIds: string[]):Promise<Address[]> {
  return await this.addressRepository.find({
    where: {
      userId: In(userIds),
      isPrimary: true
    }
  })
}

You can use custom finder function if you want more control over how you fetch values from keys loaded to the dataloader.

OneToOne
User's Profile :-
You can easily resolve a user's profile by

//user.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
...

@ResolveField()
async profile(
  @Parent() user: User,
  @InjectLoader(Profile)loader: DataLoader<string, Profile>,
) {
  return await loader.load(user.profileId);
}

Use InjectLoader as a param decorator and pass the entity class to be resolved(Profile in this case).
This will inject a dataloader of type DataLoader<string,Profile> to the parameter loader.
You can then resovle the value(Profile) by loading key(user.profileId) to the dataloader.

ManyToMany
For ManyToMany relations, we recommend defining entity for the junction table explicitly and follow the ManyToOne use cases.

Get the favorite products of a user:-

//user.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
...

@ResolveField()
async favoriteProducts(
  @Parent() user: User,
  @InjectLoader({
    fromEntity: Product,
    resolveType: 'many-to-many',
    fieldName: 'favoriteProducts',
    // resolveInput is optional if it only contains a relations array with just the fieldName entry
    // resolveInput: {
    //   relations: ['favoriteProducts'],
    // },
  })
) {
  return await loader.load(user.id);
}

Similarly, get the list of users who marked the product as favorite:-

//product.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
...

@ResolveField()
async favoriteUsers(
  @Parent() product: Product,
  @InjectLoader({
    fromEntity: User,
    resolveType: 'many-to-many',
    fieldName: 'favoriteUsers',
  })
) {
  return await loader.load(product.id);
}

COUNT

When the resultType field is set to 'count', the dataloader retrieves the count of records. The count feature works for each of the resolveType values but does not support the use of the resolveInput field when the resultType is set to 'count'.

Given below is an example for the new count feature of the package.

//user.resolver.ts
import InjectLoader from '@keyvaluesystems/nestjs-dataloader';
import DataLoader from 'dataloader';
...
// resolve field to retrieve the count of addresses associated with the user.
@ResolveField()
async addressCount(
  @Parent() user: User,
  @InjectLoader({
    fromEntity: Address,
    resolveType: 'many',
    fieldName: 'userId',
  })loader: DataLoader<string, Record<string, number>>,
) {
  return await loader.load(user.id)[user.id];
}

Reference

Decorator Input fields:

{
  fromEntity: EntityTarget<Entity>; // Entity class from which values will be fetched.
  resolveType?: ResolveType; // 'one' | 'many' . default: 'one'
  fieldName?: string; //default: 'id'
  resolveInput?: ResolveInput<Entity>; // FindManyOptions<Entity> OR a custom finder function  used to fetch values from keys
  resultType?: ResultType; // 'entity' | 'count' . default: 'entity'
}

Description of the fields:

• fromEntity : An entity class which will be the value type of the dataloader. This input is mandatory if using object input.
• resolveType: (optional). Used to determine whether there should be one or many values for each key.
  Passing 'one' gives DataLoader<K,V>. Passing 'many' gives DataLoader<K,V[]>
• fieldName : (optional). Name of the field in the values's entity where the key will be searched. Lets say you want your dataloader to accept an id(string) as key and return an organisation as value (from Organisation entity). Lets say your Organisation entity stores id in a field named orgId, then you should pass fieldName: 'orgId'
• resolveInput: (optional). Can be used to pass additional find() options or a entirely separate custom finder function.
  • Passing additional find options: Lets say you want to join a relation Address along with Organisation. You can achieve that by passing resolveInput: { relations: [ 'Address' ] }.

    Note: Any conditions passed in the fieldName will be overwritten.

  • Passing a custom finder function: Suppose your requirements to fetch values from keys are beyond the capabilites provided so far, you can pass a custom finder function that will be used to fetch values from keys in the batchload funtion of the dataloader.
    Example: resolveInput: findOrgsByIds where findOrgsByIds is a function that accepts an array of keys and returns an array of values.

    Note: Error handling for the custom finder function will be implemented as an improvement.

  • resultType: (optional). Used to determine whether the loader should return the value or the count of the values.
    Passing 'entity' gives DataLoader<K,V> for resolveType as 'one' and gives DataLoader<K,V[]> for resolveType as 'many'. Passing 'count' gives DataLoader<K,V> where each key has the associated count with it.

Working Mechanism

The decorator adds a dataloader wrapper class(BaseDataLoader) instance in the execution context under the key BASE_DATALOADERS and injects the same to the parameter. All the dataloaders injected through this decorator will be under this key. In the BASE_DATALOADERS key, the instances will be stored under separate key, where name of the key will be same as that of the ResolveField function where it is used. The wrapper class instance has request-scope, i.e, it will be removed once the request is processed.
The wrapper class instance is added to the context only once in a request's lifecycle for each ResolveField function. As part of the resolving mechanism of GraphQL, when the ResolveField function is called multiple times, for all the calls after the first call the same wrapper class instance that is already added to the context will be injected to the parameter.