A fully-decorated nest ready to fly
Tested, documented, and production-ready.
Nest Pret is a replicable NestJS project that solves much of the functionality required from a modern web app.
- User registration
- Password recovery
- E-mail verification, configurable between:
- required before login
- delayed until a route with
EmailVerifiedGuard
enforces it - or off
- Claims-based access control, including:
- Restricted access to routes via policies
- Restricted access to specific documents by ownership or other conditional constraints
- Serialization of response objects exposing only the fields the user has access to
- Standardized API responses, including:
- Automatic wrapping of return objects into a StandardResponse
- Metadata-based โ handlers remains returning Classes compatible with interceptors
- Handling of pagination, sorting and filtering
- Generation of OpenAPI documentation for routes with the proper combined response schema
- Secure defaults:
- Sets secure HTTP response headers
- Global validation of all request inputs
- Global validation of response values before serialization
- Rate-limiting across the app with tighter limits for account creation
- Configurable
- Config module parses and validates .env variables during bootstrap
- Config service makes them available app-wide with proper type definitions
- Deployable
- Docker compose environmets for dev and e2e testing
- Docker swarm stack ready for continous deployment
- Tested
- Complete end-to-end testing suites
- 100% coverage of all user interaction flows
Getting started
- Clone this repo
- Edit the
.env
file and add at least your mailer service information.
๐ณ ย Start using Docker
To start using Docker, run:
npm run dev
This will start a docker compose with:
- the dev database
- a mongo-express visual DB admin page at
localhost:8081
- the NestJS app in watch mode at
localhost:3000
Only for the development
env, the app will also serve documentation and a dependency graph:
- Swagger UI documentation explorer:
localhost:3000/dev-tools/docs
- Swagger documentation JSON:
localhost:3000/dev-tools/docs-json
- MermaidJS graph providing a high level view of the inter-module dependencies:
localhost:3000/dev-tools/graph
If running in Docker, you're not required to run npm install
locally, but you still might want to do so in order to get features such as auto-import and auto-complete in your code editor.
๐ง ย Start without docker
Make sure to edit .env
file to add the connection information for your mongo database, then run:
npm install
npm run start:dev
๐ง ย Running tests
To run unit and integration tests (locally):
npm run test
or
npm run test:watch
To run e2e tests:
npm run e2e
This command will lift a full docker compose configuration with the testing environment, run all the tests against it, print the output, then immediately exit and prune all docker resources created.
You can also run e2e tests in watch mode:
npm run e2e:watch
Note that running in this mode will keep the entire docker testing environment up. Once you're done with testing, you can take it down and clear everything by running:
npm run e2e:down
Tech stack
App Graph
%%{ init: { 'flowchart': { 'curve': 'monotoneX' }, 'theme':'dark' } }%%
flowchart LR
subgraph legend[ Legend ]
direction LR
subgraph legendLine1 [ ]
direction TB
ex1(Module)
ex2([Global Module]):::globalModule
ex3{{fa:fa-globe Controller}}:::controller
ex9([fa:fa-bell-concierge Service]):::service
ex4([fa:fa-briefcase Provider]):::provider
end
subgraph legendLine2 [ ]
direction TB
ex6{{fa:fa-fish-fins Global Pipe}}:::pipe
ex7{{fa:fa-bullseye Global Interceptor}}:::interceptor
ex8{{fa:fa-shield-halved Global Guard}}:::guard
ex5([fa:fa-database Model]):::model
end
end
subgraph globalModules[ ]
ConfigModule([ConfigModule]):::globalModule
JwtModule([JwtModule]):::globalModule
ConfigHostModule([ConfigHostModule]):::globalModule
MongooseCoreModule([MongooseCoreModule]):::globalModule
end
subgraph modules[" "]
direction LR
subgraph AppModule
direction LR
Pipe{{fa:fa-fish-fins ValidationPipe}}:::pipe
Serializer{{fa:fa-fish-fins RolesSerializerInterceptor}}:::interceptor
AppService([fa:fa-bell-concierge AppService]):::service
end
subgraph ConfigModule[ ]
subgraph ConfigModulePadding[ConfigModule]
end
end
subgraph ConfigHostModule[ ]
subgraph ConfigHostModulePadding[ConfigHostModule]
end
end
subgraph MongooseModule
direction LR
UserModel([fa:fa-database UserModel]):::model
EmailVerificationModel([fa:fa-database EmailVerificationModel]):::model
ForgottenPasswordModel([fa:fa-database ForgottenPasswordModel]):::model
end
subgraph MongooseCoreModule[ ]
subgraph MongooseCoreModulePadding[MongooseCoreModule]
end
end
subgraph StandardResponseModule
direction LR
Interceptor{{fa:fa-bullseye StandardResponseInterceptor}}:::interceptor
end
subgraph AuthModule
direction LR
AuthController{{fa:fa-globe AuthController}}:::controller
AuthService([fa:fa-bell-concierge AuthService]):::service
JwtStrategy(["fa:fa-briefcase JwtStrategy"]):::provider
end
subgraph UserModule
direction LR
UserController{{fa:fa-globe UserController}}:::controller
UserService([fa:fa-bell-concierge UserService]):::service
end
subgraph PoliciesModule
direction LR
CaslAbilityFactory(["fa:fa-briefcase CaslAbilityFactory"]):::provider
end
subgraph MailerModule
direction LR
MailerService([fa:fa-bell-concierge MailerService]):::service
end
subgraph JwtModule[ ]
subgraph JwtModulePadding[JwtModule]
end
end
AppModule===>MongooseModule
AppModule===>StandardResponseModule
AppModule===>AuthModule
AuthModule===>UserModule
UserModule-.->MongooseModule
UserModule===>PoliciesModule
AuthModule===>MailerModule
AuthModule-.->MongooseModule
AppModule===>UserModule
AppModule===>MailerModule
end
classDef controller fill:darkgreen
classDef provider fill:#1f2020
classDef service fill:#1f2020
classDef pipe fill:#8b0e5d
classDef guard fill:#8b0e5d
classDef interceptor fill:#8b0e5d
classDef model fill:#b83100
classDef moduleSubgraph fill:#1f2020,stroke:#81B1DB,rx:5,ry:5
classDef globalModule fill:indigo,stroke:#81B1DB,rx:5,ry:5
classDef layoutGroup fill:none,stroke:none
classDef groupStyles rx:10,ry:10
class legend groupStyles
class modules,globalModules,legendLine1,legendLine2,JwtModulePadding,MongooseCoreModulePadding,ConfigModulePadding,ConfigHostModulePadding layoutGroup
class AppModule,MongooseModule,StandardResponseModule,AuthModule,UserModule,PoliciesModule,MailerModule moduleSubgraph
style legend stroke-dasharray: 0 1 1,fill:white,fill-opacity:0.02,opacity:0.95
Models
Model Classes serve as a 'single source of truth' for all modeled data. They are used as an Interface
to create the mongoose schemas, but they are also used to create DTOs using Mapped Types.
The information on Model properties also defines input validation rules enforced when the model is expected in requests, and defines serialization rules when the model is send in responses.
Finally, model properties can also provide OpenAPI documentation information, like descriptions and usage examples.
Having all this information present in a central Model Class avoids code duplication, since derivative classes only need to pick what properties of the Model they want, without worrying about providing documentation, examples, validation rules, etc.
This means that properties on a Model Class can have up to 4 types of decorators on them:
- Schema -
@Prop()
from '@nestjs/mongoose' to add the property to the schema; - Docs -
@ApiProperty()
from '@nestjs/swagger' to add documentation and examples; - Serialization -
@Exclude()
, @Expose()
, and @Transform()
from 'class-transformer' to define serialization rules; - Validation -
@IsString()
, @IsEmail()
, @Min()
, etc... from 'class-validator' to perform input validation;
๐ Example:
@Schema()
class User {
@Prop()
@ApiProperty({ example: 'Mark' })
name: string;
@Prop({ index: { unique: true } })
@ApiProperty({ example: 'markhiggens3310@gmail.com' })
@IsEmail()
email: string;
@Exclude({ toPlainOnly: true })
@Prop()
password: string;
@Exclude({ toClassOnly: true })
@Prop()
lastSeenAt: Date;
@Exclude()
@Prop()
chatAccessKey: string;
@Expose({ groups: ['Admin'] })
@Prop({ type: Date, default: Date.now })
@IsDateString()
registeredAt: Date;
constructor(partial: Partial<User> = {}) {
Object.assign(this, partial);
}
hasCake() {
const registeredDaysAgo = (new Date().getTime() - this.registeredAt.getTime()) / 1000 / 60 / 60 / 24;
return registeredDaysAgo > 365;
}
}
Sending data
When sending data in responses, it's important to always send instances of a Model Class, or instances of DTOs created from it. You can either send a single one, or an array of them. But never send documents retrieved from the database directly in reponses! The serialization rules (and all other benefits from the model) only apply to instances of the Model or derived classes, not documents from the DB.
This also means you should not wrap the returned model in any other javascript object. If you need to add more data to the response (like pagination, filtering, additional messages, etc), you should add them using the metadata decorators provided by nest-standard-response
.
๐ Example:
@Controller('user')
export class UserController {
@Get()
@StandardResponse({
isPaginated: true,
})
public async findAll(
@StandardParam() params: StandardParams
): Promise<User[]> {
const users: UserDocument[] = await this.userModel
.find()
.limit(params.paginationInfo.limit)
.skip(params.paginationInfo.offset)
.exec();
params.setMessage('Custom message...')
params.setExtra('myCustomProperty', {
customObjProp1: 'any serializable value',
customObjProp2: { nested: true },
});
return users.map((userDoc) => new User(userDoc.toJSON()));
}
}
The response from this route would look like this:
Note the smart serialization in the response! The field registeredAt
is only present when an admin
is making the request. It would be hidden from other users because of the serialization rules in the model.
{
success: true,
message: "Custom message...",
isArray: true,
isPaginated: true,
pagination: {
limit: 10,
offset: 0,
defaultLimit: 10,
},
myCustomProperty: {
customObjProp1: 'any serializable value',
customObjProp2: { nested: true },
}
data: [{
name: "Mark",
email: 'mark8829@gmail.com',
registeredAt: '2023-11-09T13:06:37.384Z'
}, {
name: "Jane",
email: 'itsmejane@gmail.com',
registeredAt: '2023-11-09T13:06:37.384Z'
}, {
name: "Eva",
email: 'evanance@hotmail.com' },
registeredAt: '2023-11-09T13:06:37.384Z'
}]
}
Receving data
The same is true for receving data in the request params or body. Always strongly type the expected data as the Model Class or a DTO derived from it. This way the data gets auto validation from the global ValidationPipe
, plus the route gets auto documentation in Open API.
๐ Example:
const requiredFields = ['email', 'password'] as const;
const optionalFields = ['name', 'familyName', 'phone', 'birthDate'] as const;
export class CreateUserDto extends IntersectionType(
PartialType(PickType(User, optionalFields)),
PickType(User, requiredFields),
) {}
@Controller('user')
export class UserController {
@Post())
public async create(
@Body() createUserDto: CreateUserDto
): Promise<User> {
return await this.userService.create(createUserDto);
}
}
}
๐ฎ Use concrete JS classes as types, not Typescript interfaces
Typescript interfaces are completely removed from compiled code. Since we want to perform data validation and transformation at runtime, all models and DTOs must use Classes instead. TS Classes can also be used as types when needed, but they are persisted as JS Classes in the compiled code.
Most decorators work by using the JS Reflect API to store metadata inside of those classes. This often includes the type that those values had in Typescript, but can also include any other value the decorator thinks the app might need at runtime.
At any point during the app execution, other parts of the app might use Reflection to inspect this metadata before deciding on an action. That's how the ValidationPipe
can inspect a Model or DTO instance and validate the type of its properties, or how the RoleSerializerInterceptor
can retrieve the rules used serialize a value.
Reference
Auth Module ๐ช
Back to index ย โคด
- Allows account creation;
- Sends e-mail verification and keeps track of confirmation status;
- Sends forgotten password emails and allows password reset;
- Manages log-in and JWTs;
- Guard routes from unlogged users and injects the logged-in user into the request.
Policies Module ๐๏ธ
Back to index ย โคด
- Defines policies limiting any individual user to access only resources they can claim;
- Claims define which
Actions
(create, read, update, etc...) any user Role
can take on each Model
; - Claims can also define constraint queries, for example allowing a user to read the
User
model, but only for his own user; or to update Articles
, but only those authored by himself;
Note: There is no Articles
module provided by this app. This is just an example on how you can define policies for any model you want.
Policies are defined using Casl.
CaslAbilityFactory
The CaslAbilityFactory
provider exposes the createForUser
function, which is called during a request with the logged-in user information, and should return a casl Ability
object constructed using the provided can
or cannot
methods. This function is free to inspect the user object and define any custom logic it needs to limit individual access to actions
taken on models
.
Example:
if (user.roles.includes(UserRole.USER)) {
can([Action.Read, Action.Update], User, { _id: user._id });
can(Action.Read, Article);
can(Action.Update, Article, { authorId: user._id });
}
if (user.roles.includes(UserRole.MOD)) {
can([Action.Read, Action.Update], User);
can([Action.Read, Action.Update], Article);
}
if (user.roles.includes(UserRole.ADMIN)) {
can(Action.Manage, 'all');
}
Protecting routes
Just add the PoliciesGuard
to any controller or route. Since policies depend on the user object, using this guard also requires using AuthGuard
or other mechanism that guarantees log-in.
@UseGuards(AuthGuard('jwt'), PoliciesGuard)
Once this guard is in place, you can add the @CheckPolicies()
decorator to any route, and choose the claims that are required to access this route. @CheckPolicies()
expects a simple function that is called with the userAbility
object, so you can use can
or cannot
methods on it to define which Actions this route requires on which Models.
@CheckPolicies((ability: UserAbility) => ability.can(Action.List, User))
Checking policies in this way is very efficient, since requests can be denied at the Guard level, without even executing the route handler. But it is also limited: it cannot check for constraint queries since no document has been retrieved from the DB yet. If the logged-in user has access to at least one document for a given Model, it will be granted access by the guard, and you should check for constraints during the route handling.
Protecting access per-document
- The
userAbility
object is also injected in the request object, and you can retrieve it by using req.userAbility
; - If this is all you're using from the request object, it can be cleaner to inject it directly using the custom param decorator
@UserAbilityParam()
;
This allows you to retrieve documents from the database and call the can
or cannot
methods against them. Note that here these methods are called using an instance of the model (instead of on the Model class itself).
function findOne(
@UserAbilityParam() userAbility: UserAbility,
) {
const user = await this.userService.findOne(idOrEmail);
if (userAbility.cannot(Action.Read, user)) {
throw new ForbiddenException();
}
return user;
}
User Module ๐ค
Back to index ย โคด
- Defines the User model, schema and DTOs;
- Defines the services required to create, read, update, delete, list, reset password, and verify email;
- Most services from this module are consumed by the Auth module for managing accounts;
- The user controller provides routes that can be used by admins to manage users from a backend outside of the auth flow;
- Some routes can also be used by users to view or update their own profile;
EmailVerifiedGuard
If the app is configured to use delayed email verification, users will be logged in automatically after account creation, and will be allowed to login anytime without clicking the verification link.
To protect access to certain routes only to users who have verified their email, you can add the EmailVerifiedGuard
to any controller or route.
@UseGuards(EmailVerifiedGuard)
-
If the app is configured to use required email verification, users will be asked to verified their email before being allowed to log-in. In that case, this guard is redundant.
-
If the app is configured with email verification off, this guard shoud not be used, since it will never allow access to the routes under it.
-
The routes from the UserController
that allow users to view and edit their own information use this guard. If you're setting this setting to off, you should also remove this guard from that controller.
EmailOrIdPipe
Both email
and id
are unique keys in the user schema. An id
provides consistency since it should never be changed, and also provides some privacy if you need to include a user reference in a public link without exposing their email
. However, sometimes using an email can be more convenient.
That's why routes and services from the User
module accept both an id
or an email
as the target for their operations. To validate the input parameters in those cases, the app provides the EmailOrIdPipe
pipe.
@Controller('user')
export class UserController {
constructor(private readonly userService: UserService) {}
@Get(':idOrEmail')
public async findOne(
@Param('idOrEmail', EmailOrIdPipe) idOrEmail: string
): Promise<User> {
const user = await this.userService.findOne(idOrEmail);
...
}
}
When used, it makes sure the piped data is either a syntactically valid email
or a syntactically valid ObjectId
. Note that a pipe can only check for syntax. It will throw a HTTP 400 BadRequestException if the provided information is malformatted, but it's possible that the information is valid yet still doesn't match any known user from the DB.
Mailer Module ๐ฎ
Back to index ย โคด
- Automatically creates and configures a nodemailer instance using info from the .env file injected by the config module;
- Defines services for sending emails;
- Currently this module can send the following emails:
- Welcome
- Please confirm yout email
- Forgot your password?
- Your password was reset
Config Module โ๏ธ
Back to index ย โคด
- Prevents runtime errors by validating environment variables during app startup;
- Provides helpful console messages when envorinment variables are missing or invalid;
- Parses
.env
vars into a strongly typed Configuration object that can be dependency injected; - Exposes interfaces that can be used to provide types when calling the
configService.get<>()
generic method;
Example:
@Controller('books')
export class BooksController {
constructor(private readonly configService: ConfigService) {}
@Get()
public async listBooks() {
const apiConfig = this.configService.get<ApiConfig>('api');
console.log(apiConfig.internalUrl);
}
}
Standard Response Module ๐ฆ
Back to index ย โคด
StandardReponse has been exported into a separate package. The full documentation now resides in it's own repo.
-
Metadata-based wrapper to provide customizable and standardized API response objects;
-
Built-in handling of pagination, sorting and filtering;
-
Allows route handlers to keep returning classes instead of wrapper objects, so they remain fully compatible with interceptors;
@get("/books")
@StandardResponse({
isPaginated: true,
isSorted: true,
isFiltered: true,
})
async listBooks(
@StandardParam() params: StandardParams
): BookDto[] {
const {
books,
count
} = await this.bookService.list({
limit: params.pagination.limit,
offset: params.pagination.offset,
sort: params.pagination.sort,
filter: params.pagination.filter,
});
params.setPaginationInfo({ count: count })
params.setMessage('Custom message...')
return books;
}
|
{
success: true,
message: "Custom message...",
isArray: true,
isPaginated: true,
isSorted: true,
isFiltered: true,
pagination: {
limit: 10,
offset: 0,
defaultLimit: 10,
count: 33
},
sorting: {
query: ...,
sortableFields: [...],
sort: SortingInfo
},
filtering: {
query: ...,
filterableFields: [...],
filter: FilteringInfo
},
data: [
{ title: "Dune", year: 1965 },
{ title: "Jaws", year: 1974 },
{ title: "Emma", year: 1815 },
]
}
|
'/books?limit=8&offset=16&sort=-author,title&filter=author^=Frank;year>=1960;year>=1970'
โน๏ธ Check out the full documentation to learn:
Test Module ๐งช
Back to index ย โคด
- Provides end-to-end testing of all user interaction flows;
- e2e tests run in docker, using NODE_ENV=production;
- Jest runs tests in parallel, so each test file needs to instantiate the app in it's own thread;
- The DB is shared between all threads. To avoid racing conditions, the DB should never be dropped during testing. Use the provided factories to create and destroy resources instead.
To facilitate creating and destroing instances of the NestJS application, as well as registering all kinds of test users, this project provides two utility factories:
TestingServerFactory
When creating new e2e test files, use the beforeAll
hook from jest to instantiate a new NestJS app by calling await new TestingServerFactory().create()
.
This method will create a new TestingModule, mock the mailer service, start the app, auto-increment the port number to avoid conflicts, and return an instance with methods to retrieve all created resources, like the getModule()
, getApp()
and getBaseUrl()
.
Since this gives you access to the underlying NestJS TestingModule, you can reach any part of the nest app by using the get()
and resolve()
methods on the module.
UserStubFactory
To create stub users for testing access control and serialization, use the UserStubFactory. It provides methods for creating regular users, users with verified emails, admin users, etc. It also provides methods to login those users and get their access tokens, as well as to delete them.
The test DB is dropped only once before starting e2e tests. It's a good idea to delete any resource you created in the DB during a test inside the afterAll
hook.
Example:
describe('BooksController (e2e)', () => {
let app: INestApplication;
let stub: UserStubFactory;
let verifiedUser: FakeUser;
let verifiedUserToken: string;
beforeAll(async () => {
const testingServer = await new TestingServerFactory().create();
const testingModule = testingServer.getModule();
app = testingServer.getApp();
booksService = await testingModule.resolve(BooksService);
stub = new UserStubFactory(testingServer);
verifiedUser = await stub.registerNewVerifiedUser({ firstName: 'Martha' });
verifiedUserToken = await stub.getLoginTokenForUser(verifiedUser);
});
afterAll(async () => {
await stub.deleteUser(verifiedUser.email);
await app.close();
});
}
Back to index ย โคด
๐ ย TODO Milestones
- Add a mgob instance to the production docker swarm for automated mongo backups (and add its configurations via .env)
- Add some tool to the production docker swarm to expose server metrics
- Add user consent forms with versioned policies
- Add option for log-in using social media accounts
License
MIT License
Copyright (c) 2022 Ricardo Simioni