@memberjunction/core
The @memberjunction/core library is the foundational package of the MemberJunction ecosystem. It provides a comprehensive, tier-independent interface for metadata management, entity data access, view and query execution, transaction management, security, and more. All MemberJunction applications -- whether running on the server, in the browser, or via API -- depend on this package.
Installation
npm install @memberjunction/core
Architecture Overview
flowchart TB
subgraph Application["Application Layer"]
style Application fill:#2d6a9f,stroke:#1a4971,color:#fff
App["Your Application<br/>(Angular, Node.js, etc.)"]
end
subgraph Core["@memberjunction/core"]
style Core fill:#2d8659,stroke:#1a5c3a,color:#fff
MD["Metadata"]
BE["BaseEntity"]
RV["RunView"]
RQ["RunQuery"]
RR["RunReport"]
TG["TransactionGroup"]
DS["Datasets"]
LC["LocalCacheManager"]
TM["TelemetryManager"]
LOG["Logging"]
end
subgraph Providers["Provider Layer"]
style Providers fill:#7c5295,stroke:#563a6b,color:#fff
SP["Server Provider<br/>(SQLServerDataProvider)"]
CP["Client Provider<br/>(GraphQLDataProvider)"]
end
subgraph Data["Data Layer"]
style Data fill:#b8762f,stroke:#8a5722,color:#fff
DB[("SQL Server<br/>Database")]
API["GraphQL API"]
end
App --> MD
App --> BE
App --> RV
App --> RQ
MD --> SP
MD --> CP
BE --> SP
BE --> CP
RV --> SP
RV --> CP
SP --> DB
CP --> API
API --> DB
The package uses a provider model that allows the same application code to run transparently on different tiers. On the server, a SQLServerDataProvider communicates directly with the database. On the client, a GraphQLDataProvider routes requests through the GraphQL API. Your code does not need to know which provider is active.
Key Features
- Metadata-Driven Architecture -- Complete access to MemberJunction metadata including entities, fields, relationships, and permissions
- Entity Data Access -- Type-safe base classes for loading, saving, and manipulating entity records with dirty tracking and validation
- View Execution -- Powerful view running capabilities for both stored and dynamic views with filtering, pagination, and aggregation
- Query Execution -- Secure parameterized query execution with Nunjucks templates and SQL injection protection
- Transaction Management -- Support for grouped database transactions with atomic commits
- Provider Architecture -- Flexible provider model supporting different execution environments (server, client, API)
- Bulk Data Loading -- Dataset system for efficient loading of related entity collections
- Local Caching -- Intelligent local cache manager with TTL, LRU eviction, and differential updates
- Vector Embeddings -- Built-in support for AI-powered text embeddings and similarity search
- Enhanced Logging -- Structured logging with metadata, categories, severity levels, and verbose control
- Telemetry -- Session-level event tracking for performance monitoring and pattern detection
- BaseEngine Pattern -- Abstract engine base class for building singleton services with automatic data loading
Module Structure
flowchart LR
subgraph index["index.ts (Public API)"]
style index fill:#64748b,stroke:#475569,color:#fff
EX["Exports"]
end
subgraph generic["generic/"]
style generic fill:#2d6a9f,stroke:#1a4971,color:#fff
metadata["metadata.ts"]
baseEntity["baseEntity.ts"]
providerBase["providerBase.ts"]
entityInfo["entityInfo.ts"]
securityInfo["securityInfo.ts"]
interfaces["interfaces.ts"]
transactionGroup["transactionGroup.ts"]
baseEngine["baseEngine.ts"]
compositeKey["compositeKey.ts"]
logging["logging.ts"]
runQuery["runQuery.ts"]
runReport["runReport.ts"]
localCacheManager["localCacheManager.ts"]
telemetryManager["telemetryManager.ts"]
util["util.ts"]
queryCache["QueryCache.ts"]
databaseProvider["databaseProviderBase.ts"]
end
subgraph views["views/"]
style views fill:#2d8659,stroke:#1a5c3a,color:#fff
runView["runView.ts"]
viewInfo["viewInfo.ts"]
end
EX --> metadata
EX --> baseEntity
EX --> runView
EX --> providerBase
EX --> runQuery
EX --> baseEngine
metadata.ts | Primary entry point for accessing MemberJunction metadata and creating entity objects |
baseEntity.ts | Foundation class for all entity record manipulation with state tracking and events |
providerBase.ts | Abstract base class all providers extend, contains caching and refresh logic |
entityInfo.ts | Entity metadata classes: EntityInfo, EntityFieldInfo, EntityRelationshipInfo, etc. |
securityInfo.ts | Security classes: UserInfo, RoleInfo, AuthorizationInfo, AuditLogTypeInfo |
interfaces.ts | Core interfaces: IMetadataProvider, IEntityDataProvider, IRunViewProvider, etc. |
compositeKey.ts | CompositeKey and KeyValuePair for multi-field primary key support |
transactionGroup.ts | TransactionGroupBase for atomic multi-entity operations |
baseEngine.ts | BaseEngine abstract singleton for building services with auto-loaded data |
runQuery.ts | RunQuery class for secure parameterized query execution |
runReport.ts | RunReport class for report generation |
logging.ts | LogStatus, LogError, LogStatusEx, LogErrorEx, verbose controls |
localCacheManager.ts | LocalCacheManager for client-side caching with TTL and LRU eviction |
telemetryManager.ts | TelemetryManager for operation tracking and pattern detection |
queryCache.ts / QueryCacheConfig.ts | LRU query result cache with TTL support |
databaseProviderBase.ts | DatabaseProviderBase for server-side SQL execution and transactions |
util.ts | Utility functions: TypeScriptTypeFromSQLType, FormatValue, CodeNameFromString |
runView.ts | RunView class and RunViewParams for executing stored and dynamic views |
viewInfo.ts | View metadata classes: ViewInfo, ViewColumnInfo, ViewFilterInfo |
Core Components
Metadata
The Metadata class is the primary entry point for accessing MemberJunction metadata and instantiating entity objects. It delegates to a provider set at application startup.
import { Metadata } from '@memberjunction/core';
const md = new Metadata();
await md.Refresh();
const entities = md.Entities;
const applications = md.Applications;
const currentUser = md.CurrentUser;
const roles = md.Roles;
const queries = md.Queries;
Metadata Properties
Applications | ApplicationInfo[] | All applications in the system |
Entities | EntityInfo[] | All entity definitions with fields, relationships, permissions |
CurrentUser | UserInfo | Current authenticated user (client-side only) |
Roles | RoleInfo[] | System roles |
AuditLogTypes | AuditLogTypeInfo[] | Available audit log types |
Authorizations | AuthorizationInfo[] | Authorization definitions |
Libraries | LibraryInfo[] | Registered libraries |
Queries | QueryInfo[] | Query definitions |
QueryFields | QueryFieldInfo[] | Query field metadata |
QueryCategories | QueryCategoryInfo[] | Query categorization |
QueryPermissions | QueryPermissionInfo[] | Query-level permissions |
VisibleExplorerNavigationItems | ExplorerNavigationItem[] | Navigation items visible to the current user |
AllExplorerNavigationItems | ExplorerNavigationItem[] | All navigation items (including hidden) |
ProviderType | 'Database' | 'Network' | Whether the active provider connects directly to DB or via network |
LocalStorageProvider | ILocalStorageProvider | Persistent local storage (IndexedDB, file, memory) |
Helper Methods
const entityId = md.EntityIDFromName('Users');
const entityName = md.EntityNameFromID('12345');
const entityInfo = md.EntityByName('users');
const entity = md.EntityByID('12345');
const name = await md.GetEntityRecordName('Users', compositeKey);
const names = await md.GetEntityRecordNames(infoArray);
const isFavorite = await md.GetRecordFavoriteStatus(userId, 'Orders', key);
await md.SetRecordFavoriteStatus(userId, 'Orders', key, true);
const deps = await md.GetRecordDependencies('Orders', primaryKey);
const entityDeps = await md.GetEntityDependencies('Orders');
const dupes = await md.GetRecordDuplicates(duplicateRequest);
const mergeResult = await md.MergeRecords(mergeRequest);
const changes = await md.GetRecordChanges<RecordChangeEntity>('Users', primaryKey);
const txGroup = await md.CreateTransactionGroup();
GetEntityObject()
GetEntityObject<T>() is the correct way to create entity instances. It uses the MemberJunction class factory to ensure the proper subclass is instantiated and supports two overloads.
flowchart LR
subgraph Creation["Entity Creation Flow"]
style Creation fill:#2d6a9f,stroke:#1a4971,color:#fff
GEO["GetEntityObject()"] --> CF["ClassFactory<br/>Lookup"]
CF --> SC["Subclass<br/>Instantiation"]
SC --> NR["NewRecord()<br/>(auto-called)"]
NR --> READY["Entity Ready"]
end
subgraph Loading["Entity Loading Flow"]
style Loading fill:#2d8659,stroke:#1a5c3a,color:#fff
GEO2["GetEntityObject()<br/>with CompositeKey"] --> CF2["ClassFactory<br/>Lookup"]
CF2 --> SC2["Subclass<br/>Instantiation"]
SC2 --> LD["Load()<br/>from Database"]
LD --> READY2["Entity Ready<br/>(with data)"]
end
Creating New Records
const customer = await md.GetEntityObject<CustomerEntity>('Customers');
customer.Name = 'Acme Corp';
await customer.Save();
const order = await md.GetEntityObject<OrderEntity>('Orders', contextUser);
Loading Existing Records
import { CompositeKey } from '@memberjunction/core';
const user = await md.GetEntityObject<UserEntity>('Users', CompositeKey.FromID(userId));
const userByEmail = await md.GetEntityObject<UserEntity>('Users',
CompositeKey.FromKeyValuePair('Email', 'user@example.com'));
const orderItem = await md.GetEntityObject<OrderItemEntity>('OrderItems',
CompositeKey.FromKeyValuePairs([
{ FieldName: 'OrderID', Value: orderId },
{ FieldName: 'ProductID', Value: productId }
]));
const order = await md.GetEntityObject<OrderEntity>('Orders',
CompositeKey.FromID(orderId), contextUser);
BaseEntity
The BaseEntity class is the foundation for all entity record manipulation. All entity classes generated by CodeGen extend it.
Field Access
const name = user.FirstName;
user.FirstName = 'Jane';
const value = user.Get('FirstName');
user.Set('FirstName', 'Jane');
const field = user.Fields.find(f => f.Name === 'Email');
console.log(field.Dirty);
console.log(field.IsUnique);
console.log(field.IsPrimaryKey);
console.log(field.ReadOnly);
Save and Delete
import { EntitySaveOptions } from '@memberjunction/core';
const success = await entity.Save();
const options = new EntitySaveOptions();
options.IgnoreDirtyState = true;
options.SkipEntityAIActions = true;
options.SkipEntityActions = true;
await entity.Save(options);
await entity.Delete();
GetAll() for Spread Operator
BaseEntity uses getter/setter properties, so the spread operator will not capture field values. Use GetAll() instead.
const data = { ...entity };
const data = { ...entity.GetAll(), customField: 'value' };
State Tracking and Events
BaseEntity provides comprehensive state tracking and lifecycle events.
import { BaseEntityEvent } from '@memberjunction/core';
if (entity.IsSaving) { }
if (entity.IsDeleting) { }
if (entity.IsLoading) { }
if (entity.IsBusy) { }
const subscription = entity.RegisterEventHandler((event: BaseEntityEvent) => {
switch (event.type) {
case 'save_started':
console.log(`Save started (${event.saveSubType})`);
break;
case 'save':
console.log('Save completed');
break;
case 'delete_started':
console.log('Delete started');
break;
case 'delete':
console.log('Delete completed, old values:', event.payload?.OldValues);
break;
case 'load_started':
console.log('Load started for key:', event.payload?.CompositeKey);
break;
case 'load_complete':
console.log('Load completed');
break;
case 'new_record':
console.log('NewRecord() called');
break;
}
});
subscription.unsubscribe();
Awaiting In-Progress Operations
await entity.EnsureSaveComplete();
await entity.EnsureDeleteComplete();
await entity.EnsureLoadComplete();
Save Debouncing
Multiple rapid calls to Save() or Delete() are automatically debounced -- the second call receives the same result as the first.
const promise1 = entity.Save();
const promise2 = entity.Save();
const [result1, result2] = await Promise.all([promise1, promise2]);
Global Event Subscription
Monitor all entity operations across the application.
import { MJGlobal, MJEventType, BaseEntity, BaseEntityEvent } from '@memberjunction/core';
MJGlobal.Instance.GetEventListener(true).subscribe((event) => {
if (event.event === MJEventType.ComponentEvent &&
event.eventCode === BaseEntity.BaseEventCode) {
const entityEvent = event.args as BaseEntityEvent;
console.log(`[${entityEvent.baseEntity.EntityInfo.Name}] ${entityEvent.type}`);
}
});
Validation
const result = entity.Validate();
if (!result.Success) {
for (const error of result.Errors) {
console.error(`${error.Source}: ${error.Message}`);
}
}
CompositeKey
The CompositeKey class provides flexible primary key representation supporting both single and multi-field primary keys.
import { CompositeKey, KeyValuePair } from '@memberjunction/core';
const key = CompositeKey.FromID('abc-123');
const key2 = CompositeKey.FromKeyValuePair('Email', 'user@example.com');
const key3 = CompositeKey.FromKeyValuePairs([
{ FieldName: 'OrderID', Value: orderId },
{ FieldName: 'ProductID', Value: productId }
]);
const value = key.GetValueByFieldName('ID');
const str = key.ToString();
const concat = key.ToConcatenatedString();
const valid = key.Validate();
RunView
The RunView class provides powerful view execution capabilities for both stored and dynamic queries.
flowchart LR
subgraph Params["RunViewParams"]
style Params fill:#2d6a9f,stroke:#1a4971,color:#fff
SV["Stored View<br/>(ViewID/ViewName)"]
DV["Dynamic View<br/>(EntityName + Filter)"]
end
subgraph RunView["RunView"]
style RunView fill:#2d8659,stroke:#1a5c3a,color:#fff
RV["RunView()"]
RVS["RunViews()"]
end
subgraph Result["RunViewResult"]
style Result fill:#b8762f,stroke:#8a5722,color:#fff
S["Success"]
R["Results[]"]
TC["TotalRowCount"]
AG["AggregateResults"]
end
SV --> RV
DV --> RV
DV --> RVS
RV --> S
RV --> R
RVS --> S
RV --> AG
Basic Usage
import { RunView, RunViewParams } from '@memberjunction/core';
const rv = new RunView();
const result = await rv.RunView({
ViewName: 'Active Users',
ExtraFilter: "CreatedDate > '2024-01-01'"
});
const typedResult = await rv.RunView<UserEntity>({
EntityName: 'Users',
ExtraFilter: 'IsActive = 1',
OrderBy: 'LastName ASC, FirstName ASC',
Fields: ['ID', 'FirstName', 'LastName', 'Email'],
ResultType: 'entity_object'
});
if (typedResult.Success) {
const users = typedResult.Results;
console.log(`Found ${users.length} users`);
}
Batch Multiple Views
Use RunViews (plural) to execute multiple independent queries in a single operation.
const [users, roles, permissions] = await rv.RunViews([
{
EntityName: 'Users',
ExtraFilter: 'IsActive = 1',
ResultType: 'entity_object'
},
{
EntityName: 'Roles',
OrderBy: 'Name',
ResultType: 'entity_object'
},
{
EntityName: 'Entity Permissions',
ResultType: 'simple'
}
]);
Aggregates
Request aggregate calculations that run in parallel with the main query, unaffected by pagination.
const result = await rv.RunView<OrderEntity>({
EntityName: 'Orders',
ExtraFilter: "Status = 'Completed'",
MaxRows: 50,
Aggregates: [
{ expression: 'SUM(TotalAmount)', alias: 'TotalRevenue' },
{ expression: 'COUNT(*)', alias: 'OrderCount' },
{ expression: 'AVG(TotalAmount)', alias: 'AverageOrder' }
]
});
ResultType and Fields Optimization
const mutableResult = await rv.RunView<UserEntity>({
EntityName: 'Users',
ResultType: 'entity_object'
});
const readOnlyResult = await rv.RunView<{ ID: string; Name: string }>({
EntityName: 'Users',
Fields: ['ID', 'Name'],
ResultType: 'simple'
});
const countResult = await rv.RunView({
EntityName: 'Users',
ExtraFilter: 'IsActive = 1',
ResultType: 'count_only'
});
RunViewParams Reference
ViewID | string | ID of stored view to run |
ViewName | string | Name of stored view to run |
ViewEntity | BaseEntity | Pre-loaded view entity (for performance) |
EntityName | string | Entity name for dynamic views |
ExtraFilter | string | Additional SQL WHERE clause |
OrderBy | string | SQL ORDER BY clause |
Fields | string[] | Field names to return (simple mode only) |
UserSearchString | string | User search term |
MaxRows | number | Maximum rows to return |
StartRow | number | Row offset for pagination |
ResultType | 'simple' | 'entity_object' | 'count_only' | Result format |
IgnoreMaxRows | boolean | Bypass entity MaxRows setting |
SaveViewResults | boolean | Store run results for future exclusion |
ExcludeUserViewRunID | string | Exclude records from a specific prior run |
ExcludeDataFromAllPriorViewRuns | boolean | Exclude all previously returned records |
ForceAuditLog | boolean | Force audit log entry |
CacheLocal | boolean | Use LocalCacheManager for caching |
CacheLocalTTL | number | Cache TTL in milliseconds |
BypassCache | boolean | Skip all server-side caching (read and write). Use for maintenance queries that need true DB state after direct SQL inserts. |
Aggregates | AggregateExpression[] | Aggregate expressions to compute |
RunQuery
The RunQuery class provides secure execution of parameterized stored queries with Nunjucks templates and SQL injection protection.
import { RunQuery, RunQueryParams } from '@memberjunction/core';
const rq = new RunQuery();
const result = await rq.RunQuery({
QueryID: '12345',
Parameters: {
StartDate: '2024-01-01',
EndDate: '2024-12-31',
Status: 'Active'
}
});
const namedResult = await rq.RunQuery({
QueryName: 'Monthly Sales Report',
CategoryPath: '/Sales/',
Parameters: { Month: 12, Year: 2024 }
});
const adhocResult = await rq.RunQuery({
SQL: 'SELECT TOP 100 Name, Status FROM __mj.vwUsers WHERE IsActive = 1'
});
if (result.Success) {
console.log(`Rows: ${result.RowCount}, Time: ${result.ExecutionTime}ms`);
} else {
console.error('Query failed:', result.ErrorMessage);
}
SQL Security Filters
Parameterized queries use Nunjucks templates with built-in SQL injection protection filters:
sqlString | Escapes strings, wraps in quotes | {{ name | sqlString }} produces 'O''Brien' |
sqlNumber | Validates numeric values | {{ amount | sqlNumber }} produces 1000.5 |
sqlDate | Formats dates as ISO 8601 | {{ date | sqlDate }} produces '2024-01-15T00:00:00.000Z' |
sqlBoolean | Converts to SQL bit | {{ flag | sqlBoolean }} produces 1 |
sqlIdentifier | Brackets identifiers | {{ table | sqlIdentifier }} produces [UserAccounts] |
sqlIn | Formats arrays for IN clauses | {{ list | sqlIn }} produces ('A', 'B', 'C') |
sqlLikeContains | Wraps value with % for LIKE contains | {{ term | sqlLikeContains }} produces '%Conference%' |
sqlLikeBegins | Appends % for LIKE begins-with | {{ term | sqlLikeBegins }} produces 'Conference%' |
sqlLikeEnds | Prepends % for LIKE ends-with | {{ term | sqlLikeEnds }} produces '%Conference' |
sqlNoKeywordsExpression | Blocks dangerous SQL keywords | Allows Revenue DESC, blocks DROP TABLE |
RunReport
Execute reports by ID.
import { RunReport, RunReportParams } from '@memberjunction/core';
const rr = new RunReport();
const result = await rr.RunReport({ ReportID: '12345' });
TransactionGroup
Group multiple entity operations into an atomic transaction.
import { Metadata } from '@memberjunction/core';
const md = new Metadata();
const txGroup = await md.CreateTransactionGroup();
await txGroup.AddTransaction(entity1);
await txGroup.AddTransaction(entity2);
const results = await txGroup.Submit();
Each TransactionResult in the returned array contains a Success flag. If any operation fails, all are rolled back.
Datasets
Datasets enable efficient bulk loading of related entity collections in a single operation, reducing database round trips.
flowchart TB
subgraph Dataset["Dataset System"]
style Dataset fill:#2d6a9f,stroke:#1a4971,color:#fff
DEF["Dataset Definition<br/>(name, description)"]
ITEMS["Dataset Items<br/>(entity, filter, code)"]
end
subgraph Loading["Loading Strategies"]
style Loading fill:#2d8659,stroke:#1a5c3a,color:#fff
FRESH["GetDatasetByName()<br/>(always fresh)"]
CACHED["GetAndCacheDatasetByName()<br/>(uses cache if valid)"]
CHECK["IsDatasetCacheUpToDate()<br/>(check freshness)"]
CLEAR["ClearDatasetCache()<br/>(invalidate)"]
end
subgraph Storage["Cache Storage"]
style Storage fill:#b8762f,stroke:#8a5722,color:#fff
IDB["IndexedDB<br/>(Browser)"]
FS["File System<br/>(Node.js)"]
MEM["Memory<br/>(Fallback)"]
end
DEF --> ITEMS
ITEMS --> FRESH
ITEMS --> CACHED
CACHED --> IDB
CACHED --> FS
CACHED --> MEM
import { DatasetItemFilterType } from '@memberjunction/core';
const md = new Metadata();
const dataset = await md.GetAndCacheDatasetByName('ProductCatalog');
const filters: DatasetItemFilterType[] = [
{ ItemCode: 'Products', Filter: 'IsActive = 1' },
{ ItemCode: 'Categories', Filter: 'ParentID IS NULL' }
];
const filteredDataset = await md.GetAndCacheDatasetByName('ProductCatalog', filters);
if (filteredDataset.Success) {
for (const item of filteredDataset.Results) {
console.log(`Loaded ${item.Results.length} records for ${item.EntityName}`);
}
}
const isUpToDate = await md.IsDatasetCacheUpToDate('ProductCatalog');
await md.ClearDatasetCache('ProductCatalog');
BaseEngine
The BaseEngine abstract class is a singleton pattern for building engine/service classes that auto-load and auto-refresh data from entities or datasets.
import { BaseEngine, BaseEnginePropertyConfig } from '@memberjunction/core';
export class MyEngine extends BaseEngine<MyEngine> {
public static get Instance(): MyEngine {
return super.getInstance<MyEngine>();
}
public MyData: SomeEntity[] = [];
protected get Config(): BaseEnginePropertyConfig[] {
return [
{
PropertyName: 'MyData',
EntityName: 'Some Entity',
Filter: 'IsActive = 1',
OrderBy: 'Name ASC',
AutoRefresh: true
}
];
}
}
await MyEngine.Instance.Config(false, contextUser);
const data = MyEngine.Instance.MyData;
Key features:
- Singleton per class via
BaseSingleton
- Declarative data loading via
BaseEnginePropertyConfig
- Automatic refresh when entities are saved or deleted (debounced)
- Local caching support via
CacheLocal and CacheLocalTTL options
- Supports both entity and dataset loading
LocalCacheManager
The LocalCacheManager provides intelligent client-side caching for RunView and RunQuery results with TTL, LRU eviction, and differential updates.
import { LocalCacheManager } from '@memberjunction/core';
const cache = LocalCacheManager.Instance;
cache.Init(localStorageProvider);
const stats = cache.GetStats();
console.log(`Entries: ${stats.totalEntries}, Hits: ${stats.hits}, Misses: ${stats.misses}`);
await cache.ClearAll();
To use caching with RunView, set CacheLocal: true in your RunViewParams:
const result = await rv.RunView({
EntityName: 'Products',
ExtraFilter: 'IsActive = 1',
CacheLocal: true,
CacheLocalTTL: 300000
});
To bypass all caching for a specific query (e.g., maintenance actions that need to see
records inserted via direct SQL that bypassed BaseEntity.Save()), set BypassCache: true:
const result = await rv.RunView({
EntityName: 'Members',
ExtraFilter: 'State IS NOT NULL',
BypassCache: true,
IgnoreMaxRows: true
});
Cross-Server Cache Invalidation
When multiple MJAPI server instances share a Redis-backed ILocalStorageProvider, cache invalidation propagates automatically across all instances. The system uses two complementary mechanisms:
1. BaseEngine path (engine-managed data):
When BaseEntity.Save() fires, BaseEngine catches the MJGlobal event, updates its in-memory arrays, and calls syncLocalCacheForConfig() → LocalCacheManager.UpsertSingleEntity() → Redis SetItem() → pub/sub notification. Other servers receive the notification via OnExternalCacheChange() and refresh their engine data.
2. LocalCacheManager path (all cached data):
LocalCacheManager independently subscribes to MJGlobal BaseEntityEvent events. When any entity is saved or deleted, it finds all cached RunView fingerprints for that entity via a reverse index and either updates them in-place (unfiltered queries) or invalidates them (filtered queries). This ensures that all cached data — not just engine-managed data — stays consistent across servers.
MJAPI-A: BaseEntity.Save()
→ MJGlobal event
→ LocalCacheManager.HandleBaseEntityEvent()
→ Find all cached fingerprints for this entity
→ UpsertSingleEntity() or InvalidateRunViewResult()
→ Redis SetItem() → PUBLISH on mj:__pubsub__
→ MJAPI-B receives → DispatchCacheChange()
→ BaseEngine.OnExternalCacheChange() refreshes arrays
Registering for change notifications:
const fingerprint = LocalCacheManager.Instance.GenerateRunViewFingerprint(params);
const unsubscribe = LocalCacheManager.Instance.RegisterChangeCallback(
fingerprint,
(event: CacheChangedEvent) => {
console.log(`Cache updated by server ${event.SourceServerId}`);
}
);
unsubscribe();
Requirements for cross-server invalidation:
- Redis-backed
ILocalStorageProvider (@memberjunction/redis-provider)
enablePubSub: true in Redis provider config
StartListening() called after provider creation
OnCacheChanged wired to LocalCacheManager.DispatchCacheChange()
Storage Provider Implementations
LocalCacheManager and ProviderBase delegate persistence to an ILocalStorageProvider. MemberJunction ships with several implementations:
InMemoryLocalStorageProvider | @memberjunction/core | Server (Node.js) | None — data lost on restart | Native references (no serialization) |
BrowserLocalStorageProvider | @memberjunction/graphql-dataprovider | Browser | localStorage | JSON-serialized internally |
BrowserIndexedDBStorageProvider | @memberjunction/graphql-dataprovider | Browser | IndexedDB | Native objects via structured clone |
RedisLocalStorageProvider | @memberjunction/redis-provider | Server (Node.js) | Redis — shared across instances, survives restarts | JSON-serialized internally |
For production server deployments, the Redis provider is recommended. See the @memberjunction/redis-provider README for setup instructions.
Generic-typed interface
ILocalStorageProvider is generic — SetItem<T>(key, value, category?) and GetItem<T>(key, category?) thread the value's type through the call:
interface UserCacheEntry { userId: string; roles: string[]; }
await provider.SetItem<UserCacheEntry>('user:1', { userId: 'u-1', roles: ['admin'] }, 'Users');
const user = await provider.GetItem<UserCacheEntry>('user:1', 'Users');
Each implementation handles serialization for its medium internally:
- IndexedDB stores objects natively via the structured clone algorithm —
Date, Map, Set, typed arrays, and nested objects are preserved as-is on retrieval. No JSON.parse on read — significantly faster for cache-heavy workloads.
- localStorage and Redis JSON-encode/decode internally because their underlying media are string-only.
Date instances become ISO strings on round-trip; Map/Set become plain objects.
- In-memory stores object references directly — same identity returned on read.
Class instances (with prototype methods) lose their prototype on retrieval across all providers; store the underlying data shape (e.g. via entity.GetAll() for BaseEntity).
Batched reads via GetItems<T>
For workflows that need many cached entries at once — most notably the smart-cache-check warm-load path that reads ~85 fingerprints per coalesced engine batch — the interface exposes a batched read:
GetItems<T = unknown>(keys: string[], category?: string): Promise<Map<string, T | null>>;
Returns a Map keyed by the input keys. Missing keys map to null. Implementations leverage their backend's native batching primitive:
- IndexedDB: single read transaction with N parallel
get() calls inside it. Trades ~N transactions of overhead for one transaction's commit cost — significant on hot paths because IDB serializes transactions on the same object store. For 85 keys, this is the difference between ~425ms of IDB bookkeeping and ~10ms.
- Redis: one
MGET command, one network round-trip, N values returned. ~N× faster than individual GET calls which each pay full RTT.
- localStorage / in-memory: implemented as a tight loop for API uniformity (no batching benefit on synchronous backends).
Used internally by LocalCacheManager.GetRunViewResults which the smart-cache-check flow calls in two passes (one for the per-fingerprint cache-status payload, one to materialize 'current' entries after the server response). Available to consumer code anywhere multiple cached entries are needed at once.
IndexedDB schema versioning
BrowserIndexedDBStorageProvider derives its IDB DB_VERSION from the @memberjunction/graphql-dataprovider package version (major * 1000 + minor). Patch releases share the same DB version (cache survives); minor releases trigger a one-time onupgradeneeded that wipes all object stores and recreates them empty. Cache repopulates on first use after the upgrade.
This is intentional: it sidesteps the "did this PR change cache format?" review burden — every minor naturally rolls forward to a clean cache. The cost is one slow page load per user per minor (~1s vs. the warm-load path), which is negligible for monthly LTS cadence and well below the perceptual threshold for "instant" on subsequent loads.
For emergency mid-minor cache schema changes, set MANUAL_CACHE_REVISION in storage-providers.ts to force an extra wipe within the same minor release.
Comprehensive Guide: For a deep dive into the full caching architecture — LocalCacheManager internals, differential updates, eviction policies, BaseEngine integration, Redis cross-server sync, GraphQL cache invalidation subscriptions, and deployment topologies — see the Caching & Pub/Sub Guide.
DatabaseProviderBase
An abstract class for server-side providers that need direct SQL execution and transaction control.
abstract class DatabaseProviderBase extends ProviderBase {
abstract ExecuteSQL<T>(query: string, parameters?: unknown[], options?: ExecuteSQLOptions): Promise<T[]>;
abstract BeginTransaction(): Promise<void>;
abstract CommitTransaction(): Promise<void>;
abstract RollbackTransaction(): Promise<void>;
}
Provider Architecture
MemberJunction uses a provider model set once at application startup via SetProvider().
import { SetProvider } from '@memberjunction/core';
SetProvider(myProvider);
This single call configures the provider for Metadata, BaseEntity, RunView, RunReport, and RunQuery simultaneously.
Metadata Caching Optimization
Subsequent provider instances can reuse cached metadata from the first loaded instance to avoid redundant database calls in multi-user server environments.
const config = new ProviderConfigDataBase(
connectionPool,
'__mj',
undefined,
undefined,
false
);
Security and Permissions
const md = new Metadata();
const user = md.CurrentUser;
console.log(user.Email, user.IsActive, user.Type);
const entity = md.EntityByName('Orders');
console.log(entity.AllowCreateAPI);
console.log(entity.AllowUpdateAPI);
console.log(entity.AllowDeleteAPI);
const permissions = entity.Permissions;
for (const perm of permissions) {
console.log(perm.RoleName, perm.CanCreate, perm.CanRead, perm.CanUpdate, perm.CanDelete);
}
Logging
Basic Logging
import { LogStatus, LogError } from '@memberjunction/core';
LogStatus('Operation completed successfully');
LogError('Something went wrong', null, additionalData);
LogStatus('Writing to file', '/logs/output.log');
Enhanced Logging
import { LogStatusEx, LogErrorEx, IsVerboseLoggingEnabled, SetVerboseLogging } from '@memberjunction/core';
LogStatusEx({
message: 'Detailed trace information',
verboseOnly: true,
category: 'BatchProcessor'
});
LogErrorEx({
message: 'Failed to process request',
error: new Error('Network timeout'),
severity: 'critical',
category: 'NetworkError',
metadata: { url: 'https://api.example.com', timeout: 5000 },
includeStack: true
});
SetVerboseLogging(true);
if (IsVerboseLoggingEnabled()) { }
Verbose logging is controlled by the MJ_VERBOSE environment variable (Node.js), global variable, localStorage item, or URL parameter (browser).
TelemetryManager
Session-level performance tracking with pattern detection.
import { TelemetryManager } from '@memberjunction/core';
const tm = TelemetryManager.Instance;
tm.SetEnabled(true);
const eventId = tm.StartEvent('RunView', 'MyComponent.LoadData', {
EntityName: 'Users',
ResultType: 'entity_object'
});
tm.EndEvent(eventId, { cacheHit: false, resultCount: 50 });
const patterns = tm.GetPatterns({ category: 'RunView', minCount: 2 });
Vector Embeddings Support
BaseEntity includes built-in methods for generating and managing vector embeddings for text fields.
import { BaseEntity, SimpleEmbeddingResult } from '@memberjunction/core';
export class MyEntityServer extends MyEntity {
public async Save(): Promise<boolean> {
await this.GenerateEmbeddingsByFieldName([
{
fieldName: 'Description',
vectorFieldName: 'DescriptionVector',
modelFieldName: 'DescriptionVectorModelID'
}
]);
return await super.Save();
}
protected async EmbedTextLocal(textToEmbed: string): Promise<SimpleEmbeddingResult> {
return { vector: [...], modelID: '...' };
}
}
Features:
- Dirty Detection -- Only generates embeddings when source text changes
- Null Handling -- Clears vector fields when source text is empty
- Parallel Processing -- Multiple embeddings generated concurrently
Utility Functions
import {
TypeScriptTypeFromSQLType,
FormatValue,
CodeNameFromString,
SQLFullType,
SQLMaxLength
} from '@memberjunction/core';
TypeScriptTypeFromSQLType('nvarchar');
TypeScriptTypeFromSQLType('int');
TypeScriptTypeFromSQLType('bit');
TypeScriptTypeFromSQLType('datetime');
FormatValue('money', 1234.5);
FormatValue('nvarchar', longText, 2, 'USD', 50);
CodeNameFromString('First Name');
Error Handling
RunView and RunQuery do NOT throw exceptions on failure. Always check Success:
const result = await rv.RunView<UserEntity>({
EntityName: 'Users',
ExtraFilter: 'IsActive = 1'
});
if (result.Success) {
const users = result.Results;
} else {
console.error('View failed:', result.ErrorMessage);
}
For BaseEntity operations, check the return value and LatestResult:
const saved = await entity.Save();
if (!saved) {
const error = entity.LatestResult;
console.error('Save failed:', error.Message);
if (error.ValidationErrors?.length > 0) {
for (const ve of error.ValidationErrors) {
console.error(`${ve.Source}: ${ve.Message}`);
}
}
}
Best Practices
- Always use
Metadata.GetEntityObject() to create entity instances -- never use new directly
- Use generic types with
RunView<T> and GetEntityObject<T> for type safety
- Use
RunViews (plural) to batch multiple independent queries into one operation
- Use
ResultType: 'simple' with Fields for read-only data to improve performance
- Check
Success on RunView/RunQuery results -- these methods do not throw on failure
- Pass
contextUser in server-side code for proper security and audit tracking
- Use
GetAll() instead of the spread operator on BaseEntity instances
- Override both
Load() and LoadFromData() in subclasses that need custom loading logic
- Use transactions for related operations that must succeed or fail together
- Leverage entity metadata for dynamic UI generation and validation
Dependencies
| @memberjunction/global | Core global utilities, class factory, and singleton patterns |
| rxjs | Reactive programming support for observables and event streams |
| zod | Schema validation for entity fields |
| debug | Debug logging utilities with namespace support |
Related Packages
Provider Implementations
Entity Extensions
UI Frameworks
AI Integration
Communication
Actions
Breaking Changes
v2.131.0
- Entity State Tracking: New
IsSaving, IsDeleting, IsLoading, and IsBusy getters on BaseEntity.
- Operation Lifecycle Events: New event types
save_started, delete_started, load_started, and load_complete.
- Delete Debouncing:
Delete() now has the same debouncing behavior as Save().
- Global Event Broadcasting: All operation events are broadcast globally via MJGlobal.
v2.59.0
- Enhanced Logging Functions: New
LogStatusEx and LogErrorEx with structured logging. Existing LogStatus and LogError remain fully backward compatible.
- Verbose Logging Control: New
IsVerboseLoggingEnabled() and SetVerboseLogging() functions.
v2.58.0
- GetEntityObject() auto-calls NewRecord(): No longer necessary to call
NewRecord() manually.
- UUID Generation: Entities with non-auto-increment uniqueidentifier primary keys get UUIDs automatically.
v2.52.0
- LoadFromData() is now async: Update calls to use
await.
TypeScript Support
This library is written in TypeScript and provides full type definitions. All generated entity classes include proper typing for IntelliSense support. The package uses TypeScript strict mode and enforces strong typing throughout -- any types are not used.
License
ISC License - see LICENSE file for details.
Virtual Entities
Virtual entities are read-only entities backed by SQL views rather than physical database tables. They appear in the metadata catalog alongside regular entities but have no underlying base table — only a base view. This makes them ideal for exposing aggregated data, cross-database views, or complex computed datasets as first-class entities.
flowchart LR
subgraph Regular["Regular Entity"]
RT[Base Table] --> RV[Base View]
RV --> RE[Entity Metadata]
end
subgraph Virtual["Virtual Entity"]
VV[SQL View Only] --> VE[Entity Metadata]
end
RE --> API[GraphQL API / RunView]
VE --> API
style Virtual fill:#e8d5f5,stroke:#7b2d8e
style Regular fill:#d5e8f5,stroke:#2d5f8e
Key Properties
VirtualEntity | false | true |
BaseTable | Physical table name | Same as BaseView |
AllowCreateAPI | Configurable | Always false |
AllowUpdateAPI | Configurable | Always false |
| Stored procedures | Generated | None |
Read-Only Enforcement
Virtual entities are enforced as read-only at multiple layers:
Using Virtual Entities
import { Metadata, RunView } from '@memberjunction/core';
const rv = new RunView();
const result = await rv.RunView({
EntityName: 'Sales Summary',
ExtraFilter: `RegionID = '${regionId}'`,
ResultType: 'simple'
});
const md = new Metadata();
const entity = md.EntityByName('Sales Summary');
console.log(entity.VirtualEntity);
console.log(entity.BaseView);
Full Guide: See Virtual Entities Guide for config-driven creation, LLM-assisted field decoration, field metadata, and troubleshooting.
IS-A Type Relationships (Type Inheritance)
MemberJunction supports IS-A type relationships (also called Table-Per-Type / TPT) where a child entity shares its parent's primary key and inherits all parent fields. This enables type hierarchies like Meeting IS-A Product or Webinar IS-A Meeting IS-A Product.
erDiagram
Product ||--o{ Meeting : "IS-A"
Product ||--o{ Publication : "IS-A"
Meeting ||--o{ Webinar : "IS-A"
Product {
uuid ID PK
string Name
decimal Price
}
Meeting {
uuid ID PK,FK
datetime StartTime
int MaxAttendees
}
Webinar {
uuid ID PK,FK
string PlatformURL
boolean IsRecorded
}
How It Works
Child entities share the parent's primary key (same UUID). At runtime, BaseEntity uses persistent composition — each child instance holds a live reference to its parent entity through _parentEntity. All field access, dirty tracking, validation, and save/delete orchestration flow through this composition chain automatically.
EntityInfo IS-A Properties
const md = new Metadata();
const meeting = md.EntityByName('Meetings');
meeting.IsChildType;
meeting.ParentEntityInfo;
meeting.ParentChain;
meeting.AllParentFields;
meeting.ParentEntityFieldNames;
const product = md.EntityByName('Products');
product.IsParentType;
product.ChildEntities;
product.AllowMultipleSubtypes;
BaseEntity Set/Get Routing
For IS-A child entities, parent fields are automatically routed to the parent entity:
const meetingEntity = await md.GetEntityObject<MeetingEntity>('Meetings');
meetingEntity.Set('StartTime', new Date());
meetingEntity.Set('Name', 'Annual Conference');
meetingEntity.Get('Name');
meetingEntity.Dirty;
Disjoint vs Overlapping Subtypes
IS-A relationships support two modes, controlled by the parent entity's AllowMultipleSubtypes flag:
Disjoint (default) — A parent record can be at most ONE child type. The parent auto-chains to its single child via ISAChild, and save/delete delegates through the full chain.
const meeting = await md.GetEntityObject<MeetingEntity>('Meetings', key);
meeting.ISAChild;
meeting.ISAChildren;
meeting.LeafEntity;
Overlapping (AllowMultipleSubtypes = true) — A parent record can simultaneously exist as MULTIPLE child types. The parent does not auto-chain; instead, ISAChildren returns an informational list of which child entity types have records for this PK.
const person = await md.GetEntityObject<PersonEntity>('Persons', key);
person.ISAChild;
person.ISAChildren;
person.LeafEntity;
const member = await md.GetEntityObject<MemberEntity>('Members', key);
member.ISAParent;
member.Save();
Save & Delete Orchestration
- Save — Parent entities are saved first (inner-to-outer), then the child. On server, a shared SQL transaction wraps the entire chain.
- Delete (disjoint) — Child is deleted first, then parents. Disjoint subtype enforcement prevents a parent from being multiple child types simultaneously.
- Delete (overlapping) — Child is deleted, then the parent is checked for remaining children. If other children still exist, the parent is preserved. If no children remain, the parent is also deleted.
Record Change Propagation (Overlapping)
When saving through one branch of an overlapping hierarchy, Record Change entries are automatically propagated to sibling branches that share the same ancestor. This ensures complete audit history across all child types. Propagation is handled at the provider level (SQLServerDataProvider.PropagateRecordChangesToSiblings) using a single SQL batch for efficiency.
Full Guide: See IS-A Relationships Guide for the complete data model, runtime object model, save/delete orchestration sequences, overlapping subtypes, Record Change propagation, provider implementations, CodeGen integration, and troubleshooting.
Organic Keys (Cross-System Matching)
MemberJunction supports Organic Keys for establishing relationships between entities based on shared business data (email addresses, phone numbers, domains, etc.) rather than foreign key constraints. This is essential for cross-system integrations where external platforms (Mailchimp, QuickBooks, HubSpot, etc.) share data values but not primary keys.
Key capabilities:
- Direct matching: Field-to-field comparison with configurable normalization (LowerCaseTrim, Trim, ExactMatch, Custom)
- Compound keys: Match on multiple fields simultaneously (FirstName + LastName + DOB)
- Transitive matching: Bridge through intermediate tables via SQL views for multi-hop relationships
- Bidirectional: Configure on both sides for complete cross-system navigation
- CodeGen integration: Declare organic keys in
additionalSchemaInfo.json — CodeGen creates bridge views, inserts metadata, and generates form panels automatically
const organicKeys = entity.OrganicKeys;
const params = EntityInfo.BuildOrganicKeyViewParams(record, relatedEntity, organicKey);
Full Guide: See Organic Keys Guide for the complete schema, all 4 query patterns, normalization strategies, CodeGen configuration, Angular UI integration, and an end-to-end setup walkthrough.
Documentation
For detailed guides on specific topics, see the docs/ folder:
- Virtual Entities — Config-driven creation, LLM decoration, read-only enforcement
- IS-A Relationships — Type inheritance, save/delete orchestration, provider integration
- Organic Keys — Cross-system matching by shared business data (email, phone, domain), CodeGen integration, transitive views
- RunQuery Pagination — Parameterized queries with pagination support
- Full-Text Search — Database-native FTS via
Metadata.FullTextSearch(), SQL Server FREETEXT / PostgreSQL tsvector, provider architecture, Knowledge Hub integration
Scoring Utilities
ComputeRRF(rankedLists, k?) — Reciprocal Rank Fusion for combining ranked result lists from different retrieval methods. Score-scale independent — works on ordinal position, making it ideal for fusing vector similarity results with full-text search results. Located in @memberjunction/core (exported from src/generic/scoring/ReciprocalRankFusion.ts).
ScoredCandidate — Interface for RRF input/output: { ID: string, Score: number, Metadata?: Record<string, unknown> }
Support
For support, documentation, and examples, visit MemberJunction.com.