🎩 You're Invited:Meet the Socket team at Black Hat in Las Vegas, August 3-6.RSVP →
Sign In

jamgate

Package Overview
Dependencies
Maintainers
1
Versions
24
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

jamgate - npm Package Compare versions

Comparing version
0.10.5
 to
0.11.0
+34
-0
dist/embeddings/embedder.js

@@ -28,2 +28,36 @@ // Optional local embedding backend (Phase 3, item 4).

/**
* Is this text in a language the bundled model can actually represent?
*
* all-MiniLM-L6-v2 is an **English** model. Fed Persian, Greek, Chinese or Japanese it still
* returns a 384-dimensional vector — it just does not mean anything, and measurement shows
* exactly what it degenerates into: a similarity score for *"is this the same script"*.
*
* 0.62 "ποδήλατο" (bicycle) ~ a Greek memory about studying Greek in Athens
* 0.46 "自転車" (bicycle) ~ a Chinese memory about coffee and languages
* 0.42 "دوچرخه" (bicycle) ~ a Persian memory about studying Greek and Linux
* 0.27 "コーヒー" (coffee) ~ a CHINESE memory that is partly about coffee
*
* Every one of those bicycles is unrelated to the memory it matched, and each outscores the
* true coffee match. Above the 0.35 recall floor (D-063), so a non-English user with the
* optional package installed got recall dominated by "is it in my language" noise — and after
* the tokenizer fix (D-065) their lexical recall finally worked, only for this to bury it.
*
* So the semantic layer is applied only to text the model was trained on. Non-Latin text is
* not embedded at all: no vector is stored, no semantic score is computed, and recall falls
* back to the fuzzy lexical path — which, unlike the embedding, genuinely works in any script.
* Mixed text (a Latin sentence with a few foreign words) still embeds; the bar is what the
* text is mostly made of, not purity.
*
* This is a property of the bundled model, not of embeddings in general. A multilingual model
* behind the same `Embedder` interface would lift the restriction, and that is the reason to
* want one.
*/
export function isModelLanguage(text) {
const letters = text.match(/\p{L}/gu);
if (!letters || letters.length === 0)
return true; // digits/symbols only — harmless either way
const latin = letters.filter((c) => /\p{Script=Latin}/u.test(c)).length;
return latin / letters.length >= 0.8;
}
/**
* Try to build the real Transformers.js embedder. Returns null (never throws) when the

@@ -30,0 +64,0 @@ * optional dependency or the model is unavailable, so the caller can fall back to fuzzy

+12
-5
dist/gate/junk.js

@@ -0,1 +1,2 @@

import { splitWords } from "./relevance.js";
// Non-fact detection (gate layer 1, D-043).

@@ -67,8 +68,14 @@ //

const WEATHER = /\b(it'?s|it is|its)\s+(raining|snowing|drizzling|pouring|sunny|cloudy|overcast|foggy|windy|humid|hot|cold)\b|\b(raining|snowing|drizzling|pouring)\b|\b(the )?weather (in|is|today)\b|-?\d{1,2}\s?°\s?[cf]\b/i;
/** Unicode-aware content tokens: letters/numbers only, punctuation and symbols dropped. */
/**
* Unicode-aware content tokens: letters/numbers only, punctuation and symbols dropped.
*
* Splitting is shared with the recall scorer (`splitWords`) so the gate and recall cannot
* disagree about where the words are. They did: this function was Unicode-aware from the start
* — the Persian case is in its own test — but it split only on whitespace and punctuation, so
* a Chinese or Japanese sentence, written without spaces, counted as ONE token and every such
* memory was refused as "not a statement". The recall fix alone did nothing, because nothing
* in those languages could get past the gate to be recalled (D-065).
*/
export function meaningfulTokens(text) {
return text
.toLowerCase()
.split(/[^\p{L}\p{N}]+/u)
.filter((t) => t.length > 0);
return splitWords(text.toLowerCase());
}

@@ -75,0 +82,0 @@ /** True when `text` carries no statement: fewer than two tokens, only filler, or only

+23
-0
dist/gate/prefilter.js

@@ -29,2 +29,15 @@ /**

export const MIN_TEXT_LENGTH = 4;
/**
* Longest text that can be one memory. There was no upper bound at all: a 200 KB save was
* accepted without comment, which is not a hypothetical — an agent that means to save a fact
* about a file can pass the file. The cost lands everywhere at once (the store is read whole
* on every operation, the decision log keeps the text, embeddings mean-pool it into noise) and
* the memory is useless anyway, because a memory that long is not one fact.
*
* 32 KB is deliberately far above anything legitimate — the longest real memory in the store
* this was measured on is about 1.8 KB, and the largest ever saved through a live agent was
* 1740 characters (D-037). Anything past this is an accident worth naming rather than a
* memory worth keeping, and the rejection says so instead of silently truncating.
*/
export const MAX_TEXT_LENGTH = 32_000;
export function prefilter(text, ctx = {}) {

@@ -41,2 +54,12 @@ const t = text.trim();

}
if (t.length > MAX_TEXT_LENGTH) {
return {
ok: false,
reason: `too long (${t.length} characters, maximum ${MAX_TEXT_LENGTH}). A memory is one ` +
"durable fact, not a document — if this is file or transcript content, save the " +
"conclusion instead. Split it into separate facts and save them one at a time.",
// Never echo tens of kilobytes of unknown content into the decision log.
redact: true,
};
}
// Credentials are checked before every other content rule so that no later rule can

@@ -43,0 +66,0 @@ // reject a secret for a lesser reason and log it in full on the way out (D-042).

+52
-5
dist/gate/relevance.js

@@ -40,8 +40,55 @@ // Fuzzy, dependency-free relevance scoring for recall (Phase 3, item 2).

]);
/** Split text into lowercase alphanumeric tokens, punctuation stripped. */
/**
* Fold a string for matching: decompose, drop combining marks, recompose, lowercase.
*
* This is what makes "café" match "cafe" and "Müller" match "muller". It is applied to the
* query and to the memory identically, so the only effect is that more things compare equal —
* a query can never be folded out of a match its text would have had.
*/
function fold(text) {
return text.normalize("NFKD").replace(/\p{M}+/gu, "").normalize("NFC").toLowerCase();
}
/**
* Split text into lowercase tokens, punctuation stripped.
*
* This split used to be `/[^a-z0-9]+/`, which does not ignore non-ASCII text — it DELETES it.
* Every Persian, Greek, Cyrillic, Arabic, Hebrew, Chinese, Japanese and Korean character was
* dropped on the floor, so a memory written in any of them tokenized to nothing and could
* never be recalled, not even by pasting a word straight out of its own text. Accented Latin
* was mangled the same way: "café" became "caf", "Müller" became "m" and "ller". The store
* this was found on already held Persian memories that had been unrecallable since the day
* they were saved (D-065).
*
* `\p{L}\p{N}` with the `u` flag keeps a letter or a digit in ANY script.
*
* Whitespace is not enough on its own, though: Chinese and Japanese are written without it, so
* a whole sentence arrives as ONE token and an exact-word query matches nothing (the trigram
* scorer cannot help — Dice similarity between a two-character query and a twenty-character
* sentence is near zero even when the query is literally inside it). Runs of Han, Hiragana and
* Katakana are therefore split into single characters, the standard cheap substitute for a
* word segmenter. It over-matches a little on one-character queries; it is the difference
* between "recall works" and "recall does not exist" for those languages. Korean is left alone
* — it uses spaces, and already tokenizes correctly.
*/
const CJK_CLASS = "\\p{Script=Han}\\p{Script=Hiragana}\\p{Script=Katakana}";
const CJK = new RegExp(`[${CJK_CLASS}]`, "u");
/** One CJK character, OR a run of non-CJK letters/digits. The negative lookahead is what keeps
* the second branch from swallowing CJK — `\p{L}` matches Han too, so a plain alternation
* would consume "iPhone用のアプリ" whole after the first Latin character. */
const CJK_SEGMENT = new RegExp(`[${CJK_CLASS}]|(?:(?![${CJK_CLASS}])[\\p{L}\\p{N}])+`, "gu");
/**
* Split text into words without folding — the shared notion of "where are the words" for both
* recall scoring and the junk filter, which has to count words to decide whether a save is a
* statement at all. It is exported because those two must agree: when only the scorer knew how
* to segment Chinese, a Chinese memory was *rejected by the gate* for having "fewer than two
* meaningful words" and the improved scorer never got to see it (D-065).
*/
export function splitWords(text) {
const tokens = text.split(/[^\p{L}\p{N}]+/u).filter(Boolean);
if (!CJK.test(text))
return tokens; // fast path: nothing to segment
return tokens.flatMap((t) => CJK.test(t) ? [...t.matchAll(CJK_SEGMENT)].map((m) => m[0]) : [t]);
}
export function tokenize(text) {
return text
.toLowerCase()
.split(/[^a-z0-9]+/)
.filter(Boolean);
return splitWords(fold(text));
}

@@ -48,0 +95,0 @@ /**

+31
-1
dist/store/fileStore.js

@@ -11,2 +11,3 @@ import { promises as fs } from "node:fs";

import { memoryRelevance, MIN_RELEVANCE } from "../gate/relevance.js";
import { isModelLanguage } from "../embeddings/embedder.js";
import { DEFAULT_DUP_THRESHOLD, DEFAULT_RELATED_MIN, DEFAULT_SEMANTIC_MIN, blendRelevance, cosineSimilarity, } from "../embeddings/vector.js";

@@ -93,2 +94,7 @@ /** Shortest id prefix `forget` will resolve. A v4 UUID's first 8 hex characters are

return undefined;
// The bundled model is English-only, and on other scripts its "similarity" degenerates
// into "same script" — measured, with unrelated words outscoring true matches (D-065).
// Skipping it entirely is what leaves those languages on the lexical path, which works.
if (!isModelLanguage(text))
return undefined;
try {

@@ -113,2 +119,8 @@ return await this.embedder.embed(text);

}
const e = err;
if (e.code === "EACCES" || e.code === "EPERM") {
throw new Error(`jamgate cannot read the memory store at ${this.path} — permission denied. ` +
"Check the file's owner and mode (a service running as another user is the usual " +
"cause), or point JAMGATE_STORE at a path this process can read.");
}
throw err;

@@ -118,3 +130,21 @@ }

return { schemaVersion: CURRENT_SCHEMA_VERSION, memories: [] };
return migrate(JSON.parse(raw), this.ttl);
let parsed;
try {
parsed = JSON.parse(raw);
}
catch (err) {
// A corrupt store used to surface as a bare `Expected property name or '}' in JSON at
// position 2` through the MCP error channel — no file path, no cause, no next step. The
// user sees their agent report a JSON syntax error and has no way to connect that to a
// file on their own disk. Every save and every recall fails from then on, so this is the
// message someone reads at the worst possible moment. Say which file, say that nothing
// was destroyed (the writer refuses to overwrite what it could not read), and say what
// to do. (D-065)
throw new Error(`jamgate cannot read the memory store at ${this.path} — it is not valid JSON ` +
`(${err.message}). Your memories have NOT been modified or deleted: ` +
"Jamgate refuses to overwrite a store it could not parse. Restore the file from a " +
"backup, or move it aside to start a fresh store — inspect it first, the contents " +
"are plain text and usually recoverable by hand.");
}
return migrate(parsed, this.ttl);
}

@@ -121,0 +151,0 @@ async readAll() {

+1
-1
dist/version.js

@@ -9,2 +9,2 @@ /**

*/
export const VERSION = "0.10.5";
export const VERSION = "0.11.0";
+1
-1
package.json
{
"name": "jamgate",
"version": "0.10.5",
"version": "0.11.0",
"mcpName": "io.github.amirj4m/jamgate",

@@ -5,0 +5,0 @@ "description": "A neutral, cross-agent memory quality gate for AI agents, delivered as an MCP server — a gate, not a store.",

+21
-1
README.md

@@ -284,3 +284,7 @@ # Jamgate

By default, recall is **fuzzy lexical** matching (stemming, typo-tolerance, trigrams) —
fast, deterministic, and dependency-free, but blind to synonyms. To also match on
fast, deterministic, and dependency-free, but blind to synonyms. It works in **any script**:
Persian, Greek, Cyrillic, Arabic, Hebrew, Chinese, Japanese and Korean all tokenize and
recall, and accents fold so `café` and `cafe` find each other. (Stemming is English-only, and
Chinese/Japanese are segmented per character rather than per word — good enough to find a term
inside a sentence, not a real word segmenter.) To also match on
*meaning* (so "automobile" recalls a memory about your "car"), install the optional

@@ -310,2 +314,8 @@ embedding backend:

the README's example is — short, single-fact memories.
- **English only.** all-MiniLM-L6-v2 is an English model, and on other scripts its
"similarity" collapses into *"is this the same language"* — measured, with the Greek for
*bicycle* scoring 0.62 against an unrelated Greek memory. So non-Latin text is deliberately
**not** embedded: those languages stay on fuzzy lexical recall, which works properly in
every script. Nothing is lost by installing the package if you write in Persian or Japanese;
nothing is gained either.

@@ -764,2 +774,12 @@ ## Namespaces (scopes)

clear. Don't.
- **A memory is text, and recall puts it in your agent's context.** The gate decides *whether*
something is worth keeping, not whether it is safe to act on. If a memory contains
instructions — because you saved a page that contained them, or an agent inferred a fact
from an untrusted source — those words come back verbatim on the next recall, in a place the
model reads. This is inherent to every memory system; Jamgate reduces the surface (it never
scrapes screens, never mines chat logs, refuses credentials, and requires an explicit
`save_memory` call) but it cannot make text inert. Treat your memory store as trusted input
and review what goes in — `jamgate export` prints all of it.
- **One memory is one fact, up to 32 KB.** Larger saves are refused rather than truncated. If
you want a document remembered, save the conclusion.

@@ -766,0 +786,0 @@ ## How it compares