umudhasanli/flare-redact

International secret & PII redaction for JS/TS — logs, prompts, HTTP, datasets. 20+ languages, checksum-validated national IDs, reversible LLM layer. Zero deps, ReDoS-safe. - umudhasanli/flare-...

8 stars · 0 forks · TypeScript Topics: nodejs, redaction, security, privacy, typescript, openai, zero-dependency, data-privacy, dlp, gdpr, anonymization, pii, data-masking, log-redaction, github-actions, ai-privacy, secret-scanning, pii-redaction, llm, llm-security

README

flare-redact

Hide secrets & PII in logs, prompts, and text — before they leak.

🌐 International by default — 24 languages 🇬🇧 🇨🇳 🇮🇳 🇪🇸 🇸🇦 🇫🇷 🇵🇹 🇷🇺 🇯🇵 🇩🇪 🇰🇷 🇹🇷 🇮🇹 🇮🇷 🇵🇱 🇺🇦 🇳🇱 🇻🇳 🇮🇩 🇹🇭 🇬🇷 🇮🇱 🇦🇿 🇷🇴

Every leaked secret has the same origin story: someone logged an object, and a password, token, or API key was sitting inside it. The code looked innocent — logger.info({ user }) — but user carried a session token, and now it's in your log aggregator, your error tracker, and three vendors' systems forever. flare-redact is one function you wrap around that data. It reads the content, not just the field names, so it catches the AWS key someone pasted into a free-text note, the JWT in an Authorization header, the card number in a stack trace — and masks them, keeping just enough of a hint to stay debuggable. import { redact } from 'flare-redact';

redact('User alice@corp.com paid with 4242 4242 4242 4242, token ghp_' + 'a'.repeat(36)); // → 'User a@ paid with * 4242, token ghp_' Nothing to configure. No list of field paths to maintain. No native build step.

The same problem now has a new address: your LLM calls. Wrap your OpenAI or Anthropic client and detected secrets are stripped from prompts and restored in the reply — the model never sees those original values, while references survive. Jump to it ↓

🔍 Context-aware — spans carry risk and confidence 🔐 Secure vaults — opaque tokens, optional AES-GCM persistence 🎭 Useful test data — keyed pseudonyms and typed surrogates

🤖 LLM boundary — strips secrets before OpenAI/Anthropic 🌍 24-language secret vocabulary — plus checksum-validated IDs 🪶 Zero runtime dependencies — Node, browser, and edge

Contents

Install Redact anything Redact prompts before they reach an LLM Ways to hide a value Reversible redaction Contextual and model-assisted PII Build a private chat app Your own words See what leaks, and why Guard your logger in one line One policy, everywhere Anonymize a dataset for staging Guard what leaves your app Streams Fail a build when a secret sneaks in CLI What it catches Multilingual secret vocabulary and IDs Custom detectors & allowlists API Security boundaries Why not a field allowlist?

Install npm install flare-redact Node 20+, and it runs in the browser and edge runtimes too — zero dependencies. Upgrading from 0.8? Read the 0.9 migration notes before changing protected transform or vault behavior. Redact anything Strings, arrays, and objects, recursively. The shape you pass in is the shape you get back. import { redact } from 'flare-redact';

redact({ user: 'bob@corp.com', password: 'hunter2', tokens: ['ghp_' + 'b'.repeat(36)], note: 'my aws key is AKIAIOSFODNN7EXAMPLE', }); // → // { // user: 'b@', // password: '*', // sensitive field name // tokens: ['ghp_***'], // note: 'my aws key is AKIA***', // found inside free text // } Redact prompts before they reach an LLM Your app sends user data to OpenAI or Anthropic. Somewhere in that prompt is a customer's email, an API key, or a card number — and now it's left your systems. Wrap the client once, and detected secrets are stripped from prompts and put back in the reply. The model never sees those original values; your code keeps the references it needs. import { wrapOpenAI } from 'flare-redact/llm';

const openai = wrapOpenAI(new OpenAI());

const res = await openai.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: 'Email the invoice to alice@corp.com, card 4242 4242 4242 4242' }], }); your app sends → Email the invoice to alice@corp.com, card 4242 4242 4242 4242 the model sees → Email the invoice to [FR_EMAIL_7f2a…], card [FR_CREDIT_CARD_19be…] your app gets → Sent to alice@corp.com. Card 4242 4242 4242 4242 wasn't stored.

wrapAnthropic does the same for messages.create, including the system prompt. Both handle streaming — placeholders are restored even when one is split across chunks. There's a redactPrompt(text) too if you'd rather hold the vault yourself. Ways to hide a value Pick a mode depending on whether you still need to reason about the data after it's hidden. redact('bob@corp.com', { mode: 'mask' }); // 'b@' (default) redact('bob@corp.com', { mode: 'label' }); // '[REDACTED:email]'

const protectedOptions = { transformSecret: process.env.FLARE_REDACT_SECRET }; redact('bob@corp.com', { ...protectedOptions, mode: 'hash' }); // 'email_3baf4d28d7c88317a…' — HMAC-SHA-256 fingerprint

redact('bob@corp.com', { ...protectedOptions, mode: 'pseudonym' }); // 'kqz@rwmp.dnu' — keyed, deterministic, keeps character classes

redact('bob@corp.com', { ...protectedOptions, mode: 'surrogate' }); // 'user_93a78c61e204@example.invalid' — type-consistent synthetic value Protected deterministic modes require transformSecret; they never silently fall back to a public unsalted fingerprint. hash is useful for correlation, pseudonym retains the original character shape, and surrogate emits typed synthetic values such as reserved-domain emails and Luhn-valid card numbers. Use a separate secret per environment or correlation domain. pseudonym is deliberately not described as format-preserving encryption. It is non-reversible pseudonymization, not NIST FF1. The old fpe name remains as a compatibility alias but is deprecated. Or replace everything with one fixed string: redact(payload, { mask: '█' }); redact(payload, { mask: ({ detector }) => `<${detector.id}>` }); Reversible redaction When you need the originals back — the LLM case above, or handing data to a system you don't trust and getting it back — use a vault. It swaps each secret for a stable placeholder and remembers the mapping. import { createVault } from 'flare-redact';

const vault = createVault(); const safe = vault.redact('charge bob@corp.com on card 4242 4242 4242 4242'); // 'charge [FR_EMAIL_7f2ad4…] on card [FR_CREDIT_CARD_19be63…]'

vault.restore(safe); // 'charge bob@corp.com on card 4242 4242 4242 4242' The same value gets the same placeholder inside one vault, so references survive the round trip. Default placeholders include 96 random bits instead of a global sequence number. Human-readable [EMAIL_1] counters remain available through createVault({ placeholderStyle: 'readable' }) for trusted local workflows. The mapping is as sensitive as the original data. Encrypt it before persistence: import { sealVault, openVault, restore } from 'flare-redact';

const encrypted = await sealVault(vault, process.env.FLARE_REDACT_VAULT_PASSWORD); await fs.writeFile('session.vault.json', JSON.stringify(encrypted), { mode: 0o600 });

const entries = await openVault(encrypted, process.env.FLARE_REDACT_VAULT_PASSWORD); restore(safe, new Map(entries)); Sealed vaults use PBKDF2-SHA-256 with a fresh salt and AES-256-GCM with a fresh nonce. Wrong passwords and modified files fail closed. From the CLI, --vault and --restore use encrypted files by default. Passwords come from FLARE_REDACT_VAULT_PASSWORD (or the variable named by --vault-password-env) so they do not appear in shell history: export FLARE_REDACT_VAULT_PASSWORD='use-a-secret-manager-in-production' flare-redact --vault session.vault.json < input.txt > safe.txt flare-redact --restore session.vault.json < safe.txt > restored.txt Contextual and model-assisted PII Structured identifiers and credentials are best handled by deterministic rules and checksum validators. Names and addresses need context, so three conservative detectors are opt-in: const findings = scan( 'Customer name: Alice Example; address: 120 Cedar Street; DOB: 1990-04-23', { enable: ['contextual'] }, );

// person_name, street_address, date_of_birth // each finding includes risk, confidence, and the exact sensitive span For broader multilingual free-text PII, connect a local model without coupling the zero-dependency core to one ML runtime: const policy = { semanticProvider: { async detect(text) { return [{ detector: 'person_model', label: 'Person', why: 'Local multilingual NER result.', start: 12, end: 25, confidence: 0.94, risk: 'high', }]; }, }, minConfidence: 0.8, };

const safe = await redactAsync(input, policy); Semantic and deterministic spans enter the same overlap arbitration. Higher-risk, higher-priority, and better-validated findings win instead of whichever regular expression happens to run first. Build a private chat app If you're building a chat interface — over your own local model or any API — a session is the drop-in layer. One session holds one vault, so a value keeps the same placeholder across every turn: mask the user's message on the way in, restore the model's reply on the way out. It's model-agnostic and synchronous. Run npm run benchmark on your own target runtime instead of relying on a hardware-independent latency claim. import { createSession } from 'flare-redact';

const session = createSession({ enable: ['pii'] });

// on the way in — the model only ever sees placeholders const prompt = session.redact(userMessage); const reply = await myModel.generate(prompt);

// on the way out — the user sees the real values back show(session.restore(reply)); Streaming? Restore token by token, even when a placeholder is split across chunks: const out = session.stream(); for await (const chunk of modelStream) process(out.push(chunk.text)); process(out.flush()); session.redactMessages([{ role, content }]) masks a whole chat array at once, and session.reset() starts a fresh conversation. Detected original values stay local while your app keeps a reversible reference. Your own words Detectors can't know your product codenames, project names, or internal jargon — so hand them a list. terms catches exactly the words you name (any language, longest match first, word-boundary safe), one-way or reversibly. // one-way, with your own replacement text redact('Launch Project Zeus with Falcon', { terms: { 'Project Zeus': '[CLASSIFIED]', 'Falcon': '[CLASSIFIED]' }, }); // → 'Launch [CLASSIFIED] with [CLASSIFIED]'

// reversible — send to a model, get it back const vault = createVault({ terms: ['Project Zeus'] }); const safe = vault.redact('ship Project Zeus'); // 'ship [FR_CUSTOM_TERM_a17c…]' vault.restore(safe); // 'ship Project Zeus' The same works from the CLI, including a full round-trip — mask, send the safe text anywhere, then restore what comes back:

add words inline or from a file, and write an encrypted vault

later, restore the originals from that map

scan('deploy with password=hunter2 and AKIAIOSFODNN7EXAMPLE'); // → // [ // { detector: 'generic_assignment', label: 'Assigned secret', // why: 'A value assigned to a sensitive-looking field name…', start: 12, … }, // { detector: 'aws_access_key', label: 'AWS access key ID', // why: 'Pairs with a secret key to control cloud resources and billing.', start: 33, … }, // ] Need just the shape of it? import { isClean, summary } from 'flare-redact';

isClean(payload); // → false summary(payload); // → { total: 3, byDetector: { email: 1, github_token: 1, sensitive_key: 1 } } Guard your logger in one line wrapConsole patches console.* so every argument is redacted on the way out, and hands you a function to undo it. import { wrapConsole } from 'flare-redact';

const restore = wrapConsole(); console.log('session', { user: 'bob@x.io', token: 'ghp_…' }); // session { user: 'b@', token: 'ghp_***' } restore(); Prefer to be explicit? Bind your options once and reuse it: import { createRedactor } from 'flare-redact';

const safe = createRedactor({ enable: ['high_entropy'] }); logger.info(safe.redact({ event: 'checkout', user })); One policy, everywhere Define what "sensitive" means once, and apply it at every layer — your app, your logger, your HTTP boundary, your LLM calls. Every adapter takes the same options object, so a secret is masked the same way across the whole system. import { definePolicy } from 'flare-redact'; const policy = { enable: ['high_entropy'], allow: ['status@acme.com'] }; pino — reads the values, not a list of field paths you have to maintain: import pino from 'pino'; import { pinoRedact } from 'flare-redact/pino';

const log = pino(pinoRedact(policy)); log.info({ user: 'bob@corp.com' }); // → {"user":"b@"} winston — a format that redacts every field, symbol metadata left intact: import winston from 'winston'; import { winstonRedact } from 'flare-redact/winston';

winston.format.combine(winston.format(winstonRedact(policy))(), winston.format.json()); HTTP — a safe-to-log snapshot of a request; the live request is untouched: import { httpRedactor } from 'flare-redact/http';

app.use(httpRedactor(policy)); app.use((req, _res, next) => { logger.info(req.redacted()); next(); }); // Authorization and Cookie headers, and any secret in the body or query, are masked. Same policy object flows into flare-redact/llm, wrapConsole, createVault, and redactStream too. Streams Pipe any log stream through it — secrets are masked line by line, even when one is split across chunks. import { redactStream } from 'flare-redact/stream';

process.stdin.pipe(redactStream()).pipe(process.stdout); Anonymize a dataset for staging Point it at a JSON or CSV dump with --mode surrogate and you get deterministic, typed synthetic values. The same input maps the same way in every row under one key, so joins survive without calling the transformation encryption or anonymity. export FLARE_REDACT_SECRET='read-this-from-your-secret-manager' flare-redact --csv --mode surrogate < customers.csv > customers.safe.csv Alice,alice@corp.com,4242 4242 4242 4242 Alice,user_93a78c61e204@example.invalid,7042 5270 7797 8927 Bob,bob@corp.com,5555 5555 5555 4444 → Bob,user_441ae72c0901@example.invalid,0888 2706 6232 0274 Alice,alice@corp.com,4242 4242 4242 4242 Alice,user_93a78c61e204@example.invalid,7042 5270 7797 8927

redactCsv(text, opts) is available from flare-redact/csv for the same thing in code. Guard what leaves your app Stop PII from reaching an analytics, telemetry, or webhook endpoint — wrap fetch and name the hosts you don't trust with the real data. Every other request goes through untouched, so your real API calls are never altered. import { wrapFetch } from 'flare-redact/fetch';

const fetch = wrapFetch(globalThis.fetch, { hosts: ['api.segment.io', 'telemetry.vendor.com'] }); // bodies sent to those hosts are redacted; everything else is left alone Fail a build when a secret sneaks in scan from code, or --scan from the CLI (which exits non-zero on a hit) — drop it into CI or a pre-commit hook. File scans report file:line:column, while machine-readable JSON and SARIF reports never echo the matched secret value:

• run: git ls-files '.env' '.log' '.json' | xargs npx flare-redact --scan

Detector Finds

private_key PEM private key blocks (RSA/EC/OpenSSH/PGP)

aws_access_key AWS access key IDs (AKIA…, ASIA…)

github_token GitHub PATs and OAuth tokens (ghp_…, github_pat_…)

gitlab_token GitLab PATs (glpat-…)

slack_token Slack tokens (xoxb-…)

stripe_key Stripe secret / restricted keys (sk_live_…, rk_…)

openai_key OpenAI API keys (sk-…)

google_api_key Google API keys (AIza…)

sendgrid_key SendGrid API keys (SG.…)

twilio_key Twilio SIDs / keys (AC…, SK…)

npm_token npm tokens (npm_…)

jwt JSON Web Tokens

bearer_token Authorization: Bearer …

basic_auth Authorization: Basic …

url_credentials passwords inside connection strings

generic_assignment password=, api_key: …, secret=… (any language)

email email addresses

obfuscated_email bracket-obfuscated emails such as name [at] host [dot] tld

credit_card card numbers (Luhn-validated)

iban IBANs (mod-97 validated)

discord_bot_token / telegram_bot_token chat bot tokens

shopify_token / square_token commerce tokens

digitalocean_token / azure_storage_key cloud secrets

sentry_dsn / new_relic_key observability secrets

Opt in with enable:

Detector / tag Finds

high_entropy long random-looking tokens of any format (entropy-based)

crypto Bitcoin & Ethereum addresses, BIP39 seed phrases

finance SWIFT/BIC, US ABA routing numbers

vehicle VINs (checksum-validated)

network IPs, MAC addresses, coordinates, internal URLs

phone E.164 phone numbers

Plus object values whose key name is sensitive (password, token, authorization, cookie, cvv, …) are masked regardless of content. Multilingual secret vocabulary and IDs Secrets like API keys and card numbers don't care what language your app is in. Neither does this — but the word-based checks do, so words like password, secret, and token are recognized as assignments and as object keys in all 24 languages below:

🇬🇧 English password 🇨🇳 Chinese 密码 🇮🇳 Hindi पासवर्ड

🇪🇸 Spanish contraseña 🇸🇦 Arabic كلمة المرور 🇫🇷 French mot de passe

🇵🇹 Portuguese senha 🇷🇺 Russian пароль 🇯🇵 Japanese パスワード

🇩🇪 German passwort 🇰🇷 Korean 비밀번호 🇹🇷 Turkish şifre

🇮🇹 Italian segreto 🇮🇷 Persian رمز عبور 🇵🇱 Polish hasło

🇺🇦 Ukrainian пароль 🇳🇱 Dutch wachtwoord 🇻🇳 Vietnamese mật khẩu

🇮🇩 Indonesian kata sandi 🇹🇭 Thai รหัสผ่าน 🇬🇷 Greek κωδικός

🇮🇱 Hebrew סיסמה 🇦🇿 Azerbaijani şifrə 🇷🇴 Romanian parolă

National IDs are opt-in and checksum-validated, so a random run of digits is never mistaken for one. Enable a whole group or a single country by tag: redact(text, { enable: ['pii'] }); // every national ID below redact(text, { enable: ['tr', 'de'] }); // just Turkish and German

Detector Country Validated by

iban 🌐 international (on by default) ISO 13616 mod-97

tr_tckn 🇹🇷 Turkey TCKN checksum

de_tax_id 🇩🇪 Germany ISO 7064 mod-11,10

es_dni 🇪🇸 Spain (DNI/NIE) control letter mod-23

it_codice_fiscale 🇮🇹 Italy odd/even table

br_cpf 🇧🇷 Brazil two mod-11 digits

nl_bsn 🇳🇱 Netherlands 11-test

pl_pesel 🇵🇱 Poland weighted mod-10

ca_sin 🇨🇦 Canada Luhn

us_ssn 🇺🇸 United States issued-range rules

uk_nhs 🇬🇧 United Kingdom (NHS) weighted mod-11

Every algorithm has its own tests against known-valid and known-invalid numbers, so enabling them won't turn your logs into a wall of [REDACTED]. Custom detectors & allowlists Teach it your own secrets, and tell it what to leave alone: redact(text, { custom: [{ id: 'internal_ticket', label: 'Internal ticket', why: 'Leaks internal issue-tracker IDs.', pattern: /\bACME-\d{4,6}\b/g, mask: () => '[TICKET]', default: true, }], allow: ['support@acme.com'], // never redact these exact values red

source