A powerful TypeScript/JavaScript SDK for the BitBuffet API that allows you to extract structured data from any web content using Zod schemas or raw markdown content in under two seconds.
npm install bitbuffet
# or
yarn add bitbuffet
# or
pnpm add bitbuffet
import { BitBuffet } from 'bitbuffet';
import { z } from 'zod';
// Define your data structure with Zod
const ArticleSchema = z.object({
title: z.string().describe('The main title of the article'),
author: z.string().describe('The author of the article'),
publishDate: z.string().describe('When the article was published'),
content: z.string().describe('The main content/body of the article'),
tags: z.array(z.string()).describe('Article tags or categories'),
summary: z.string().describe('A brief summary of the article')
});
type Article = z.infer<typeof ArticleSchema>;
// Initialize the client with your API key
const client = new BitBuffet('your-api-key-here');
// Extract structured data from any URL
try {
const result: Article = await client.extract(
'https://example.com/article',
ArticleSchema
);
console.log(`Title: ${result.title}`);
console.log(`Author: ${result.author}`);
console.log(`Published: ${result.publishDate}`);
console.log(`Tags: ${result.tags.join(', ')}`);
} catch (error) {
console.error('Extraction failed:', error);
}
import { BitBuffet } from 'bitbuffet';
const client = new BitBuffet('your-api-key-here');
// Extract raw markdown content
try {
const markdown: string = await client.extract(
'https://example.com/article',
{ format: 'markdown' }
);
console.log('Raw markdown content:');
console.log(markdown);
} catch (error) {
console.error('Extraction failed:', error);
}
Choose between structured JSON extraction or raw markdown content:
Extracts structured data according to your Zod schema:
const ProductSchema = z.object({
name: z.string(),
price: z.number(),
description: z.string()
});
const product = await client.extract(
'https://example.com/product',
ProductSchema,
{ format: 'json' } // Optional - this is the default
);
Returns the raw markdown content of the webpage:
const markdown = await client.extract(
'https://example.com/article',
{ format: 'markdown' }
);
Note: When using format: 'markdown', do not provide a schema as the second parameter.
The SDK supports two extraction methods via the format parameter:
// Format 1a: Schema with optional config (format defaults to 'json')
const result = await client.extract(
'https://example.com/article',
ArticleSchema,
{
format: 'json', // Optional - this is the default
reasoning_effort: 'high'
}
);
// Format 1b: Schema without config (format defaults to 'json')
const result = await client.extract(
'https://example.com/article',
ArticleSchema
);
// Format 2a: Config object with format specified
const markdown = await client.extract(
'https://example.com/article',
{
format: 'markdown',
reasoning_effort: 'medium',
prompt: 'Focus on main content'
}
);
// Format 2b: Minimal markdown extraction
const markdown = await client.extract(
'https://example.com/article',
{ format: 'markdown' }
);
JSON Format Requirements:
format: 'json' is optional (default behavior)Markdown Format Requirements:
format: 'markdown' MUST be specified in config objectFormat Parameter Location:
Customize the extraction process with various options:
// JSON extraction with configuration
const result = await client.extract(
'https://example.com/complex-page',
ArticleSchema,
{
format: 'json', // Optional - default behavior
reasoning_effort: 'high', // 'medium' | 'high' - Higher effort for complex pages
prompt: 'Focus on extracting the main article content, ignoring ads and navigation',
temperature: 0.1, // Lower for more consistent results (0.0 - 1.5)
// OR use top_p instead of temperature
// top_p: 0.9
},
30000 // Timeout in milliseconds (default: 30000)
);
// Markdown extraction with configuration
const markdown = await client.extract(
'https://example.com/article',
{
format: 'markdown',
reasoning_effort: 'medium',
prompt: 'Focus on the main content, ignore navigation and ads'
},
30000
);
const ProductSchema = z.object({
name: z.string(),
price: z.number(),
currency: z.string(),
description: z.string(),
images: z.array(z.string().url()),
inStock: z.boolean(),
rating: z.number().min(0).max(5).optional(),
reviews: z.number().optional()
});
const product = await client.extract(
'https://shop.example.com/product/123',
ProductSchema,
{ reasoning_effort: 'high' }
);
const NewsSchema = z.object({
headline: z.string(),
subheadline: z.string().optional(),
author: z.object({
name: z.string(),
bio: z.string().optional()
}),
publishedAt: z.string(),
category: z.string(),
content: z.string(),
relatedArticles: z.array(z.object({
title: z.string(),
url: z.string().url()
})).optional()
});
const article = await client.extract(
'https://news.example.com/breaking-news',
NewsSchema
);
// Extract raw markdown for further processing
const rawContent = await client.extract(
'https://blog.example.com/post/123',
{ format: 'markdown' }
);
// Process the markdown content
const wordCount = rawContent.split(' ').length;
const hasCodeBlocks = rawContent.includes('```');
console.log(`Content has ${wordCount} words`);
console.log(`Contains code blocks: ${hasCodeBlocks}`);
BitBuffet Classnew BitBuffet(apiKey: string) // API key from https://bitbuffet.dev
extract<T>(
url: string,
schema: ZodSchema<T>,
config?: JsonConfig,
timeout?: number
): Promise<T>
extract(
url: string,
config: MarkdownConfig,
timeout?: number
): Promise<string>
Parameters:
url: The URL to extract data fromschema: Zod schema defining the expected data structure (JSON format only)config: Extraction configuration optionstimeout: Request timeout in milliseconds (default: 30000)Returns:
interface JsonConfig {
format?: 'json'; // Optional - this is the default
reasoning_effort?: 'medium' | 'high';
prompt?: string;
temperature?: number; // 0.0 - 1.5
top_p?: number; // Alternative to temperature
}
interface MarkdownConfig {
format: 'markdown'; // Required for markdown extraction
reasoning_effort?: 'medium' | 'high';
prompt?: string;
temperature?: number; // 0.0 - 1.5
top_p?: number; // Alternative to temperature
}
type ExtractionMethod = 'json' | 'markdown';
# Install dependencies
yarn install
# Run tests
yarn test
# Run tests in watch mode
yarn test:watch
# Run integration tests
yarn test:integration
# Build the project
yarn build
# Run manual test
yarn test:manual
Contributions are welcome! Please feel free to submit a Pull Request.
This project is licensed under the MIT License - see the LICENSE file for details.
For detailed documentation, examples, and API reference, visit our complete documentation.
If you encounter any issues or have questions, please open an issue on GitHub.
FAQs
TypeScript SDK for the Structured Scraper API - BitBuffet
We found that bitbuffet demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
Malicious packages published to npm, PyPI, Go Modules, crates.io, and Packagist impersonate developer tooling to fetch staged malware, steal credentials and wallets, and enable remote access.
Security News
Microsoft has released an open source toolkit for enforcing runtime security policies on AI agents as adoption accelerates faster than governance controls.
Security News
Multiple high-impact npm maintainers confirm they have been targeted in the same social engineering campaign that compromised Axios.