Product
Introducing SSO
Streamline your login process and enhance security by enabling Single Sign-On (SSO) on the Socket platform, now available for all customers on the Enterprise plan, supporting 20+ identity providers.
@mendable/firecrawl-js
Advanced tools
Readme
The Firecrawl JavaScript SDK is a library that allows you to easily scrape and crawl websites, and output the data in a format ready for use with language models (LLMs). It provides a simple and intuitive interface for interacting with the Firecrawl API.
To install the Firecrawl JavaScript SDK, you can use npm:
npm install @mendable/firecrawl-js
FIRECRAWL_API_KEY
or pass it as a parameter to the FirecrawlApp
class.Here's an example of how to use the SDK with error handling:
import FirecrawlApp from '@mendable/firecrawl-js';
async function main() {
try {
// Initialize the FirecrawlApp with your API key
const app = new FirecrawlApp({ apiKey: "YOUR_API_KEY" });
// Scrape a single URL
const url = 'https://mendable.ai';
const scrapedData = await app.scrapeUrl(url);
console.log(scrapedData);
// Crawl a website
const crawlUrl = 'https://mendable.ai';
const params = {
crawlerOptions: {
excludes: ['blog/'],
includes: [], // leave empty for all pages
limit: 1000,
},
pageOptions: {
onlyMainContent: true
}
};
const crawlResult = await app.crawlUrl(crawlUrl, params);
console.log(crawlResult);
} catch (error) {
console.error('An error occurred:', error.message);
}
}
main();
To scrape a single URL with error handling, use the scrapeUrl
method. It takes the URL as a parameter and returns the scraped data as a dictionary.
async function scrapeExample() {
try {
const url = 'https://example.com';
const scrapedData = await app.scrapeUrl(url);
console.log(scrapedData);
} catch (error) {
console.error(
'Error occurred while scraping:',
error.message
);
}
}
scrapeExample();
To crawl a website with error handling, use the crawlUrl
method. It takes the starting URL and optional parameters as arguments. The params
argument allows you to specify additional options for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format.
async function crawlExample() {
try {
const crawlUrl = 'https://example.com';
const params = {
crawlerOptions: {
excludes: ['blog/'],
includes: [], // leave empty for all pages
limit: 1000,
},
pageOptions: {
onlyMainContent: true
}
};
const waitUntilDone = true;
const timeout = 5;
const crawlResult = await app.crawlUrl(
crawlUrl,
params,
waitUntilDone,
timeout
);
console.log(crawlResult);
} catch (error) {
console.error(
'Error occurred while crawling:',
error.message
);
}
}
crawlExample();
To check the status of a crawl job with error handling, use the checkCrawlStatus
method. It takes the job ID as a parameter and returns the current status of the crawl job.
async function checkStatusExample(jobId) {
try {
const status = await app.checkCrawlStatus(jobId);
console.log(status);
} catch (error) {
console.error(
'Error occurred while checking crawl status:',
error.message
);
}
}
// Example usage, assuming you have a jobId
checkStatusExample('your_job_id_here');
The SDK handles errors returned by the Firecrawl API and raises appropriate exceptions. If an error occurs during a request, an exception will be raised with a descriptive error message. The examples above demonstrate how to handle these errors using try/catch
blocks.
Contributions to the Firecrawl JavaScript SDK are welcome! If you find any issues or have suggestions for improvements, please open an issue or submit a pull request on the GitHub repository.
The Firecrawl JavaScript SDK is open-source and released under the MIT License.
FAQs
JavaScript SDK for Firecrawl API
The npm package @mendable/firecrawl-js receives a total of 1,185 weekly downloads. As such, @mendable/firecrawl-js popularity was classified as popular.
We found that @mendable/firecrawl-js demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Streamline your login process and enhance security by enabling Single Sign-On (SSO) on the Socket platform, now available for all customers on the Enterprise plan, supporting 20+ identity providers.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.