DotnetSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
DotnetSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
DotnetSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
A dedicated crawler agent for accessing public data from mangafire.to. Built on KamiYomu.CrawlerAgents.Core, it enables efficient search, metadata extraction, and integration with the KamiYomu platform.
Search engines and social networks are always trying to crawl your pages, but they only see the javascript tags... We render your javascript in a browser, save the static HTML, and you return that to the crawlers!
Biblioteca para criação de web crawlers
A headless wrapper of the .NET WebBrowser useful in automated browser testing, or as web crawler or screen scrapper and includes both HTML and JavaScript processing capabilities.
Extension for bi-directional collection traversal.
DotnetSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
A .NET SDK for the WebCrawlerAPI - turn website into data
A lib which is used of Chinese unstructured text capture.
DotnetSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
Deterministic, configurable HTML normalization for hashing and change detection.
A .NET Standard web crawling library similar to WebMagic. It is a lightweight, modular, efficient and fast high-level web crawling. scraping framework for .NET
ExcavatorSharp is a multi-threaded server for scraping web data. It converts HTML code into a structured array of data. The library allows data scraping from multiple sites in parallel mode, within a single running application. Create scraping tasks and perform data extraction on a schedule. The library is designed for professional extraction and parsing of large volumes of data. Under the hood there are .css-selectors and xpath support, data export into .csv/.xlsx/.sql/.json, online data export, support for proxy servers, dynamic content crawling, interaction with the site via javascript and much more. The library uses .NET Sockets and Chromium Embedded Framework. The library can be used separately as crawler or parser. We support the formats sitemap.xml and robots.txt. We support the gzip / deflate compression. Attention! Only x64 versions are supported for .NET 4.5.2 and 4.6 platforms. AnyCPU build does not support! You will NOT be able to run the library when building AnyCPU. This is caused by the features of CEF.
A dedicated crawler agent for accessing public data from mangapark. Built on KamiYomu.CrawlerAgents.Core, it enables efficient search, metadata extraction, and integration with the KamiYomu platform.
MarkdownBlog: A convenient and flexible .NET-based solution for generating blogs using Markdown.
A lightweight, composable web crawler for .NET
A dedicated crawler agent for accessing public data from KamiYomu Sample Manga. Built on KamiYomu.CrawlerAgents.Core, it enables efficient search, metadata extraction, and integration with the KamiYomu platform.
A version of Sitecore Search Contrib Crawler 2.1.0 that works with Sitecore 6.6
ASP.NET Core Detection service components for identifying details about client device, browser, engine, platform, and crawler. Responsive middleware for routing base upon request client device detection to specific view. Also in the added feature of user preference made this library even more comprehensive must for developers whom to target multiple devices with view rendered and optimized directly from the server side.
A Simple and Powerful Library to Deal with Web Robots Control Strategy.
A .NET Standard web crawling library similar to WebMagic. It is a lightweight, modular, efficient and fast high-level web crawling. scraping framework for .NET
A dedicated crawler agent for accessing public data from weebcentral.com. Built on KamiYomu.CrawlerAgents.Core, it enables efficient search, metadata extraction, and integration with the KamiYomu platform.
Crawls INSDC (International Nucleotide Sequence Database Collaboration) records from ENA — enumerates every run, experiment, sample, and study connected to a project accession and persists them via the BioFSharp.INSDC.SQLite store.
It helps you to use HAP in easier and meaningful way via Reflection. It works somehow like Entity-Framework. Go to wiki in github page for tutorial : https://github.com/parsalotfy/HtmlAgilityPack_Helper/wiki
A dedicated crawler agent for accessing public data from mangafire.to. Built on KamiYomu.CrawlerAgents.Core, it enables efficient search, metadata extraction, and integration with the KamiYomu platform. Note chapters with #.#1 are unofficial chapters
Facilita a intereção com Selenium através de comandos mais simples. Além de facilitar a realização de testes unitários, também é possível realizar Web Crawler ou Web Scraping
CrawlSharp is a library useful for crawling web endpoints to identify structure and retrieve contents.
Eine Sammlung von wiederverwendbaren Klassen, die für die Entwicklung von Datenzugriff auf dem Internet nützlich sind.
LucasSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
A dedicated crawler agent for accessing public data from MangaKatana. Built on KamiYomu.CrawlerAgents.Core, it enables efficient search, metadata extraction, and integration with the KamiYomu platform.
A .NET Standard web crawling library similar to WebMagic. It is a lightweight, modular, efficient and fast high-level web crawling. scraping framework for .NET
LucasSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework
LucasSpider, a .NET Standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework