===========
aiocfscrape
A simple async Python module to bypass Cloudflare's anti-bot page.
Based on aiohttp ClientSession. Solution was inherited from cfscrape <https://github.com/Anorov/cloudflare-scrape>
_
module.
You could use it eg. with Python 3 and asyncio <https://docs.python.org/3/library/asyncio-dev.html>
_
for concurrent crawling of web resources protected with CloudFlare.
.. contents:: Table of Contents
Installation
Install with pip
.. code:: sh
pip install aiocfscrape
Basic Usage
aiocfscrape is a aiohttp.ClientSession wrapper. So aiohttp client reference <http://aiohttp.readthedocs.io/en/stable/client.html>
_
can be used as the base.
To make simple get request do the following:
.. code:: python
import asyncio
from aiocfscrape import CloudflareScraper
async def test_open_page(url):
async with CloudflareScraper() as session:
async with session.get(url) as resp:
return await resp.text()
if name == 'main':
asyncio.run(test_open_page(''))
Dependencies
- Python
3.5.3+
aiohttp <https://pypi.python.org/pypi/aiohttp>
_ >=3.1.3, <4a
js2py <https://pypi.python.org/pypi/Js2Py>
_
License
aiocfscrape is offered under the MIT license.
Changelog
1.0.0 (2020-03-25)
0.0.9 (2019-03-21)
- Update anit-bot integration to latest cfscrape version [gaardiolor]
0.0.8 (2019-03-16)
- Update anit-bot integration to latest cfscrape version [gaardiolor]
0.0.7 (2019-01-10)
- Fix error when host does not return a "Server" header [pavlodvornikov]
- Update anit-bot integration to latest cfscrape version [gaardiolor]
0.0.6 (2018-06-19)
- Update readme [pavlodvornikov]
0.0.5 (2018-06-19)
- Fix aiohttp version 3 compatibility [ape364]
- Bump js2py [ape364]
- Update anti-bot integration to latest cfscrape version [dteh]
0.0.4 (2018-03-06)
- Update anti-bot integration to latest cfscrape version [slazarov]
0.0.3 (2017-11-21)
- Fix AttributError [ape364]
0.0.2 (2016-08-26)
- Support timeout inside request [pavlodvornikov]
- Update package information [pavlodvornikov]
- Remove obsolete files [pavlodvornikov]
0.0.1 (2016-08-26)