The module provides handy URL class for URL parsing and changing.
.. image:: https://github.com/aio-libs/yarl/workflows/CI/badge.svg :target: https://github.com/aio-libs/yarl/actions?query=workflow%3ACI :align: right
.. image:: https://codecov.io/gh/aio-libs/yarl/branch/master/graph/badge.svg :target: https://codecov.io/gh/aio-libs/yarl
.. image:: https://img.shields.io/endpoint?url=https://codspeed.io/badge.json :target: https://codspeed.io/aio-libs/yarl
.. image:: https://badge.fury.io/py/yarl.svg :target: https://badge.fury.io/py/yarl
.. image:: https://readthedocs.org/projects/yarl/badge/?version=latest :target: https://yarl.aio-libs.org
.. image:: https://img.shields.io/pypi/pyversions/yarl.svg :target: https://pypi.python.org/pypi/yarl
.. image:: https://img.shields.io/matrix/aio-libs:matrix.org?label=Discuss%20on%20Matrix%20at%20%23aio-libs%3Amatrix.org&logo=matrix&server_fqdn=matrix.org&style=flat :target: https://matrix.to/#/%23aio-libs:matrix.org :alt: Matrix Room — #aio-libs:matrix.org
.. image:: https://img.shields.io/matrix/aio-libs-space:matrix.org?label=Discuss%20on%20Matrix%20at%20%23aio-libs-space%3Amatrix.org&logo=matrix&server_fqdn=matrix.org&style=flat :target: https://matrix.to/#/%23aio-libs-space:matrix.org :alt: Matrix Space — #aio-libs-space:matrix.org
Url is constructed from str:
.. code-block:: pycon
from yarl import URL url = URL('https://www.python.org/~guido?arg=1#frag') url URL('https://www.python.org/~guido?arg=1#frag')
All url parts: scheme, user, password, host, port, path, query and fragment are accessible by properties:
.. code-block:: pycon
url.scheme 'https' url.host 'www.python.org' url.path '/~guido' url.query_string 'arg=1' url.query <MultiDictProxy('arg': '1')> url.fragment 'frag'
All url manipulations produce a new url object:
.. code-block:: pycon
url = URL('https://www.python.org') url / 'foo' / 'bar' URL('https://www.python.org/foo/bar') url / 'foo' % {'bar': 'baz'} URL('https://www.python.org/foo?bar=baz')
Strings passed to constructor and modification methods are automatically encoded giving canonical representation as result:
.. code-block:: pycon
url = URL('https://www.python.org/шлях') url URL('https://www.python.org/%D1%88%D0%BB%D1%8F%D1%85')
Regular properties are percent-decoded, use raw_ versions for
getting encoded strings:
.. code-block:: pycon
url.path '/шлях'
url.raw_path '/%D1%88%D0%BB%D1%8F%D1%85'
Human readable representation of URL is available as .human_repr():
.. code-block:: pycon
url.human_repr() 'https://www.python.org/шлях'
For full documentation please read https://yarl.aio-libs.org.
::
$ pip install yarl
The library is Python 3 only!
PyPI contains binary wheels for Linux, Windows and MacOS. If you want to install
yarl on another operating system where wheels are not provided,
the the tarball will be used to compile the library from
the source code. It requires a C compiler and and Python headers installed.
To skip the compilation you must explicitly opt-in by using a PEP 517
configuration setting pure-python, or setting the YARL_NO_EXTENSIONS
environment variable to a non-empty value, e.g.:
.. code-block:: console
$ pip install yarl --config-settings=pure-python=false
Please note that the pure-Python (uncompiled) version is much slower. However, PyPy always uses a pure-Python implementation, and, as such, it is unaffected by this variable.
YARL requires multidict_ and propcache_ libraries.
The documentation is located at https://yarl.aio-libs.org.
There is no standard for boolean representation of boolean values.
Some systems prefer true/false, others like yes/no, on/off,
Y/N, 1/0, etc.
yarl cannot make an unambiguous decision on how to serialize bool values because
it is specific to how the end-user's application is built and would be different for
different apps. The library doesn't accept booleans in the API; a user should convert
bools into strings using own preferred translation protocol.
furl (https://pypi.python.org/pypi/furl)
The library has rich functionality but the furl object is mutable.
I'm afraid to pass this object into foreign code: who knows if the code will modify my url in a terrible way while I just want to send URL with handy helpers for accessing URL properties.
furl has other non-obvious tricky things but the main objection
is mutability.
URLObject (https://pypi.python.org/pypi/URLObject)
URLObject is immutable, that's pretty good.
Every URL change generates a new URL object.
But the library doesn't do any decode/encode transformations leaving the end user to cope with these gory details.
The project is hosted on GitHub_
Please file an issue on the bug tracker <https://github.com/aio-libs/yarl/issues>_ if you have found a bug
or have some suggestion in order to improve the library.
aio-libs google group: https://groups.google.com/forum/#!forum/aio-libs
Feel free to post your questions and ideas here.
The yarl package is written by Andrew Svetlov.
It's Apache 2 licensed and freely available.
.. _GitHub: https://github.com/aio-libs/yarl
.. _multidict: https://github.com/aio-libs/multidict
.. _propcache: https://github.com/aio-libs/propcache
.. You should NOT be adding new change log entries to this file, this file is managed by towncrier. You may edit previous change logs to fix problems like typo corrections or such. To add a new change log entry, please see https://pip.pypa.io/en/latest/development/#adding-a-news-entry we named the news folder "changes".
WARNING: Don't drop the next directive!
.. towncrier release notes start
(2024-10-30)
Improved performance of many ~yarl.URL methods -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1396 <https://github.com/aio-libs/yarl/issues/1396>, #1397 <https://github.com/aio-libs/yarl/issues/1397>, #1398 <https://github.com/aio-libs/yarl/issues/1398>__.
Improved performance of passing a dict or str to yarl.URL.extend_query() -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1401 <https://github.com/aio-libs/yarl/issues/1401>__.
(2024-10-28)
Added ~yarl.URL.host_port_subcomponent which returns the 3986#section-3.2.2 host and 3986#section-3.2.3 port subcomponent -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1375 <https://github.com/aio-libs/yarl/issues/1375>__.
(2024-10-21)
Fixed blocking I/O to load Python code when creating a new ~yarl.URL with non-ascii characters in the network location part -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1342 <https://github.com/aio-libs/yarl/issues/1342>__.
Migrated to using a single cache for encoding hosts -- by @bdraco <https://github.com/sponsors/bdraco>__.
Passing ip_address_size and host_validate_size to yarl.cache_configure() is deprecated in favor of the new encode_host_size parameter and will be removed in a future release. For backwards compatibility, the old parameters affect the encode_host cache size.
Related issues and pull requests on GitHub:
#1348 <https://github.com/aio-libs/yarl/issues/1348>, #1357 <https://github.com/aio-libs/yarl/issues/1357>, #1363 <https://github.com/aio-libs/yarl/issues/1363>__.
Improved performance of constructing ~yarl.URL -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1336 <https://github.com/aio-libs/yarl/issues/1336>__.
Improved performance of calling yarl.URL.build() and constructing unencoded ~yarl.URL -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1345 <https://github.com/aio-libs/yarl/issues/1345>__.
Reworked the internal encoding cache to improve performance on cache hit -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1369 <https://github.com/aio-libs/yarl/issues/1369>__.
(2024-10-18)
Improved performance of the yarl.URL.joinpath() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1304 <https://github.com/aio-libs/yarl/issues/1304>__.
Improved performance of the yarl.URL.extend_query() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1305 <https://github.com/aio-libs/yarl/issues/1305>__.
Improved performance of the yarl.URL.origin() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1306 <https://github.com/aio-libs/yarl/issues/1306>__.
Improved performance of the yarl.URL.with_path() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1307 <https://github.com/aio-libs/yarl/issues/1307>__.
Improved performance of the yarl.URL.with_query() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1308 <https://github.com/aio-libs/yarl/issues/1308>, #1328 <https://github.com/aio-libs/yarl/issues/1328>.
Improved performance of the yarl.URL.update_query() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1309 <https://github.com/aio-libs/yarl/issues/1309>, #1327 <https://github.com/aio-libs/yarl/issues/1327>.
Improved performance of the yarl.URL.join() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1313 <https://github.com/aio-libs/yarl/issues/1313>__.
Improved performance of ~yarl.URL equality checks -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1315 <https://github.com/aio-libs/yarl/issues/1315>__.
Improved performance of ~yarl.URL methods that modify the network location -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1316 <https://github.com/aio-libs/yarl/issues/1316>__.
Improved performance of the yarl.URL.with_fragment() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1317 <https://github.com/aio-libs/yarl/issues/1317>__.
Improved performance of calculating the hash of ~yarl.URL objects -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1318 <https://github.com/aio-libs/yarl/issues/1318>__.
Improved performance of the yarl.URL.relative() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1319 <https://github.com/aio-libs/yarl/issues/1319>__.
Improved performance of the yarl.URL.with_name() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1320 <https://github.com/aio-libs/yarl/issues/1320>__.
Improved performance of ~yarl.URL.parent -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1321 <https://github.com/aio-libs/yarl/issues/1321>__.
Improved performance of the yarl.URL.with_scheme() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1322 <https://github.com/aio-libs/yarl/issues/1322>__.
(2024-10-16)
Improved performance of the quoter when all characters are safe -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1288 <https://github.com/aio-libs/yarl/issues/1288>__.
Improved performance of unquoting strings -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1292 <https://github.com/aio-libs/yarl/issues/1292>, #1293 <https://github.com/aio-libs/yarl/issues/1293>.
Improved performance of calling yarl.URL.build() -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1297 <https://github.com/aio-libs/yarl/issues/1297>__.
(2024-10-15)
Fixed yarl.URL.build() failing to validate paths must start with a / when passing authority -- by @bdraco <https://github.com/sponsors/bdraco>__.
The validation only worked correctly when passing host.
Related issues and pull requests on GitHub:
#1265 <https://github.com/aio-libs/yarl/issues/1265>__.
Removed support for Python 3.8 as it has reached end of life -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1203 <https://github.com/aio-libs/yarl/issues/1203>__.
Improved performance of constructing ~yarl.URL when the net location is only the host -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1271 <https://github.com/aio-libs/yarl/issues/1271>__.
(2024-10-13)
Improved performance of converting ~yarl.URL to a string -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1234 <https://github.com/aio-libs/yarl/issues/1234>__.
Improved performance of yarl.URL.joinpath() -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1248 <https://github.com/aio-libs/yarl/issues/1248>, #1250 <https://github.com/aio-libs/yarl/issues/1250>.
Improved performance of constructing query strings from ~multidict.MultiDict -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1256 <https://github.com/aio-libs/yarl/issues/1256>__.
Improved performance of constructing query strings with int values -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1259 <https://github.com/aio-libs/yarl/issues/1259>__.
(2024-10-12)
Improved performance of calling yarl.URL.build() -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1222 <https://github.com/aio-libs/yarl/issues/1222>__.
Improved performance of all ~yarl.URL methods that create new ~yarl.URL objects -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1226 <https://github.com/aio-libs/yarl/issues/1226>__.
Improved performance of ~yarl.URL methods that modify the network location -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1229 <https://github.com/aio-libs/yarl/issues/1229>__.
(2024-10-11)
Fixed validation with yarl.URL.with_scheme() when passed scheme is not lowercase -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1189 <https://github.com/aio-libs/yarl/issues/1189>__.
Started building armv7l wheels -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1204 <https://github.com/aio-libs/yarl/issues/1204>__.
Improved performance of constructing unencoded ~yarl.URL objects -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1188 <https://github.com/aio-libs/yarl/issues/1188>__.
Added a cache for parsing hosts to reduce overhead of encoding ~yarl.URL -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1190 <https://github.com/aio-libs/yarl/issues/1190>__.
Improved performance of constructing query strings from ~collections.abc.Mapping -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1193 <https://github.com/aio-libs/yarl/issues/1193>__.
Improved performance of converting ~yarl.URL objects to strings -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1198 <https://github.com/aio-libs/yarl/issues/1198>__.
(2024-10-08)
Switched to using the propcache package for property caching
-- by @bdraco <https://github.com/sponsors/bdraco>__.
The propcache package is derived from the property caching
code in yarl and has been broken out to avoid maintaining it for multiple
projects.
Related issues and pull requests on GitHub:
#1169 <https://github.com/aio-libs/yarl/issues/1169>__.
Started testing with Hypothesis -- by @webknjaz <https://github.com/sponsors/webknjaz>__ and @bdraco <https://github.com/sponsors/bdraco>__.
Special thanks to @Zac-HD <https://github.com/sponsors/Zac-HD>__ for helping us get started with this framework.
Related issues and pull requests on GitHub:
#860 <https://github.com/aio-libs/yarl/issues/860>__.
Improved performance of yarl.URL.is_default_port() when no explicit port is set -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1168 <https://github.com/aio-libs/yarl/issues/1168>__.
Improved performance of converting ~yarl.URL to a string when no explicit port is set -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1170 <https://github.com/aio-libs/yarl/issues/1170>__.
Improved performance of the yarl.URL.origin() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1175 <https://github.com/aio-libs/yarl/issues/1175>__.
Improved performance of encoding hosts -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1176 <https://github.com/aio-libs/yarl/issues/1176>__.
(2024-09-27)
Improved performance of calling yarl.URL.build() with authority -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1163 <https://github.com/aio-libs/yarl/issues/1163>__.
(2024-09-26)
Started rejecting ASCII hostnames with invalid characters. For host strings that
look like authority strings, the exception message includes advice on what to do
instead -- by @mjpieters <https://github.com/sponsors/mjpieters>__.
Related issues and pull requests on GitHub:
#880 <https://github.com/aio-libs/yarl/issues/880>, #954 <https://github.com/aio-libs/yarl/issues/954>.
Fixed IPv6 addresses missing brackets when the ~yarl.URL was converted to a string -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1157 <https://github.com/aio-libs/yarl/issues/1157>, #1158 <https://github.com/aio-libs/yarl/issues/1158>.
Added ~yarl.URL.host_subcomponent which returns the 3986#section-3.2.2 host subcomponent -- by @bdraco <https://github.com/sponsors/bdraco>__.
The only current practical difference between ~yarl.URL.raw_host and ~yarl.URL.host_subcomponent is that IPv6 addresses are returned bracketed.
Related issues and pull requests on GitHub:
#1159 <https://github.com/aio-libs/yarl/issues/1159>__.
(2024-09-23)
No significant changes.
(2024-09-23)
Added ~yarl.URL.path_safe to be able to fetch the path without %2F and %25 decoded -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1150 <https://github.com/aio-libs/yarl/issues/1150>__.
Restore decoding %2F (/) in URL.path -- by @bdraco <https://github.com/sponsors/bdraco>__.
This change restored the behavior before #1057 <https://github.com/aio-libs/yarl/issues/1057>__.
Related issues and pull requests on GitHub:
#1151 <https://github.com/aio-libs/yarl/issues/1151>__.
Improved performance of processing paths -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1143 <https://github.com/aio-libs/yarl/issues/1143>__.
(2024-09-09)
Allowed scheme replacement for relative URLs if the scheme does not require a host -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#280 <https://github.com/aio-libs/yarl/issues/280>, #1138 <https://github.com/aio-libs/yarl/issues/1138>.
Allowed empty host for URL schemes other than the special schemes listed in the WHATWG URL spec -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1136 <https://github.com/aio-libs/yarl/issues/1136>__.
Loosened restriction on integers as query string values to allow classes that implement __int__ -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1139 <https://github.com/aio-libs/yarl/issues/1139>__.
Improved performance of normalizing paths -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1137 <https://github.com/aio-libs/yarl/issues/1137>__.
(2024-09-08)
Added URL.extend_query()() method, which can be used to extend parameters without replacing same named keys -- by @bdraco <https://github.com/sponsors/bdraco>__.
This method was primarily added to replace the inefficient hand rolled method currently used in aiohttp.
Related issues and pull requests on GitHub:
#1128 <https://github.com/aio-libs/yarl/issues/1128>__.
Improved performance of the Cython cached_property implementation -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1122 <https://github.com/aio-libs/yarl/issues/1122>__.
Simplified computing ports by removing unnecessary code -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1123 <https://github.com/aio-libs/yarl/issues/1123>__.
Improved performance of encoding non IPv6 hosts -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1125 <https://github.com/aio-libs/yarl/issues/1125>__.
Improved performance of URL.build()() when the path, query string, or fragment is an empty string -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1126 <https://github.com/aio-libs/yarl/issues/1126>__.
Improved performance of the URL.update_query()() method -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1130 <https://github.com/aio-libs/yarl/issues/1130>__.
Improved performance of processing query string changes when arguments are str -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1131 <https://github.com/aio-libs/yarl/issues/1131>__.
(2024-09-06)
Fixed joining a path when the existing path was empty -- by @bdraco <https://github.com/sponsors/bdraco>__.
A regression in URL.join()() was introduced in #1082 <https://github.com/aio-libs/yarl/issues/1082>__.
Related issues and pull requests on GitHub:
#1118 <https://github.com/aio-libs/yarl/issues/1118>__.
Added URL.without_query_params()() method, to drop some parameters from query string -- by @hongquan <https://github.com/sponsors/hongquan>__.
Related issues and pull requests on GitHub:
#774 <https://github.com/aio-libs/yarl/issues/774>, #898 <https://github.com/aio-libs/yarl/issues/898>, #1010 <https://github.com/aio-libs/yarl/issues/1010>__.
The previously protected types _SimpleQuery, _QueryVariable, and _Query are now available for use externally as SimpleQuery, QueryVariable, and Query -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1050 <https://github.com/aio-libs/yarl/issues/1050>, #1113 <https://github.com/aio-libs/yarl/issues/1113>.
Replaced all ~typing.Optional with ~typing.Union -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1095 <https://github.com/aio-libs/yarl/issues/1095>__.
Significantly improved performance of parsing the network location -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1112 <https://github.com/aio-libs/yarl/issues/1112>__.
Added internal types to the cache to prevent future refactoring errors -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1117 <https://github.com/aio-libs/yarl/issues/1117>__.
(2024-09-04)
Fixed a TypeError with MultiDictProxy and Python 3.8 -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1084 <https://github.com/aio-libs/yarl/issues/1084>, #1105 <https://github.com/aio-libs/yarl/issues/1105>, #1107 <https://github.com/aio-libs/yarl/issues/1107>__.
Improved performance of encoding hosts -- by @bdraco <https://github.com/sponsors/bdraco>__.
Previously, the library would unconditionally try to parse a host as an IP Address. The library now avoids trying to parse a host as an IP Address if the string is not in one of the formats described in 3986#section-3.2.2.
Related issues and pull requests on GitHub:
#1104 <https://github.com/aio-libs/yarl/issues/1104>__.
(2024-09-04)
URL.join()() has been changed to match
3986 and align with
/ operation() and URL.joinpath()()
when joining URLs with empty segments.
Previously urllib.parse.urljoin was used,
which has known issues with empty segments
(python/cpython#84774 <https://github.com/python/cpython/issues/84774>_).
Due to the semantics of URL.join()(), joining an
URL with scheme requires making it relative, prefixing with ./.
.. code-block:: pycon
URL("https://web.archive.org/web/").join(URL("./https://github.com/aio-libs/yarl")) URL('https://web.archive.org/web/https://github.com/aio-libs/yarl')
Empty segments are honored in the base as well as the joined part.
.. code-block:: pycon
URL("https://web.archive.org/web/https://").join(URL("github.com/aio-libs/yarl")) URL('https://web.archive.org/web/https://github.com/aio-libs/yarl')
-- by @commonism <https://github.com/sponsors/commonism>__
This change initially appeared in 1.9.5 but was reverted in 1.9.6 to resolve a problem with query string handling.
Related issues and pull requests on GitHub:
#1039 <https://github.com/aio-libs/yarl/issues/1039>, #1082 <https://github.com/aio-libs/yarl/issues/1082>.
Added ~yarl.URL.absolute which is now preferred over URL.is_absolute() -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1100 <https://github.com/aio-libs/yarl/issues/1100>__.
(2024-09-04)
Added missing type on ~yarl.URL.port -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1097 <https://github.com/aio-libs/yarl/issues/1097>__.
(2024-09-03)
Covered the ~yarl.URL object with types -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1084 <https://github.com/aio-libs/yarl/issues/1084>__.
Cache parsing of IP Addresses when encoding hosts -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1086 <https://github.com/aio-libs/yarl/issues/1086>__.
Covered the ~yarl.URL object with types -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1084 <https://github.com/aio-libs/yarl/issues/1084>__.
Improved performance of handling ports -- by @bdraco <https://github.com/sponsors/bdraco>__.
Related issues and pull requests on GitHub:
#1081 <https://github.com/aio-libs/yarl/issues/1081>__.
(2024-09-01)
Removed support 3986#section-3.2.3 port normalization when the scheme is not one of http, https, wss, or ws -- by @bdraco <https://github.com/sponsors/bdraco>__.
Support for port normalization was recently added in #1033 <https://github.com/aio-libs/yarl/issues/1033>__ and contained code that would do blocking I/O if the scheme was not one of the four listed above. The code has been removed because this library is intended to be safe for usage with asyncio.
Related issues and pull requests on GitHub:
#1076 <https://github.com/aio-libs/yarl/issues/1076>__.
Improved performance of property caching -- by @bdraco <https://github.com/sponsors/bdraco>__.
The reify implementation from aiohttp was adapted to replace the internal cached_property implementation.
Related issues and pull requests on GitHub:
#1070 <https://github.com/aio-libs/yarl/issues/1070>__.
(2024-08-30)
Reverted 3986 compatible URL.join()() honoring empty segments which was introduced in #1039 <https://github.com/aio-libs/yarl/issues/1039>__.
This change introduced a regression handling query string parameters with joined URLs. The change was reverted to maintain compatibility with the previous behavior.
Related issues and pull requests on GitHub:
#1067 <https://github.com/aio-libs/yarl/issues/1067>__.
(2024-08-30)
Joining URLs with empty segments has been changed
to match 3986.
Previously empty segments would be removed from path, breaking use-cases such as
.. code-block:: python
URL("https://web.archive.org/web/") / "https://github.com/"
Now / operation() and URL.joinpath()()
keep empty segments, but do not introduce new empty segments.
e.g.
.. code-block:: python
URL("https://example.org/") / ""
does not introduce an empty segment.
-- by @commonism <https://github.com/sponsors/commonism>__ and @youtux <https://github.com/sponsors/youtux>__
Related issues and pull requests on GitHub:
#1026 <https://github.com/aio-libs/yarl/issues/1026>__.
The default protocol ports of well-known URI schemes are now taken into account
during the normalization of the URL string representation in accordance with
3986#section-3.2.3.
Specified ports are removed from the str representation of a ~yarl.URL
if the port matches the scheme's default port -- by @commonism <https://github.com/sponsors/commonism>__.
Related issues and pull requests on GitHub:
#1033 <https://github.com/aio-libs/yarl/issues/1033>__.
URL.join()() has been changed to match
3986 and align with
/ operation() and URL.joinpath()()
when joining URLs with empty segments.
Previously urllib.parse.urljoin was used,
which has known issues with empty segments
(python/cpython#84774 <https://github.com/python/cpython/issues/84774>_).
Due to the semantics of URL.join()(), joining an
URL with scheme requires making it relative, prefixing with ./.
.. code-block:: pycon
URL("https://web.archive.org/web/").join(URL("./https://github.com/aio-libs/yarl")) URL('https://web.archive.org/web/https://github.com/aio-libs/yarl')
Empty segments are honored in the base as well as the joined part.
.. code-block:: pycon
URL("https://web.archive.org/web/https://").join(URL("github.com/aio-libs/yarl")) URL('https://web.archive.org/web/https://github.com/aio-libs/yarl')
-- by @commonism <https://github.com/sponsors/commonism>__
Related issues and pull requests on GitHub:
#1039 <https://github.com/aio-libs/yarl/issues/1039>__.
Stopped decoding %2F (/) in URL.path, as this could lead to code incorrectly treating it as a path separator
-- by @Dreamsorcerer <https://github.com/sponsors/Dreamsorcerer>__.
Related issues and pull requests on GitHub:
#1057 <https://github.com/aio-libs/yarl/issues/1057>__.
Dropped support for Python 3.7 -- by @Dreamsorcerer <https://github.com/sponsors/Dreamsorcerer>__.
Related issues and pull requests on GitHub:
#1016 <https://github.com/aio-libs/yarl/issues/1016>__.
On the Contributing docs page,
a link to the Towncrier philosophy has been fixed.
Related issues and pull requests on GitHub:
#981 <https://github.com/aio-libs/yarl/issues/981>__.
The pre-existing / magic method()
has been documented in the API reference -- by @commonism <https://github.com/sponsors/commonism>__.
Related issues and pull requests on GitHub:
#1026 <https://github.com/aio-libs/yarl/issues/1026>__.
A flaw in the logic for copying the project directory into a
temporary folder that led to infinite recursion when TMPDIR
was set to a project subdirectory path. This was happening in Fedora
and its downstream due to the use of pyproject-rpm-macros <https://src.fedoraproject.org/rpms/pyproject-rpm-macros>__. It was
only reproducible with pip wheel and was not affecting the
pyproject-build users.
-- by @hroncok <https://github.com/sponsors/hroncok>__ and @webknjaz <https://github.com/sponsors/webknjaz>__
Related issues and pull requests on GitHub:
#992 <https://github.com/aio-libs/yarl/issues/992>, #1014 <https://github.com/aio-libs/yarl/issues/1014>.
Support Python 3.13 and publish non-free-threaded wheels
Related issues and pull requests on GitHub:
#1054 <https://github.com/aio-libs/yarl/issues/1054>__.
The CI/CD setup has been updated to test arm64 wheels
under macOS 14, except for Python 3.7 that is unsupported
in that environment -- by @webknjaz <https://github.com/sponsors/webknjaz>__.
Related issues and pull requests on GitHub:
#1015 <https://github.com/aio-libs/yarl/issues/1015>__.
Removed unused type ignores and casts -- by @hauntsaninja <https://github.com/sponsors/hauntsaninja>__.
Related issues and pull requests on GitHub:
#1031 <https://github.com/aio-libs/yarl/issues/1031>__.
port, scheme, and raw_host are now cached_property -- by @bdraco <https://github.com/sponsors/bdraco>__.
aiohttp accesses these properties quite often, which cause urllib to build the _hostinfo property every time. port, scheme, and raw_host are now cached properties, which will improve performance.
Related issues and pull requests on GitHub:
#1044 <https://github.com/aio-libs/yarl/issues/1044>, #1058 <https://github.com/aio-libs/yarl/issues/1058>.
Started raising TypeError when a string value is passed into
yarl.URL.build() as the port argument -- by @commonism <https://github.com/sponsors/commonism>__.
Previously the empty string as port would create malformed URLs when rendered as string representations. (#883 <https://github.com/aio-libs/yarl/issues/883>__)
The leading -- has been dropped from the PEP 517 <https://peps.python.org/pep-517>__ in-tree build
backend config setting names. --pure-python is now just pure-python
-- by @webknjaz <https://github.com/sponsors/webknjaz>__.
The usage now looks as follows:
.. code-block:: console
$ python -m build \
--config-setting=pure-python=true \
--config-setting=with-cython-tracing=true
(#963 <https://github.com/aio-libs/yarl/issues/963>__)
A step-by-step Release Guide guide has
been added, describing how to release yarl -- by @webknjaz <https://github.com/sponsors/webknjaz>__.
This is primarily targeting maintainers. (#960 <https://github.com/aio-libs/yarl/issues/960>__)
Coverage collection has been implemented for the Cython modules
-- by @webknjaz <https://github.com/sponsors/webknjaz>__.
It will also be reported to Codecov from any non-release CI jobs.
To measure coverage in a development environment, yarl can be installed in editable mode:
.. code-block:: console
$ python -Im pip install -e .
Editable install produces C-files required for the Cython coverage plugin to map the measurements back to the PYX-files.
#961 <https://github.com/aio-libs/yarl/issues/961>__
It is now possible to request line tracing in Cython builds using the
with-cython-tracing PEP 517 <https://peps.python.org/pep-517>__ config setting
-- @webknjaz <https://github.com/sponsors/webknjaz>__.
This can be used in CI and development environment to measure coverage on Cython modules, but is not normally useful to the end-users or downstream packagers.
Here's a usage example:
.. code-block:: console
$ python -Im pip install . --config-settings=with-cython-tracing=true
For editable installs, this setting is on by default. Otherwise, it's off unless requested explicitly.
The following produces C-files required for the Cython coverage plugin to map the measurements back to the PYX-files:
.. code-block:: console
$ python -Im pip install -e .
Alternatively, the YARL_CYTHON_TRACING=1 environment variable
can be set to do the same as the PEP 517 <https://peps.python.org/pep-517>__ config setting.
#962 <https://github.com/aio-libs/yarl/issues/962>__
yarl.URL.joinpath() -- by @gmacon <https://github.com/sponsors/gmacon>. (#862 <https://github.com/aio-libs/yarl/issues/862>, #866 <https://github.com/aio-libs/yarl/issues/866>__)yarl.URL.__truediv__() operations (URL / segment) -- by @mjpieters <https://github.com/sponsors/mjpieters>. (#871 <https://github.com/aio-libs/yarl/issues/871>, #884 <https://github.com/aio-libs/yarl/issues/884>__)@mjpieters <https://github.com/sponsors/mjpieters>. (#876 <https://github.com/aio-libs/yarl/issues/876>, #882 <https://github.com/aio-libs/yarl/issues/882>__)URL.missing_port(), URL.__bytes__()
and the encoding argument to yarl.URL.joinpath()
-- by @mjpieters <https://github.com/sponsors/mjpieters>. (#891 <https://github.com/aio-libs/yarl/issues/891>)Integrated Cython 3 to enable building yarl under Python 3.12 -- by @mjpieters <https://github.com/sponsors/mjpieters>. (#829 <https://github.com/aio-libs/yarl/issues/829>, #881 <https://github.com/aio-libs/yarl/issues/881>__)
Declared modern setuptools.build_meta as the PEP 517 <https://peps.python.org/pep-517>__ build
backend in pyproject.toml explicitly -- by @webknjaz <https://github.com/sponsors/webknjaz>. (#886 <https://github.com/aio-libs/yarl/issues/886>)
Converted most of the packaging setup into a declarative setup.cfg
config -- by @webknjaz <https://github.com/sponsors/webknjaz>. (#890 <https://github.com/aio-libs/yarl/issues/890>)
The packaging is replaced from an old-fashioned setup.py to an
in-tree PEP 517 <https://peps.python.org/pep-517>__ build backend -- by @webknjaz <https://github.com/sponsors/webknjaz>__.
Whenever the end-users or downstream packagers need to build yarl from
source (a Git checkout or an sdist), they may pass a config_settings
flag --pure-python. If this flag is not set, a C-extension will be built
and included into the distribution.
Here is how this can be done with pip:
.. code-block:: console
$ python -m pip install . --config-settings=--pure-python=false
This will also work with -e | --editable.
The same can be achieved via pypa/build:
.. code-block:: console
$ python -m build --config-setting=--pure-python=false
Adding -w | --wheel can force pypa/build produce a wheel from source
directly, as opposed to building an sdist and then building from it. (#893 <https://github.com/aio-libs/yarl/issues/893>__)
.. attention::
v1.9.3 was the only version using the --pure-python setting name.
Later versions dropped the -- prefix, making it just pure-python.
Declared Python 3.12 supported officially in the distribution package metadata
-- by @edgarrmondragon <https://github.com/sponsors/edgarrmondragon>. (#942 <https://github.com/aio-libs/yarl/issues/942>)
A regression test for no-host URLs was added per #821 <https://github.com/aio-libs/yarl/issues/821>__
and 3986 -- by @kenballus <https://github.com/sponsors/kenballus>. (#821 <https://github.com/aio-libs/yarl/issues/821>, #822 <https://github.com/aio-libs/yarl/issues/822>__)
Started testing yarl against Python 3.12 in CI -- by @mjpieters <https://github.com/sponsors/mjpieters>. (#881 <https://github.com/aio-libs/yarl/issues/881>)
All Python 3.12 jobs are now marked as required to pass in CI
-- by @edgarrmondragon <https://github.com/sponsors/edgarrmondragon>. (#942 <https://github.com/aio-libs/yarl/issues/942>)
MyST is now integrated in Sphinx -- by @webknjaz <https://github.com/sponsors/webknjaz>__.
This allows the contributors to author new documents in Markdown
when they have difficulties with going straight RST. (#953 <https://github.com/aio-libs/yarl/issues/953>__)
yarl.URL.__truediv__() and absolute URLs with empty paths causing the raw path to lack the leading /.
(#854 <https://github.com/aio-libs/yarl/issues/854>_)#850 <https://github.com/aio-libs/yarl/issues/850>_)This release was never published to PyPI, due to issues with the build process.
URL.joinpath(*elements), to create a new URL appending multiple path elements. (#704 <https://github.com/aio-libs/yarl/issues/704>_)URL.__truediv__()() return NotImplemented if called with an
unsupported type — by @michaeljpeters <https://github.com/sponsors/michaeljpeters>__.
(#832 <https://github.com/aio-libs/yarl/issues/832>_).. segments would otherwise go beyond the URL path root.
(#536 <https://github.com/aio-libs/yarl/issues/536>_)#792 <https://github.com/aio-libs/yarl/issues/792>_)#793 <https://github.com/aio-libs/yarl/issues/793>_)yarl.URL.build() raise a TypeError if the host argument is None — by @paulpapacz <https://github.com/sponsors/paulpapacz>__. (#808 <https://github.com/aio-libs/yarl/issues/808>_)update_query() getting rid of the query when the argument
is empty but not None. (#845 <https://github.com/aio-libs/yarl/issues/845>_)#220 <https://github.com/aio-libs/yarl/issues/220>_This is the first release that started shipping wheels for Python 3.11.
#694 <https://github.com/aio-libs/yarl/issues/694>, #699 <https://github.com/aio-libs/yarl/issues/699>, #700 <https://github.com/aio-libs/yarl/issues/700>, #701 <https://github.com/aio-libs/yarl/issues/701>, #702 <https://github.com/aio-libs/yarl/issues/702>, #703 <https://github.com/aio-libs/yarl/issues/703>, #739 <https://github.com/aio-libs/yarl/issues/739>_URL.raw_suffix, URL.suffix, URL.raw_suffixes, URL.suffixes, URL.with_suffix. (#613 <https://github.com/aio-libs/yarl/issues/613>_)yarl.URL.human_repr().
(#665 <https://github.com/aio-libs/yarl/issues/665>_)multidict:index docs. (#665 <https://github.com/aio-libs/yarl/issues/665>_)#672 <https://github.com/aio-libs/yarl/issues/672>_)#646 <https://github.com/aio-libs/yarl/issues/646>, #699 <https://github.com/aio-libs/yarl/issues/699>, #701 <https://github.com/aio-libs/yarl/issues/701>_with_port() to stop reencoding parts of the URL that were already encoded. (#623 <https://github.com/aio-libs/yarl/issues/623>_)__bytes__() magic method so that bytes(url) will work and use optimal ASCII encoding.
(#582 <https://github.com/aio-libs/yarl/issues/582>_)#622 <https://github.com/aio-libs/yarl/issues/622>_)musl tag targeting typical Alpine Linux runtimes. (#622 <https://github.com/aio-libs/yarl/issues/622>_)#622 <https://github.com/aio-libs/yarl/issues/622>_)%e2%82%f8). All non-decodable percent-sequences are now preserved.
#517 <https://github.com/aio-libs/yarl/issues/517>_#535 <https://github.com/aio-libs/yarl/issues/535>_.c files in TarBall distribution.
#530 <https://github.com/aio-libs/multidict/issues/530>_aarch64, i686, ppc64le, s390x architectures on
Linux as well as x86_64.
#507 <https://github.com/aio-libs/yarl/issues/507>_#526 <https://github.com/aio-libs/yarl/issues/526>_human_repr() now always produces valid representation equivalent to the original URL (if the original URL is valid).
#511 <https://github.com/aio-libs/yarl/issues/511>_#514 <https://github.com/aio-libs/yarl/issues/514>_% which is not followed by two hexadecimal digits.
#516 <https://github.com/aio-libs/yarl/issues/516>_% followed by a space and hexadecimal digit.
#520 <https://github.com/aio-libs/yarl/issues/520>_with_query()/update_query() methods for key=[val1, val2] case.
#528 <https://github.com/aio-libs/yarl/issues/528>_#492 <https://github.com/aio-libs/yarl/issues/492>_URL.build(), with_xxx() and in / operator.
#502 <https://github.com/aio-libs/yarl/issues/502>_origin().
#504 <https://github.com/aio-libs/yarl/issues/504>_yarl._quoting_c C-extension into published PyPI dists.
#485 <https://github.com/aio-libs/yarl/issues/485>_#484 <https://github.com/aio-libs/yarl/issues/484>_Convert host to lowercase on URL building.
#386 <https://github.com/aio-libs/yarl/issues/386>_
Allow using mod operator (%) for updating query string (an alias for update_query() method).
#435 <https://github.com/aio-libs/yarl/issues/435>_
Allow use of sequences such as list and tuple in the values
of a mapping such as dict to represent that a key has many values::
url = URL("http://example.com")
assert url.with_query({"a": [1, 2]}) == URL("http://example.com/?a=1&a=2")
#443 <https://github.com/aio-libs/yarl/issues/443>_
Support URL.build() with scheme and path (creates a relative URL).
#464 <https://github.com/aio-libs/yarl/issues/464>_
Cache slow IDNA encode/decode calls.
#476 <https://github.com/aio-libs/yarl/issues/476>_
Add @final / Final type hints
#477 <https://github.com/aio-libs/yarl/issues/477>_
Support URL authority/raw_authority properties and authority argument of URL.build() method.
#478 <https://github.com/aio-libs/yarl/issues/478>_
Hide the library implementation details, make the exposed public list very clean.
#483 <https://github.com/aio-libs/yarl/issues/483>_
#409 <https://github.com/aio-libs/yarl/issues/409>_#426 <https://github.com/aio-libs/yarl/issues/426>_Query and QueryVariable type aliases in __init__.pyi, now they are prefixed with underscore.
#431 <https://github.com/aio-libs/yarl/issues/431>_#451 <https://github.com/aio-libs/yarl/issues/451>_str.isascii() in Python 3.6
#389 <https://github.com/aio-libs/yarl/issues/389>_Distinguish an empty password in URL from a password not provided at all (#262)
Fixed annotations for optional parameters of URL.build (#309)
Use None as default value of user parameter of URL.build (#309)
Enforce building C Accelerated modules when installing from source tarball, use
YARL_NO_EXTENSIONS environment variable for falling back to (slower) Pure Python
implementation (#329)
Drop Python 3.5 support
Fix quoting of plus in path by pure python version (#339)
Don't create a new URL if fragment is unchanged (#292)
Included in error message the path that produces starting slash forbidden error (#376)
Skip slow IDNA encoding for ASCII-only strings (#387)
Fix annotations for query parameter (#207)
An incoming query sequence can have int variables (the same as for Mapping type) (#208)
Add URL.explicit_port property (#218)
Give a friendlier error when port can't be converted to int (#168)
bool(URL()) now returns False (#272)
build (#199)cached_property (#195)str subclasses in URL constructor (#190)Forbid inheritance, replace __init__ with __new__ (#171)
Support PEP-561 (provide type hinting marker) (#182)
netloc (#170)Use fast path if quoted string does not need requoting (#154)
Speed up quoting/unquoting by _Quoter and _Unquoter classes (#155)
Drop yarl.quote and yarl.unquote public functions (#155)
Add custom string writer, reuse static buffer if available (#157) Code is 50-80 times faster than Pure Python version (was 4-5 times faster)
Don't recode IP zone (#144)
Support encoded=True in yarl.URL.build() (#158)
Fix updating query with multiple keys (#160)
TypeError by url.query_string() after
url.with_query({}) (empty mapping) (#141)raw_path_qs attribute (#137)strict parameter as no-op in quote / unquotestrict parameter as no-op for sake of compatibility with
aiohttp 2.2Drop strict mode (#123)
Fix "ValueError: Unallowed PCT %" when there's a "%" in the URL (#124)
Document encoded parameter (#102)
Support relative URLs like '?key=value' (#100)
Unsafe encoding for QS fixed. Encode ; character in value parameter (#104)
Process passwords without user names (#95)
Properly support paths without leading slash in URL.with_path() (#90)
Enable type annotation checks
Normalize path (#86)
Clear query and fragment parts in .with_path() (#85)
Unexpected compare behavior (#73)
Do not quote or unquote + if not a query string. (#74)
Added URL.build class method (#58)
Added path_qs attribute (#42)
: in pathLoad from pickle without _cache (#56)
Percent-encoded pluses in path variables become spaces (#59)
Allow to quote % in non strict mode (#21)
Incorrect parsing of query parameters with %3B (;) inside (#34)
Fix core dumps (#41)
tmpbuf - compiling error (#43)
Added URL.update_path() method
Added URL.update_query() method (#47)
quote / unquote.Support more verbose error messages in .with_query() (#24)
Don't percent-encode @ and : in path (#32)
Don't expose yarl.quote and yarl.unquote, these functions are
part of private API
str but all classes inherited from str also (#25)int as value for .with_query()setup.py (#20)typing.NamedTuple fields but indexes on URL construction_encode class methodquote() and unquote() as public API'/path?arg')relative() (#16)with_query() parameteris_default_port()Avoid doubling slashes when joining paths (#13)
Appending path starting from slash is forbidden (#12)
kwargs support for with_query() (#10)Document with_query(), with_fragment() and origin()
Allow None for with_query() and with_fragment()
FAQs
Yet another URL library
We found that yarl demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
At its inaugural meeting, the JSR Working Group outlined plans for an open governance model and a roadmap to enhance JavaScript package management.
Security News
Research
An advanced npm supply chain attack is leveraging Ethereum smart contracts for decentralized, persistent malware control, evading traditional defenses.
Security News
Research
Attackers are impersonating Sindre Sorhus on npm with a fake 'chalk-node' package containing a malicious backdoor to compromise developers' projects.