DB-API interface to Microsoft SQL Server for Python. (new Cython-based version)
.. default-role:: code
.. image:: https://circleci.com/gh/noisycomputation/pymssql-linux/tree/master.svg?style=shield :target: https://circleci.com/gh/noisycomputation/pymssql-linux
This repository has been forked from upstream pymssql and modified to work with both Python 3.7 and 3.8, whilst continuing to build successfully for 2.7 for a short while, for sentimental but quite fundamental reasons.
There is no interest whatsoever in making this work in Windows.
Folks, upstream pymssql was taken out back and shot because Microsoft released native drivers into the open source. Using MS drivers on SQL Server 2005 requires version 11.0 or earlier of the MS driver, which is nearly impossible to compile on a modern *nix system due to an entire butterfly network of dependencies on long-deprecated system libraries. If you've ever tried to compile something non-trivial on Linux during the mid-90s, you'll know the pain of which I speak.
See above. If Microsoft never honored us by gifting us a bunch of deprecated code that hadn't been maintained for a decade, the upstream pymssql would have been the only game in town and would have happily chugged along. But Microsoft did honor us, which led to the behind-the-courthouse death of pymssql, which led to the earthly hell of trying to compile Microsoft-gathered *nix code that relied on libraries from the early 2000s, et cetera, ad infinitum.
I have zero interest in figuring out how to build pymssql in a Windows CI. If someone wants to work with me to port pymssql to the Commodore 64, the Amiga 500, or Windows 2.1, let us waste no time. But forget Windows >= 3.1.
It must be childishly easy to get the official drivers working on a Windows box anyways.
One of the reasons the upstream repository became such a tangle of inconsistent and outdated information is that it contained a large amount of build-related cruft that accumulated over the years. And that cruft was in addition to a README that contained build instructions that maybe worked two versions ago.
To avoid this fate, this repository will maintain only those CI configs that are being used to build, test, and deploy this package to PyPi. Currently, that universe consists of one CI, circleci, chosen only because it was the repository of the most valid build instructions at the time this repo was forked from upstream. This may change in the future.
Though this README may provide some additional context,
the only set of build instructions that are guaranteed
to be valid will be contained in the config file(s) for
the current CI system being used. For the branch to which
this README document pertains, the CI system is circleci
and the build instructions are at .circleci/config.yml
Though referencing a CI configuration file is not as user-friendly as writing a soliloquy, it has the benefit of being verifiably accurate. If the circleci status badge at the top of this README indicates that the last build succeeded, then the circleci build instructions are valid.
The testing suite has been inherited from upstream. While it shows as passing in circleci, it actually fails with multiple errors, and the successful result in circleci appears to be the result of the tests being passed into circleci as shell scripts, whose error-ful exit codes are not being passed to circleci.
Bringing this project under test will be left to a future maintainer, hopefully a future maintainer of the upstream package. This maintainer tested the resulting packages in the maintainer's use case, and the packages have passed.
Yes, that is anecdotal.
Since the tests do not work, they have been disabled in
.circleci/config.yml.
The remainder of this document is from the upstream repository and is likely outdated, especially as it pertains to build instructions. As my kinsfolk say, szerokiej drogi.
To build pymssql-linux, you should have:
To build, simply run python setup.py build in the project
root directory.
It is possible to build a binary wheel package of pymssql-linux that pulls in and compiles a known-working version of FreeTDS. This option may become necessary if FreeTDS code evolves in a way that breaks compatibility with pymssql-linux, the development of which is, after all, frozen. Follow the binary wheel build instructions below in this case.
Details about the discontinuation of the original project and a discussion of alternatives to pymssql can be found at: https://github.com/pymssql/pymssql/issues/668
This fork is being maintained because pymssql works with older SQL Server versions that use deprecated TLS versions 1.0 and 1.1. Alternatives that utilize Microsoft's native SQL driver require the installation of version 11.0 of the driver, which is difficult to achieve cleanly due to multiple dependencies on deprecated library versions.
A simple database interface for Python_ that builds on top of FreeTDS_ to
provide a Python DB-API (PEP-249) interface to Microsoft SQL Server.
.. _Microsoft SQL Server: http://www.microsoft.com/sqlserver/ .. _Python: http://www.python.org/ .. _PEP-249: http://www.python.org/dev/peps/pep-0249/ .. _FreeTDS: http://www.freetds.org/
There is a Google Group for discussion at:
https://groups.google.com/forum/?fromgroups#!forum/pymssql
To build manylinux Python wheels, ensure you have docker and docker-compose installed, and run the following in the project root directory:
.. code-block:: bash
docker-compose up -d
docker exec pymssql-linux_x86_x64_1 ./io/dev/build_manylinux_wheels.sh
docker exec pymssql-linux_i686_1 ./io/dev/build_manylinux_wheels.sh
docker-compose down
To run unit tests, run the following before bringing the containers down:
.. code-block:: bash
docker exec pymssql-linux_x86_x64_1 ./io/dev/test_manylinux_wheels.sh
docker exec pymssql-linux_i686_1 ./io/dev/test_manylinux_wheels.sh
If the build suceeds, the dist directory in the project root will
contain .whl files for Python versions >= 3.7. These can be installed
by running pip install <filename.whl>.
.. default-role:: code
Fix circleci build script
Update docker-compose.yml to manylinux2010 images
Prepare package for upload to PyPi as pymssql-linux
Add Python 3.8 support.
Drop support for Python < 3.6
Fix messages that say "pymssql<=3.0" should be just "<'. Thanks Eric Moyer for reporting.
Versions 3.0.1 & 3.0.2 were released for the same reason, but with various readme typos that pypi now requires a new version release to fix.
Release "stub" version that errors during install to notify of project's discontinuation.
To install the last working released version, install with a version specifier like "pymmsql<3.0".
E.g. pip install "pymssql<3.0"
For details and alternatives, see: https://github.com/pymssql/pymssql/issues/668
Drop support for versions of FreeTDS older than 0.91.
Add Python 3.7 support
Drop Python 3.3 support
Support for new in SQL Server 2008 DATE, TIME and DATETIME2 data
types (GH-156). The following conditions need to be additionally met so
values of these column types can be returned from the database as their
native corresponding Python data types instead of as strings:
Thanks Ed Avis for the implementation. (GH-331)
tds_version _mssql connection property value for TDS version.
7.1 is actually 7.1 and not 8.0.We now publish Linux PEP 513 manylinux wheels on PyPI.
Windows official binaries: Rollback changes to Windows binaries we had implemented in pymssql 2.1.2; go back to using:
.. attention:: Windows users: You need to download and install additional DLLs
pymssql version 2.1.2 includes a change in the official Windows binaries:
FreeTDS isn't statically linked as it happened up to release 2.1.1, as that
FreeTDS copy lacked SSL support.
Please see http://pymssql.org/en/latest/freetds.html#windows for futher
details.
We are trying to find a balance between security and convenience and will
be evaluating the situation for future releases. Your feedback is greatly
welcome.
Add ability to set TDS protocol version from pymssql when connecting to SQL Server. For the remaining pymssql 2.1.x releases its default value will be 7.1 (GH-323)
Add Dockerfile and a Docker image and instructions on how to use it (GH-258). This could be a convenient way to use pymssql without having to build stuff. See http://pymssql.readthedocs.org/en/latest/intro.html#docker Thanks Marc Abramowitz.
Floating point values are now accepted as Stored Procedure arguments (GH-287). Thanks Runzhou Li (Leo) for the report and Bill Adams for the implementation.
Send pymssql version in the appname TDS protocol login record field when the application doesn't provide one (GH-354)
Fix a couple of very common causes of segmentation faults in presence of network a partition between a pymssql-based app and SQL Server (GH-147, GH-271) Thanks Marc Abramowitz. See also GH-373.
Fix failures and inconsistencies in query parameter interpolation when UTF-8-encoded literals are present (GH-185). Thanks Bill Adams. Also, GH-291.
Fix login_timeout parameter of pymssql.connect() (GH-318)
Fixed some cases of cursor.rowcont having a -1 value after iterating
over the value returned by pymssql cursor fetchmany() and fetchone()
methods (GH-141)
Remove automatic treatment of string literals passed in queries that start
with '0x' as hexadecimal values (GH-286)
Fix build fatal error when using Cython >= 0.22 (GH-311)
Add Appveyor hosted CI setup for running tests on Windows (GH-347)
Travis CI: Use newer, faster, container-based infrastructure. Also, test against more than one FreeTDS version.
Make it possible to build official release files (sdist, wheels) on Travis & AppVeyor.
Custom message handlers (GH-139)
The DB-Library API includes a callback mechanism so applications can provide functions known as message handlers that get passed informative messages sent by the server which then can be logged, shown to the user, etc.
_mssql now allows you to install your own message handlers written in
Python. See the _msssql examples and reference sections of the
documentation for more details.
Thanks Marc Abramowitz.
Compatibility with Azure
It is now possible to transparently connect to SQL Server instances_
accessible as part of the Azure_ cloud services.
.. note:: If you need to connect to Azure make sure you use FreeTDS 0.91 or newer.
Customizable per-connection initialization SQL clauses (both in pymssql
and _mssql) (GH-97)
It is now possible to customize the SQL statements sent right after the
connection is established (e.g. 'SET ANSI_NULLS ON;'). Previously
it was a hard-coded list of queries. See the _mssql.MSSQLConnection
documentation for more details.
Thanks Marc Abramowitz.
Added ability to handle instances of uuid.UUID passed as parameters for
SQL queries both in pymssql and _mssql. (GH-209)
Thanks Marat Mavlyutov.
Allow using SQL Server autocommit mode_ from pymssql at connection
opening time. This allows e.g. DDL statements like DROP DATABASE to be
executed. (GH-210)
Thanks Marat Mavlyutov.
Documentation: Explicitly mention minimum versions supported of Python (2.6) and SQL Server (2005).
Incremental enhancements to the documentation.
.. _SQL Server instances: http://www.windowsazure.com/en-us/services/sql-database/ .. _Azure: https://www.windowsazure.com/ .. _SQL Server autocommit mode: http://msdn.microsoft.com/en-us/library/ms187878%28v=sql.105%29.aspx
Handle errors when calling Stored Procedures via the .callproc() pymssql
cursor method. Now it will raise a DB-API DatabaseException; previously
it allowed a _mssql.MSSQLDatabaseException exception to surface.
Fixes in tds_version _mssql connections property value
Made it work with TDS protocol version 7.2. (GH-211)
The value returned for TDS version 7.1 is still 8.0 for backward compatibility (this is because such feature got added in times when Microsoft documentation labeled the two protocol versions that followed 7.0 as 8.0 and 9.0; later it changed them to 7.1 and 7.2 respectively) and will be corrected in a future release (2.2).
PEP 249 compliance (GH-251)
Added type constructors to increase compatibility with other libraries.
Thanks Aymeric Augustin.
pymssql: Made handling of integer SP params more robust (GH-237)
Check lower bound value when convering integer values from to Python to SQL (GH-238)
Completed migration of the test suite from nose to py.test.
Added a few more test cases to our suite.
Tests: Modified a couple of test cases so the full suite can be run against SQL Server 2005.
Added testing of successful build of documentation to Travis CI script.
Build process: Cleanup intermediate and ad-hoc anciliary files (GH-231, GH-273)
setup.py: Fixed handling of release tarballs contents so no extraneous files are shipped and the documentation tree is actually included. Also, removed unused code.
Marc Abramowitz <http://marc-abramowitz.com/>_Sphinx-based documentation (GH-149)
Read it online at http://pymssql.org/
Thanks, Ramiro Morales!
See:
"Green" support (GH-135)
Lets you use pymssql with cooperative multi-tasking systems like gevent and have pymssql call a callback when it is waiting for a response from the server. You can set this callback to yield to another greenlet, coroutine, etc. For example, for gevent, you could do::
def wait_callback(read_fileno):
gevent.socket.wait_read(read_fileno)
pymssql.set_wait_callback(wait_callback)
The above is useful if you're say, running a gunicorn server with the gevent worker. With this callback in place, when you send a query to SQL server and are waiting for a response, you can yield to other greenlets and process other requests. This is super useful when you have high concurrency and/or slow database queries and lets you use less gunicorn worker processes and still handle high concurrency.
Better error messages.
E.g.: For a connection failure, instead of:
pymssql.OperationalError: (20009, 'Net-Lib error during Connection
refused')
the dberrstr is also included, resulting in:
pymssql.OperationalError: (20009, 'DB-Lib error message 20009,
severity 9:\nUnable to connect: Adaptive Server is unavailable or
does not exist\nNet-Lib error during Connection refused\n')
See:
In the area of error messages, we also made this change:
execute: Raise ColumnsWithoutNamesError when as_dict=True and missing column names (GH-160)
because the previous behavior was very confusing; instead of raising an exception, we would just return row dicts with those columns missing. This prompted at least one question on the mailing list (https://groups.google.com/forum/?fromgroups#!topic/pymssql/JoZpmNZFtxM), so we thought it was better to handle this explicitly by raising an exception, so the user would understand what went wrong.
See:
Performance improvements
You are most likely to notice a difference from these when you are fetching a large number of rows.
Reworked row fetching (GH-159)
There was a rather large amount of type conversion occuring when fetching a row from pymssql. The number of conversions required have been cut down significantly with these changes. Thanks Damien, Churchill!
See:
Modify get_row() to use the CPython tuple API (GH-178)
This drops the previous method of building up a row tuple and switches to using the CPython API, which allows you to create a correctly sized tuple at the beginning and simply fill it in. This appears to offer around a 10% boost when fetching rows from a table where the data is already in memory. Thanks Damien, Churchill!
See:
MSSQLConnection: Add with (context manager) support (GH-171)
This adds with statement support for MSSQLConnection in the _mssql
module -- e.g.::
with mssqlconn() as conn:
conn.execute_query("SELECT @@version AS version")
We already have with statement support for the pymssql module.
See:
Allow passing in binary data (GH-179)
Use the bytesarray type added in Python 2.6 to signify that this is binary data and to quote it accordingly. Also modify the handling of str/bytes types checking the first 2 characters for b'0x' and insert that as binary data. See:
Add support for binding uuid.UUID instances to stored procedures input params (GH-143) Thanks, Ramiro Morales!
See:
The version number is now stored in one place, in pymssql_version.h This makes it easier to update the version number and not forget any places, like I did with pymssql 2.0.1
Improved support for using py.test as test runner (GH-183)
Improved PEP-8 and pylint compliance
GH-142 ("Change how *.pyx files are included in package") - this
should prevent pymssql.pyx and _mssql.pyx from getting copied into the
root of your virtualenv. Thanks, @Arfrever!
GH-145 ("Prevent error string growing with repeated failed connection attempts.")
See:
GH-151 ("err_handler: Don't clobber dberrstr with oserrstr")
GH-152 ("mssql.pyx: Zero init global last_msg* vars") See: https://github.com/pymssql/pymssql/pull/152
GH-177 ("binary columns sometimes are processed as varchar") Better mechanism for pymssql to detect that user is passing binary data.
buffer overflow fix (GH-182)
Return uniqueidentifer columns as uuid.UUID objects on Python 3
See ChangeLog_ for older history...
.. _PyPI: https://pypi.python.org/pypi/pymssql/2.0.0 .. _Travis CI: https://travis-ci.org/pymssql/pymssql .. _Cython: http://cython.org/ .. _ChangeLog: https://github.com/pymssql/pymssql/blob/master/ChangeLog
FAQs
DB-API interface to Microsoft SQL Server for Python. (new Cython-based version)
We found that pymssql-linux demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.
Security News
Research
Socket researchers found a malicious Maven package impersonating the legitimate ‘XZ for Java’ library, introducing a backdoor for remote code execution.