pymssql-linux

.. default-role:: code

.. image:: https://circleci.com/gh/noisycomputation/pymssql-linux/tree/master.svg?style=shield :target: https://circleci.com/gh/noisycomputation/pymssql-linux

The Original Pymssql Project Has Been Discontinued

This repository has been forked from upstream pymssql and modified to work with both Python 3.7 and 3.8, whilst continuing to build successfully for 2.7 for a short while, for sentimental but quite fundamental reasons.

There is no interest whatsoever in making this work in Windows.

Why Support Python 2 until its dying day?

Folks, upstream pymssql was taken out back and shot because Microsoft released native drivers into the open source. Using MS drivers on SQL Server 2005 requires version 11.0 or earlier of the MS driver, which is nearly impossible to compile on a modern *nix system due to an entire butterfly network of dependencies on long-deprecated system libraries. If you've ever tried to compile something non-trivial on Linux during the mid-90s, you'll know the pain of which I speak.

Why Not Support Windows?

See above. If Microsoft never honored us by gifting us a bunch of deprecated code that hadn't been maintained for a decade, the upstream pymssql would have been the only game in town and would have happily chugged along. But Microsoft did honor us, which led to the behind-the-courthouse death of pymssql, which led to the earthly hell of trying to compile Microsoft-gathered *nix code that relied on libraries from the early 2000s, et cetera, ad infinitum.

I have zero interest in figuring out how to build pymssql in a Windows CI. If someone wants to work with me to port pymssql to the Commodore 64, the Amiga 500, or Windows 2.1, let us waste no time. But forget Windows >= 3.1.

It must be childishly easy to get the official drivers working on a Windows box anyways.

Build Instructions

One of the reasons the upstream repository became such a tangle of inconsistent and outdated information is that it contained a large amount of build-related cruft that accumulated over the years. And that cruft was in addition to a README that contained build instructions that maybe worked two versions ago.

To avoid this fate, this repository will maintain only those CI configs that are being used to build, test, and deploy this package to PyPi. Currently, that universe consists of one CI, circleci, chosen only because it was the repository of the most valid build instructions at the time this repo was forked from upstream. This may change in the future.

Though this README may provide some additional context, the only set of build instructions that are guaranteed to be valid will be contained in the config file(s) for the current CI system being used. For the branch to which this README document pertains, the CI system is circleci and the build instructions are at .circleci/config.yml

Though referencing a CI configuration file is not as user-friendly as writing a soliloquy, it has the benefit of being verifiably accurate. If the circleci status badge at the top of this README indicates that the last build succeeded, then the circleci build instructions are valid.

Tests

The testing suite has been inherited from upstream. While it shows as passing in circleci, it actually fails with multiple errors, and the successful result in circleci appears to be the result of the tests being passed into circleci as shell scripts, whose error-ful exit codes are not being passed to circleci.

Bringing this project under test will be left to a future maintainer, hopefully a future maintainer of the upstream package. This maintainer tested the resulting packages in the maintainer's use case, and the packages have passed.

Yes, that is anecdotal.

Since the tests do not work, they have been disabled in .circleci/config.yml.

The remainder of this document is from the upstream repository and is likely outdated, especially as it pertains to build instructions. As my kinsfolk say, szerokiej drogi.

Original Pymssql Readme

To build pymssql-linux, you should have:

• python >= 3.7 including development files.
• Cython >= 0.29
• FreeTDS >= 0.95 including development files. Research your OS-specific distribution channels. (Archlinux: freetds, Debian: freetds-common, freetds-dev)
• gcc

To build, simply run python setup.py build in the project root directory.

It is possible to build a binary wheel package of pymssql-linux that pulls in and compiles a known-working version of FreeTDS. This option may become necessary if FreeTDS code evolves in a way that breaks compatibility with pymssql-linux, the development of which is, after all, frozen. Follow the binary wheel build instructions below in this case.

Details about the discontinuation of the original project and a discussion of alternatives to pymssql can be found at: https://github.com/pymssql/pymssql/issues/668

This fork is being maintained because pymssql works with older SQL Server versions that use deprecated TLS versions 1.0 and 1.1. Alternatives that utilize Microsoft's native SQL driver require the installation of version 11.0 of the driver, which is difficult to achieve cleanly due to multiple dependencies on deprecated library versions.

pymssql - DB-API interface to Microsoft SQL Server

A simple database interface for Python_ that builds on top of FreeTDS_ to provide a Python DB-API (PEP-249) interface to Microsoft SQL Server.

.. _Microsoft SQL Server: http://www.microsoft.com/sqlserver/ .. _Python: http://www.python.org/ .. _PEP-249: http://www.python.org/dev/peps/pep-0249/ .. _FreeTDS: http://www.freetds.org/

There is a Google Group for discussion at:

https://groups.google.com/forum/?fromgroups#!forum/pymssql

Building Binary Wheels

To build manylinux Python wheels, ensure you have docker and docker-compose installed, and run the following in the project root directory:

.. code-block:: bash

docker-compose up -d
docker exec pymssql-linux_x86_x64_1 ./io/dev/build_manylinux_wheels.sh
docker exec pymssql-linux_i686_1 ./io/dev/build_manylinux_wheels.sh
docker-compose down

To run unit tests, run the following before bringing the containers down:

.. code-block:: bash

docker exec pymssql-linux_x86_x64_1 ./io/dev/test_manylinux_wheels.sh
docker exec pymssql-linux_i686_1 ./io/dev/test_manylinux_wheels.sh

If the build suceeds, the dist directory in the project root will contain .whl files for Python versions >= 3.7. These can be installed by running pip install <filename.whl>.

.. default-role:: code

Recent Changes

Version 2.1.6 - 2020-06-05 (pymssql-linux)

• Fix circleci build script

• Update docker-compose.yml to manylinux2010 images

• Prepare package for upload to PyPi as pymssql-linux

Version 2.1.5 - 2019-11-27 (pymssql-linux)

• Add Python 3.8 support.

• Drop support for Python < 3.6

Version 3.0.3 - 2019-11-15

Fix messages that say "pymssql<=3.0" should be just "<'. Thanks Eric Moyer for reporting.

Versions 3.0.1 & 3.0.2 were released for the same reason, but with various readme typos that pypi now requires a new version release to fix.

Version 3.0 - 2019-11-15

Release "stub" version that errors during install to notify of project's discontinuation.

To install the last working released version, install with a version specifier like "pymmsql<3.0". E.g. pip install "pymssql<3.0"

For details and alternatives, see: https://github.com/pymssql/pymssql/issues/668

Version 2.1.4 - 2018-08-28

General

• Drop support for versions of FreeTDS older than 0.91.

• Add Python 3.7 support

• Drop Python 3.3 support

Features

• Support for new in SQL Server 2008 DATE, TIME and DATETIME2 data types (GH-156). The following conditions need to be additionally met so values of these column types can be returned from the database as their native corresponding Python data types instead of as strings:

  • Underlying FreeTDS must be 0.95 or newer.
  • TDS protocol version in use must be 7.3 or newer.

  Thanks Ed Avis for the implementation. (GH-331)

Bug fixes

• Fix tds_version _mssql connection property value for TDS version. 7.1 is actually 7.1 and not 8.0.

Version 2.1.3 - 2016-06-22 - Ramiro Morales

• We now publish Linux PEP 513 manylinux wheels on PyPI.

• Windows official binaries: Rollback changes to Windows binaries we had implemented in pymssql 2.1.2; go back to using:

  • A statically linked version of FreeTDS (v0.95.95)
  • No SSL support

Version 2.1.2 - 2016-02-10 - Ramiro Morales

.. attention:: Windows users: You need to download and install additional DLLs

pymssql version 2.1.2 includes a change in the official Windows binaries:
FreeTDS isn't statically linked as it happened up to release 2.1.1, as that
FreeTDS copy lacked SSL support.

Please see http://pymssql.org/en/latest/freetds.html#windows for futher
details.

We are trying to find a balance between security and convenience and will
be evaluating the situation for future releases. Your feedback is greatly
welcome.

Features

• Add ability to set TDS protocol version from pymssql when connecting to SQL Server. For the remaining pymssql 2.1.x releases its default value will be 7.1 (GH-323)

• Add Dockerfile and a Docker image and instructions on how to use it (GH-258). This could be a convenient way to use pymssql without having to build stuff. See http://pymssql.readthedocs.org/en/latest/intro.html#docker Thanks Marc Abramowitz.

• Floating point values are now accepted as Stored Procedure arguments (GH-287). Thanks Runzhou Li (Leo) for the report and Bill Adams for the implementation.

• Send pymssql version in the appname TDS protocol login record field when the application doesn't provide one (GH-354)

Bug fixes

• Fix a couple of very common causes of segmentation faults in presence of network a partition between a pymssql-based app and SQL Server (GH-147, GH-271) Thanks Marc Abramowitz. See also GH-373.

• Fix failures and inconsistencies in query parameter interpolation when UTF-8-encoded literals are present (GH-185). Thanks Bill Adams. Also, GH-291.

• Fix login_timeout parameter of pymssql.connect() (GH-318)

• Fixed some cases of cursor.rowcont having a -1 value after iterating over the value returned by pymssql cursor fetchmany() and fetchone() methods (GH-141)

• Remove automatic treatment of string literals passed in queries that start with '0x' as hexadecimal values (GH-286)

• Fix build fatal error when using Cython >= 0.22 (GH-311)

Internals

• Add Appveyor hosted CI setup for running tests on Windows (GH-347)

• Travis CI: Use newer, faster, container-based infrastructure. Also, test against more than one FreeTDS version.

• Make it possible to build official release files (sdist, wheels) on Travis & AppVeyor.

Version 2.1.1 - 2014-11-25 - Ramiro Morales

Features

• Custom message handlers (GH-139)

  The DB-Library API includes a callback mechanism so applications can provide functions known as message handlers that get passed informative messages sent by the server which then can be logged, shown to the user, etc.

  _mssql now allows you to install your own message handlers written in Python. See the _msssql examples and reference sections of the documentation for more details.

  Thanks Marc Abramowitz.

• Compatibility with Azure

  It is now possible to transparently connect to SQL Server instances_ accessible as part of the Azure_ cloud services.

  .. note:: If you need to connect to Azure make sure you use FreeTDS 0.91 or newer.

• Customizable per-connection initialization SQL clauses (both in pymssql and _mssql) (GH-97)

  It is now possible to customize the SQL statements sent right after the connection is established (e.g. 'SET ANSI_NULLS ON;'). Previously it was a hard-coded list of queries. See the _mssql.MSSQLConnection documentation for more details.

  Thanks Marc Abramowitz.

• Added ability to handle instances of uuid.UUID passed as parameters for SQL queries both in pymssql and _mssql. (GH-209)

  Thanks Marat Mavlyutov.

• Allow using SQL Server autocommit mode_ from pymssql at connection opening time. This allows e.g. DDL statements like DROP DATABASE to be executed. (GH-210)

  Thanks Marat Mavlyutov.

• Documentation: Explicitly mention minimum versions supported of Python (2.6) and SQL Server (2005).

• Incremental enhancements to the documentation.

.. _SQL Server instances: http://www.windowsazure.com/en-us/services/sql-database/ .. _Azure: https://www.windowsazure.com/ .. _SQL Server autocommit mode: http://msdn.microsoft.com/en-us/library/ms187878%28v=sql.105%29.aspx

Bug fixes

• Handle errors when calling Stored Procedures via the .callproc() pymssql cursor method. Now it will raise a DB-API DatabaseException; previously it allowed a _mssql.MSSQLDatabaseException exception to surface.

• Fixes in tds_version _mssql connections property value

  Made it work with TDS protocol version 7.2. (GH-211)

  The value returned for TDS version 7.1 is still 8.0 for backward compatibility (this is because such feature got added in times when Microsoft documentation labeled the two protocol versions that followed 7.0 as 8.0 and 9.0; later it changed them to 7.1 and 7.2 respectively) and will be corrected in a future release (2.2).

• PEP 249 compliance (GH-251)

  Added type constructors to increase compatibility with other libraries.

  Thanks Aymeric Augustin.

• pymssql: Made handling of integer SP params more robust (GH-237)

• Check lower bound value when convering integer values from to Python to SQL (GH-238)

Internals

• Completed migration of the test suite from nose to py.test.

• Added a few more test cases to our suite.

• Tests: Modified a couple of test cases so the full suite can be run against SQL Server 2005.

• Added testing of successful build of documentation to Travis CI script.

• Build process: Cleanup intermediate and ad-hoc anciliary files (GH-231, GH-273)

• setup.py: Fixed handling of release tarballs contents so no extraneous files are shipped and the documentation tree is actually included. Also, removed unused code.

Version 2.1.0 - 2014-02-25 - Marc Abramowitz <http://marc-abramowitz.com/>_

Features

• Sphinx-based documentation (GH-149)

  Read it online at http://pymssql.org/

  Thanks, Ramiro Morales!

  See:

  • https://github.com/pymssql/pymssql/pull/149
  • https://github.com/pymssql/pymssql/pull/162
  • https://github.com/pymssql/pymssql/pull/164
  • https://github.com/pymssql/pymssql/pull/165
  • https://github.com/pymssql/pymssql/pull/166
  • https://github.com/pymssql/pymssql/pull/167
  • https://github.com/pymssql/pymssql/pull/169
  • https://github.com/pymssql/pymssql/pull/174
  • https://github.com/pymssql/pymssql/pull/175

• "Green" support (GH-135)

  Lets you use pymssql with cooperative multi-tasking systems like gevent and have pymssql call a callback when it is waiting for a response from the server. You can set this callback to yield to another greenlet, coroutine, etc. For example, for gevent, you could do::

  def wait_callback(read_fileno):
      gevent.socket.wait_read(read_fileno)
  
  pymssql.set_wait_callback(wait_callback)
  

  The above is useful if you're say, running a gunicorn server with the gevent worker. With this callback in place, when you send a query to SQL server and are waiting for a response, you can yield to other greenlets and process other requests. This is super useful when you have high concurrency and/or slow database queries and lets you use less gunicorn worker processes and still handle high concurrency.

  See https://github.com/pymssql/pymssql/pull/135

• Better error messages.

  E.g.: For a connection failure, instead of:

  pymssql.OperationalError: (20009, 'Net-Lib error during Connection
  refused')
  

  the dberrstr is also included, resulting in:

  pymssql.OperationalError: (20009, 'DB-Lib error message 20009,
  severity 9:\nUnable to connect: Adaptive Server is unavailable or
  does not exist\nNet-Lib error during Connection refused\n')
  

  See:

  • https://github.com/pymssql/pymssql/pull/151

  In the area of error messages, we also made this change:

  execute: Raise ColumnsWithoutNamesError when as_dict=True and missing column names (GH-160)

  because the previous behavior was very confusing; instead of raising an exception, we would just return row dicts with those columns missing. This prompted at least one question on the mailing list (https://groups.google.com/forum/?fromgroups#!topic/pymssql/JoZpmNZFtxM), so we thought it was better to handle this explicitly by raising an exception, so the user would understand what went wrong.

  See:

  • https://github.com/pymssql/pymssql/pull/160
  • https://github.com/pymssql/pymssql/pull/168

• Performance improvements

  You are most likely to notice a difference from these when you are fetching a large number of rows.

  • Reworked row fetching (GH-159)

    There was a rather large amount of type conversion occuring when fetching a row from pymssql. The number of conversions required have been cut down significantly with these changes. Thanks Damien, Churchill!

    See:

    • https://github.com/pymssql/pymssql/pull/158
    • https://github.com/pymssql/pymssql/pull/159

  • Modify get_row() to use the CPython tuple API (GH-178)

    This drops the previous method of building up a row tuple and switches to using the CPython API, which allows you to create a correctly sized tuple at the beginning and simply fill it in. This appears to offer around a 10% boost when fetching rows from a table where the data is already in memory. Thanks Damien, Churchill!

    See:

    • https://github.com/pymssql/pymssql/pull/178

• MSSQLConnection: Add with (context manager) support (GH-171)

  This adds with statement support for MSSQLConnection in the _mssql module -- e.g.::

  with mssqlconn() as conn:
      conn.execute_query("SELECT @@version AS version")
  

  We already have with statement support for the pymssql module. See:

  • https://github.com/pymssql/pymssql/pull/171

• Allow passing in binary data (GH-179)

  Use the bytesarray type added in Python 2.6 to signify that this is binary data and to quote it accordingly. Also modify the handling of str/bytes types checking the first 2 characters for b'0x' and insert that as binary data. See:

  • https://github.com/pymssql/pymssql/pull/179

• Add support for binding uuid.UUID instances to stored procedures input params (GH-143) Thanks, Ramiro Morales!

  See:

  • https://github.com/pymssql/pymssql/pull/143
  • https://github.com/pymssql/pymssql/commit/1689c83878304f735eb38b1c63c31e210b028ea7

• The version number is now stored in one place, in pymssql_version.h This makes it easier to update the version number and not forget any places, like I did with pymssql 2.0.1

  • See https://github.com/pymssql/pymssql/commit/fd317df65fa62691c2af377e4661defb721b2699

• Improved support for using py.test as test runner (GH-183)

  • See: https://github.com/pymssql/pymssql/pull/183

• Improved PEP-8 and pylint compliance

Bug Fixes

• GH-142 ("Change how *.pyx files are included in package") - this should prevent pymssql.pyx and _mssql.pyx from getting copied into the root of your virtualenv. Thanks, @Arfrever!

  • See: https://github.com/pymssql/pymssql/issues/142

• GH-145 ("Prevent error string growing with repeated failed connection attempts.")

  See:

  • https://github.com/pymssql/pymssql/issues/145
  • https://github.com/pymssql/pymssql/pull/146

• GH-151 ("err_handler: Don't clobber dberrstr with oserrstr")

  • https://github.com/pymssql/pymssql/pull/151

• GH-152 ("mssql.pyx: Zero init global last_msg* vars") See: https://github.com/pymssql/pymssql/pull/152

• GH-177 ("binary columns sometimes are processed as varchar") Better mechanism for pymssql to detect that user is passing binary data.

  See: https://github.com/pymssql/pymssql/issues/177

• buffer overflow fix (GH-182)

  • See: https://github.com/pymssql/pymssql/pull/181
  • See: https://github.com/pymssql/pymssql/pull/182

• Return uniqueidentifer columns as uuid.UUID objects on Python 3

See ChangeLog_ for older history...

.. _PyPI: https://pypi.python.org/pypi/pymssql/2.0.0 .. _Travis CI: https://travis-ci.org/pymssql/pymssql .. _Cython: http://cython.org/ .. _ChangeLog: https://github.com/pymssql/pymssql/blob/master/ChangeLog