django-pev

Django Postgres Explain Visualizer (Django-PEV)

This tool captures sql queries and uploads the query plan to postgresql explain visualizer (PEV) by dalibo. This is especially helpful for debugging slow queries.

This tool also exports a graphical UI similar to pghero but is embedded within your django app.

Installation

• pip install django-pev

• Add to your urls

# urls.py
from django.urls import include, path

urlpatterns = [
    # ....

    path('django-pev/', include(('django_pev.urls', 'django_pev'), namespace='django_pev')),
]

• Add to your installed apps

# settings.py

INSTALLED_APPS = [
    # ...
    "django_pev"
]

Usage

Wrap some code with the explain context manager. All sql queries are captured alongside a stacktrace (to locate where it was called). The slowest query is accessible via .slowest.

import django_pev
from django.contrib.auth.models import User

with django_pev.explain() as e:
    # Every SQL query is captured
    list(User.objects.filter(email='[e-mail]').all())

# Rerun the slowest query with `EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS, FORMAT JSON)`
pev_response = e.slowest.visualize(
    # By default the text of the query is not uploaded for security reasons
    upload_query=True,
    # Set to false if the query is slow and you want only an explain
    analyze=True,
    # Give a helpful title for the uploaded query plan
    title="Measuring email filter",
)
print(pev_response.url)

# View the postgres explain visualization
e.slowest.visualize_in_browser()


# Print the optimization prompt
e.slowest.optimization_prompt()

# Find N+1 queries
for query, count in e.nplusones.items():
    print(f"Found N+1 query executed {count} times:")
    print(query.sql)
    print(f"Stack trace:\n{query.stack_trace}")

# View the stack trace of the slowest query
print(e.slowest.stacktrace)

# Delete the plan hosted on https://explain.dalibo.com
pev_response.delete()

Optionally configure additional settings:

# Replace the default test client used during explain with a custom class
DJANGO_PEV_EXPLAIN_TEST_CLIENT = 'django.test.Client'

How to debug a slow endpoint in production

If you have access to python manage.py shell on the production server; you can run the following code snippet to get an explain plan uploaded. In general this technique is all types of profiling.

import django_pev

from django.contrib.auth.models import User
from django.test import Client as TestClient

client = TestClient()
# Authentication
client.force_login(User.objects.get(id=1))
url = "/some_slow_url"

with django_pev.explain() as e:
    response = client.get(url)

print(e.slowest.visualize(title=f"Fetching {url}"))

TODO

• Add migration to ensure pg_stats_statement_info is correct
• Do not crash when its not available
• Add explain tab
• Add index suggester

Disclaimer

Credit goes to Pierre Giraud (@pgiraud) for PEV2 and Alex Tatiyants (@AlexTatiyants) for the original pev tool.

IN NO EVENT SHALL DALIBO BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF DALIBO HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

DALIBO SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND DALIBO HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.