edx-arch-experiments

edx-arch-experiments

|pypi-badge| |ci-badge| |codecov-badge| |pyversions-badge| |license-badge|

A plugin to include applications under development by and useful utility scripts for the architecture team at 2U.

Overview

This plugin is meant to house experimental and in-development applications from the edX architecture team at 2U that are either not appropriate (i.e. 2U-specific) or not yet ready for community consumption. It also includes some one-off scripts meant to reduce toil for the team.

Development Workflow

One Time Setup

.. code-block::

  # Clone the repository
  git clone [e-mail]:edx/edx-arch-experiments.git
  cd edx-arch-experiments

  # Set up a virtualenv using virtualenvwrapper with the same name as the repo and activate it
  mkvirtualenv -p python3.11 edx-arch-experiments

Local testing
~~~~~~~~~~~~~
Two options are available for testing changes locally:

First, via `make test-shell` a dockerized development envrionment can be launched to run `pytest` and do basic migration work.

Second, a local copy of the package can be installed into edx-platform. For example, if using devstack, copy or clone your branch into <devstack-parent>/src/edx-arch-experiments. Then, in an lms or cms shell, run ``pip install -e /edx/src/edx-arch-experiments``.  The plug-in configuration will automatically be picked up once installed, and changes will be hot reloaded.


Every time you develop something in this repo

.. code-block::

Activate the virtualenv

workon edx-arch-experiments

Grab the latest code

git checkout main git pull

Install/update the dev requirements

make requirements

Run the tests and quality checks (to verify the status before you make any changes)

make validate

Make a new branch for your changes

git checkout -b <your_github_username>/<short_description>

Using your favorite editor, edit the code to make your change.

vim …

Run your new tests

pytest ./path/to/new/tests

Run all the tests and quality checks

make validate

Commit all your changes

git commit … git push

Open a PR and ask for review.

License

The code in this repository is licensed under the AGPL 3.0 unless otherwise noted.

Please see LICENSE.txt <LICENSE.txt>_ for details.

How To Contribute

Contributions are very welcome. Please read How To Contribute <https://github.com/edx/edx-platform/blob/master/CONTRIBUTING.rst>_ for details. Even though they were written with edx-platform in mind, the guidelines should be followed for all Open edX projects.

The pull request description template should be automatically applied if you are creating a pull request from GitHub. Otherwise you can find it at PULL_REQUEST_TEMPLATE.md <.github/PULL_REQUEST_TEMPLATE.md>_.

The issue report template should be automatically applied if you are creating an issue on GitHub as well. Otherwise you can find it at ISSUE_TEMPLATE.md <.github/ISSUE_TEMPLATE.md>_.

Reporting Security Issues

Please do not report security issues in public. Please email [e-mail].

Getting Help

If you're having trouble, we have discussion forums at https://discuss.openedx.org where you can connect with others in the community.

Our real-time conversations are on Slack. You can request a Slack invitation, then join our community Slack workspace.

For more information about these options, see the Getting Help_ page.

.. _Slack invitation: https://openedx-slack-invite.herokuapp.com/ .. _community Slack workspace: https://openedx.slack.com/ .. _Getting Help: https://openedx.org/getting-help

.. |pypi-badge| image:: https://img.shields.io/pypi/v/edx-arch-experiments.svg :target: https://pypi.python.org/pypi/edx-arch-experiments/ :alt: PyPI

.. |ci-badge| image:: https://github.com/edx/edx-arch-experiments/workflows/Python%20CI/badge.svg?branch=main :target: https://github.com/edx/edx-arch-experiments/actions :alt: CI

.. |codecov-badge| image:: https://codecov.io/github/edx/edx-arch-experiments/coverage.svg?branch=main :target: https://codecov.io/github/edx/edx-arch-experiments?branch=main :alt: Codecov

.. |pyversions-badge| image:: https://img.shields.io/pypi/pyversions/edx-arch-experiments.svg :target: https://pypi.python.org/pypi/edx-arch-experiments/ :alt: Supported Python versions

.. |license-badge| image:: https://img.shields.io/github/license/edx/edx-arch-experiments.svg :target: https://github.com/edx/edx-arch-experiments/blob/main/LICENSE.txt :alt: License

Change Log

.. All enhancements and patches to edx_arch_experiments will be documented in this file. It adheres to the structure of https://keepachangelog.com/ , but in reStructuredText instead of Markdown (for ease of incorporation into Sphinx documentation and the PyPI description).

This project adheres to Semantic Versioning (https://semver.org/).

.. There should always be an "Unreleased" section for changes pending release.

Unreleased

[6.1.0] - 2024-12-10

Changed

---

• Completes code owner monitoring updates, which drops owner theme and finalizes the code owner span tags. See doc and ADR updates for more details.

  • The code_owner_theme_2 tag was dropped altogether.
  • The temporary suffix (_2) was removed from other span tags.
  • The code_owner (formerly code_owner_2) tag no longer includes the theme name.
  • The new name for the django setting is CODE_OWNER_TO_PATH_MAPPINGS (formerly CODE_OWNER_MAPPINGS).
  • The django setting CODE_OWNER_THEMES was dropped.
  • Updates the generate_code_owner_mappings.py script accordingly.

[6.0.0] - 2024-12-05

Removed
-------
- Removes CodeOwnerMonitoringMiddleware, in favor of using new signals sent from edx-django-utils's MonitoringSupportMiddleware.

Added
-----
* Adds search script datadog_search.py, for searching Datadog monitors and dashboards.

[5.1.0] - 2024-11-21

Added

• Added Datadog monitoring app which adds code owner monitoring. This is the first step in moving code owner code from edx-django-utils to this plugin.

  • Adds near duplicate of code owner middleware from edx-django-utils.
  • Adds code owner span tags for celery using Datadog span processing of celery.run spans.
  • Uses temporary span tags names using _2, like code_owner_2, for rollout and comparison with the original span tags.
  • Span tag code_owner_2_module includes the task name, where the original code_owner_module does not. In both cases, the code owner is computed the same, because it is based on a prefix match.

[5.0.0] - 2024-10-22

Removed
-------
* Deleted Datadog diagnostics plugin app and middleware, which are no longer in use in edxapp.

[4.5.0] - 2024-09-19

Added

• Datadog diagnostics middleware can now attempt to close anomalous spans. Can be enabled via Waffle flag datadog.diagnostics.close_anomalous_spans (controlled separately from logging feature).

[4.4.0] - 2024-09-10

Changed
-------
* Datadog diagnostics now logs ancestor spans when an anomaly is encountered, up to a limit of 10 (controlled by new Django setting ``DATADOG_DIAGNOSTICS_LOG_SPAN_DEPTH``). Spans are logged in full and on separate lines, so this logging is now much more verbose; consider only enabling this logging for short periods. Log format of first line has also changed slightly.

[4.3.0] - 2024-08-22

Added

• Added celery lifecycle logging for Datadog diagnostics, to be enabled using DATADOG_DIAGNOSTICS_CELERY_LOG_SIGNALS.

[4.2.0] - 2024-08-13

Fixed
-----
* Fixed loading of ``DATADOG_DIAGNOSTICS_ENABLE``, which was previously not loaded properly and therefore was always True. Also fixed loading of ``DATADOG_DIAGNOSTICS_MAX_SPANS``, which was presumably broken as well.

Removed
-------
* Removed early span-start logging. It never worked properly, possibly because workers are continually being destroyed and created, leading to high log volume.

[4.1.0] - 2024-08-09

Changed

• Datadog diagnostics will now log all span-starts for the first minute after server startup
• WARNING: Do not use this version; see 4.2.0 release notes.

[4.0.0] - 2024-08-05

Changed
-------
* Dropped support for Python 3.8; only testing with 3.11 and above now.

Added
-----
* ``DatadogDiagnosticMiddleware`` can now detect and log anomalous traces, enabled by Waffle flag ``datadog.diagnostics.detect_anomalous_trace``

[3.6.0] - 2024-07-24

Added

• New middleware edx_arch_experiments.datadog_diagnostics.middleware.DatadogDiagnosticMiddleware for logging diagnostics on traces in Datadog.

[3.5.1] - 2024-07-15

Changed
-------
* Added ``federated-content-connector`` to the generate_code_owners script.

[3.5.0] - 2024-07-11

Added

• Toggle DATADOG_DIAGNOSTICS_ENABLE for disabling that plugin quickly if needed. (Feature remains enabled by default.)

Fixed

• Limit the number of spans collected via new setting DATADOG_DIAGNOSTICS_MAX_SPANS, defaulting to 100. This may help avoid memory leaks.
• Make accidental class variables into member variables in datadog_diagnostics

[3.4.0] - 2024-07-10

Added
-----
* Added ``datadog_diagnostics`` plugin app

[3.3.2] - 2024-04-19

Changed

• Added translatable-xblocks to the generate_code_owners script.

[3.3.1] - 2024-02-26

Added
_______
* Added support for ``Python 3.12``

[3.3.0] - 2024-01-23

Changed

---

• Updated ConfigWatcher to include the IDA's name in change messages if CONFIG_WATCHER_SERVICE_NAME is set
• Enabled ConfigWatcher as a plugin for CMS

[3.2.0] - 2024-01-11

Added
_____
* Add ``codejail_service`` app for transition to containerized codejail

[3.1.1] - 2023-11-06

Fixed

---

• ConfigWatcher should now respond to model events properly now that it registers receivers with strong references. (Tested in sandbox.)

[3.1.0] - 2023-10-31

Changed
_______

* Add log message for each model the ConfigWatcher is listening to
* Ensure that ConfigWatcher only attaches receivers once

[3.0.0] - 2023-10-30

Changed

---

• Renamed ConfigWatcherApp to ConfigWatcher to be less redundant. This is technically a breaking change but the app was not in use yet.
• Enabled ConfigWatcher as a plugin for LMS

[2.2.0] - 2023-10-27

Added
_____

* Add ``edx_arch_experiments.config_watcher`` Django app for monitoring Waffle changes
* Add script to get github action errors
* Add script to republish failed events

[2.1.0] - 2023-10-10

• Add ORA2 to our code owner mapping script.

[2.0.0] - 2023-06-01

* Removes summary hook aside, now in the ai-aside repo

[1.2.0] - 2023-05-08

• Update summary hook to trigger on videos
• Remove text selection data key from summary hook html

[1.1.4] - 2023-04-14

* Add course and block ID to summary hook html

[1.1.3] - 2023-04-05

Fixed

---

• Removed default_app_config (deprecated in Django 3)

[1.1.2] - 2023-03-14

* Add "staff only" summary hook flag

[1.1.1] - 2023-03-09

• Revise summary hook HTML

[1.1.0] - 2023-03-08

* Add summary hook xblock aside

[1.0.0] - 2022-10-06

• Breaking change: Remove kafka_consumer package and plugin (migrated to edx-event-bus-kafka)

[0.2.1] - 2022-06-14

* Add new target to Makefile
* Update openedx-events

[0.2.0] - 2022-03-16

• Update consumer to use bridge and signals

[0.1.1] - 2022-03-16

* Fix GitHub actions

[0.1.0] - 2022-02-22

Added

---

• First release on PyPI.