Skip to content

Commit 905645d

Browse files
sarinaclaude
sarina
and
claude
authored
Add docs on db technologies and postgresql support (#1421)
Co-authored-by: Claude <noreply@anthropic.com>
1 parent c311b49 commit 905645d

3 files changed

Lines changed: 224 additions & 0 deletions

File tree

‎source/site_ops/concepts/databases.rst‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
.. _Open edX Databases:
2+
3+
Open edX Platform Database and Search Technologies
4+
##################################################
5+
6+
.. tags:: site operator, concept
7+
8+
The Open edX platform uses several databases and search backends, each
9+
serving different purposes.
10+
11+
Relational (SQL)
12+
****************
13+
14+
**MySQL**
15+
The long-standing primary relational database for most services (LMS,
16+
CMS/Studio), storing users, courses, enrollments, and grades.
17+
18+
**PostgreSQL**
19+
Newly supported as of the Verawood release, with migrations updated to
20+
make the Open edX platform compatible across both MySQL and PostgreSQL.
21+
Changes include dynamic SQL generation based on the database engine,
22+
cross-database field type handling, and custom psycopg2 adapter
23+
registration for Open edX's OpaqueKey types like ``CourseLocator``. A
24+
community-developed Tutor plugin (`tutor-contrib-postgresql <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_)
25+
supports PostgreSQL-based Tutor deployments. See :ref:`use postgresql`.
26+
27+
**SQLite**
28+
Used in local development and testing environments.
29+
30+
NoSQL / Document
31+
****************
32+
33+
**MongoDB**
34+
Used to store course content structure in Studio. There is ongoing work to
35+
migrate away from MongoDB toward MySQL via the Learning Core
36+
(`openedx-learning <https://github.com/openedx/openedx-core>`_) initiative.
37+
38+
Caching / In-Memory
39+
*******************
40+
41+
**Redis**
42+
Used for caching, session storage, and Celery task queuing. Superseded
43+
Memcached in modern deployments.
44+
45+
**Memcached**
46+
Legacy caching layer, largely replaced by Redis.
47+
48+
Search
49+
******
50+
51+
**Meilisearch**
52+
The current primary search engine for content library and course
53+
authoring search (introduced in Quince), used by the Authoring MFE for
54+
course and library content discovery, as well as the Catalog MFE for course searching.
55+
56+
**Typesense**
57+
A supported alternative search backend for site operators, attractive in
58+
part because it supports raft-based clustering for high availability — a
59+
limitation that has been noted with Meilisearch. See :ref:`Use Typesense search backend`.
60+
61+
.. note:: Typesense is currently (as of Verawood) only supported for LMS searching, _not_ Studio/libraries.
62+
63+
**Elasticsearch / OpenSearch**
64+
Used for older LMS-level search (course catalog, user search). OpenSearch
65+
is the current preferred variant, with ongoing community discussion about
66+
whether to deprecate it in favor of lighter-weight alternatives.
67+
68+
Message Broker
69+
**************
70+
71+
**Redis** (also serves this role) **or RabbitMQ**
72+
Used as a Celery broker for background task processing.
73+
74+
.. seealso::
75+
76+
:ref:`Use Typesense search backend`
77+
78+
:ref:`use postgresql`
79+
80+
**Maintenance chart**
81+
82+
+--------------+-------------------------------+----------------+--------------------------------+
83+
| Review Date | Working Group Reviewer | Release |Test situation |
84+
+--------------+-------------------------------+----------------+--------------------------------+
85+
| 2025-04-10 | sarina | Verawood | Pass |
86+
+--------------+-------------------------------+----------------+--------------------------------+

‎source/site_ops/how-tos/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ Site Operators: How-tos
44
.. tags:: site operator, how-to
55

66
.. toctree::
7+
:maxdepth: 1
78
:glob:
89

910
*

‎source/site_ops/how-tos/use_postgresql.rst‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
.. _use postgresql:
2+
3+
Use PostgreSQL as the Relational Database Backend
4+
#################################################
5+
6+
.. note::
7+
8+
PostgreSQL support was introduced in the Verawood release. The Tutor plugin
9+
that enables this feature (`tutor-contrib-postgresql <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_)
10+
is under active development. Migration from an existing MySQL installation is
11+
not yet supported. See `Current PostgreSQL Limitations`_ for details.
12+
13+
PostgreSQL is a powerful open source relational database system with a strong
14+
reputation for reliability, standards compliance, and extensibility. It is a
15+
common choice for production web applications and is widely supported by
16+
managed cloud database services such as Amazon RDS, Google Cloud SQL, and
17+
Azure Database for PostgreSQL.
18+
19+
Historically, the Open edX platform used MySQL as its only supported relational
20+
database. As of the Verawood release, PostgreSQL is also supported as a
21+
relational database backend for the Open edX platform, thanks to community
22+
contributions that updated the platform's Django migrations and field adapters
23+
to work correctly across both database engines.
24+
25+
Prerequisites
26+
*************
27+
28+
- A Tutor-based Open edX deployment (local or Kubernetes)
29+
- Tutor Verawood or later
30+
- A fresh Open edX installation (see `Current PostgreSQL Limitations`_)
31+
32+
Installation
33+
************
34+
35+
PostgreSQL support is provided via the community-maintained Tutor plugin
36+
``tutor-contrib-postgresql``, authored by `@qasimgulzar
37+
<https://github.com/qasimgulzar>`_.
38+
39+
.. warning::
40+
41+
This plugin is under active development and has not yet reached a stable
42+
release. Use in production environments is at your own risk. Review the
43+
`plugin repository <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_
44+
for the latest status before proceeding.
45+
46+
Step 1: Install the plugin
47+
==========================
48+
49+
.. code-block:: bash
50+
51+
pip install git+https://github.com/qasimgulzar/tutor-contrib-postgresql
52+
53+
Step 2: Enable the plugin
54+
=========================
55+
56+
.. code-block:: bash
57+
58+
tutor plugins enable postgresql
59+
60+
Step 3: Build the Open edX image
61+
=================================
62+
63+
The Open edX image must be rebuilt to include the necessary PostgreSQL client
64+
libraries and Python packages (``psycopg2``).
65+
66+
.. code-block:: bash
67+
68+
tutor images build openedx
69+
70+
Step 4: Launch the platform
71+
============================
72+
73+
.. code-block:: bash
74+
75+
tutor local launch
76+
77+
This will initialize a fresh Open edX database in PostgreSQL and run all
78+
Django migrations against it.
79+
80+
.. _Current PostgreSQL Limitations:
81+
82+
Current Limitations
83+
*******************
84+
85+
Fresh installations only
86+
========================
87+
88+
At this time, PostgreSQL support is available for fresh Open edX installations
89+
only. There is no supported or documented migration path from an existing MySQL
90+
database to PostgreSQL. Operators with an existing MySQL-backed deployment who
91+
wish to switch to PostgreSQL will need to wait for migration tooling and
92+
documentation to be developed.
93+
94+
If you are interested in contributing to or tracking the development of
95+
MySQL-to-PostgreSQL migration support, see the `tutor-contrib-postgresql
96+
repository <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_ and the
97+
`original platform pull request
98+
<https://github.com/openedx/openedx-platform/pull/35762>`_.
99+
100+
Known rough edges
101+
=================
102+
103+
PostgreSQL support was introduced with the following known limitations that may
104+
be addressed in subsequent releases.
105+
106+
- Opaque key adapter coverage for v2 content libraries (beta libraries in
107+
Studio) may be incomplete. Runtime errors related to ``LearningContextKey``
108+
or ``UsageKey`` serialization, if encountered, should be reported to the
109+
plugin maintainer.
110+
- The ``tutor-contrib-postgresql`` plugin does not yet have a stable release.
111+
Install directly from the GitHub repository as shown above.
112+
113+
Getting Help
114+
************
115+
116+
PostgreSQL support for the Open edX platform is a community-driven effort. If
117+
you encounter issues or have questions:
118+
119+
- Open an issue on the `tutor-contrib-postgresql repository
120+
<https://github.com/qasimgulzar/tutor-contrib-postgresql/issues>`_
121+
- Post on the `Open edX discussion forum <https://discuss.openedx.org>`_
122+
123+
124+
.. seealso::
125+
126+
:ref:`Open edX Databases`
127+
128+
`Tutor PostgreSQL plugin <https://github.com/qasimgulzar/tutor-contrib-postgresql>`_
129+
130+
131+
**Maintenance chart**
132+
133+
+--------------+-------------------------------+----------------+--------------------------------+
134+
| Review Date | Working Group Reviewer | Release |Test situation |
135+
+--------------+-------------------------------+----------------+--------------------------------+
136+
| 2025-04-10 | sarina | Verawood | Pass |
137+
+--------------+-------------------------------+----------------+--------------------------------+

0 commit comments

Comments
 (0)