Skip to content

Commit feae6ed

Browse files
Copilotasadali145
Copilot
and
asadali145
committed
Add ol_openedx_ai_static_translations plugin from PR #758
Co-authored-by: asadali145 <52656433+asadali145@users.noreply.github.com> Agent-Logs-Url: https://github.com/mitodl/open-edx-plugins/sessions/eaab10e1-8468-4675-9118-cd9b1fdac54e
1 parent ccd2b72 commit feae6ed

23 files changed

Lines changed: 7453 additions & 0 deletions

File tree

‎src/ol_openedx_ai_static_translations/CHANGELOG.rst‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
Change Log
2+
----------
3+
4+
..
5+
All enhancements and patches to ol_openedx_ai_static_translations will be documented
6+
in this file. It adheres to the structure of https://keepachangelog.com/ ,
7+
but in reStructuredText instead of Markdown (for ease of incorporation into
8+
Sphinx documentation and the PyPI description).
9+
10+
This project adheres to Semantic Versioning (https://semver.org/).
11+
.. There should always be an "Unreleased" section for changes pending release.

‎src/ol_openedx_ai_static_translations/LICENSE.txt‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
Copyright (C) 2022 MIT Open Learning
2+
3+
All rights reserved.
4+
5+
Redistribution and use in source and binary forms, with or without
6+
modification, are permitted provided that the following conditions are met:
7+
8+
* Redistributions of source code must retain the above copyright notice, this
9+
list of conditions and the following disclaimer.
10+
11+
* Redistributions in binary form must reproduce the above copyright notice,
12+
this list of conditions and the following disclaimer in the documentation
13+
and/or other materials provided with the distribution.
14+
15+
* Neither the name of the copyright holder nor the names of its
16+
contributors may be used to endorse or promote products derived from
17+
this software without specific prior written permission.
18+
19+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
20+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
21+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
22+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
23+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
25+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
26+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
27+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
28+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎src/ol_openedx_ai_static_translations/README.rst‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
OL Open edX AI Static Translations
2+
====================================
3+
4+
An Open edX plugin that provides AI-powered static translation management. It syncs translation keys, translates them using LLM providers, and creates pull requests with translated content.
5+
6+
Purpose
7+
*******
8+
9+
This plugin provides the ``sync_and_translate_language`` management command for syncing and translating Open edX static strings (frontend JSON and backend PO files) using LLM providers (OpenAI, Gemini, Mistral) with optional glossary support.
10+
11+
Setup
12+
=====
13+
14+
For detailed installation instructions, please refer to the `plugin installation guide <../../docs#installation-guide>`_.
15+
16+
Installation required in:
17+
18+
* Studio (CMS)
19+
20+
Configuration
21+
=============
22+
23+
This plugin shares settings with ``ol_openedx_course_translations``. Ensure the following settings are configured:
24+
25+
.. code-block:: python
26+
27+
TRANSLATIONS_PROVIDERS: {
28+
"default_provider": "mistral",
29+
"openai": {"api_key": "", "default_model": "gpt-5.2"},
30+
"gemini": {"api_key": "", "default_model": "gemini-3-pro-preview"},
31+
"mistral": {"api_key": "", "default_model": "mistral-large-latest"},
32+
}
33+
TRANSLATIONS_GITHUB_TOKEN: <YOUR_GITHUB_TOKEN>
34+
TRANSLATIONS_REPO_PATH: ""
35+
TRANSLATIONS_REPO_URL: "https://github.com/mitodl/mitxonline-translations.git"
36+
37+
Usage
38+
=====
39+
40+
.. code-block:: bash
41+
42+
# Sync and translate a language
43+
./manage.py cms sync_and_translate_language el
44+
45+
# With specific provider and model
46+
./manage.py cms sync_and_translate_language el --provider openai --model gpt-5.2 --glossary
47+
48+
License
49+
*******
50+
51+
The code in this repository is licensed under the BSD 3-Clause license unless
52+
otherwise noted.
53+
54+
Please see `LICENSE.txt <LICENSE.txt>`_ for details.

‎src/ol_openedx_ai_static_translations/ol_openedx_ai_static_translations/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
"""
2+
MIT's Open edX AI static translations plugin
3+
"""

‎src/ol_openedx_ai_static_translations/ol_openedx_ai_static_translations/apps.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
"""
2+
ol_openedx_ai_static_translations Django application initialization.
3+
"""
4+
5+
from django.apps import AppConfig
6+
from edx_django_utils.plugins import PluginSettings
7+
from openedx.core.djangoapps.plugins.constants import ProjectType, SettingsType
8+
9+
10+
class OLOpenedXAIStaticTranslationsConfig(AppConfig):
11+
"""
12+
Configuration for the ol_openedx_ai_static_translations Django application.
13+
"""
14+
15+
name = "ol_openedx_ai_static_translations"
16+
verbose_name = "OL AI Static Translations"
17+
18+
plugin_app = {
19+
PluginSettings.CONFIG: {
20+
ProjectType.CMS: {
21+
SettingsType.COMMON: {PluginSettings.RELATIVE_PATH: "settings.cms"},
22+
},
23+
},
24+
}

‎src/ol_openedx_ai_static_translations/ol_openedx_ai_static_translations/constants.py‎

Lines changed: 235 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
"""Constants for AI static translation synchronization."""
2+
3+
# LLM Provider names
4+
PROVIDER_DEEPL = "deepl"
5+
PROVIDER_GEMINI = "gemini"
6+
PROVIDER_MISTRAL = "mistral"
7+
PROVIDER_OPENAI = "openai"
8+
9+
# Learner-facing frontend applications that require translation
10+
LEARNER_FACING_APPS = [
11+
"frontend-app-learning",
12+
"frontend-app-learner-dashboard",
13+
"frontend-app-learner-record",
14+
"frontend-app-account",
15+
"frontend-app-profile",
16+
"frontend-app-authn",
17+
"frontend-app-catalog",
18+
"frontend-app-discussions",
19+
"frontend-component-header",
20+
"frontend-component-footer",
21+
"frontend-app-ora",
22+
"frontend-platform",
23+
]
24+
25+
# Plural forms configuration for different languages
26+
# Based on GNU gettext plural forms specification
27+
# See: https://www.gnu.org/software/gettext/manual/html_node/Plural-forms.html
28+
PLURAL_FORMS = {
29+
# Languages with no plural forms (nplurals=1)
30+
"ja": "nplurals=1; plural=0;", # Japanese
31+
"ko": "nplurals=1; plural=0;", # Korean
32+
"zh": "nplurals=1; plural=0;", # Chinese (all variants)
33+
"th": "nplurals=1; plural=0;", # Thai
34+
"vi": "nplurals=1; plural=0;", # Vietnamese
35+
"id": "nplurals=1; plural=0;", # Indonesian
36+
"ms": "nplurals=1; plural=0;", # Malay
37+
"km": "nplurals=1; plural=0;", # Khmer
38+
"bo": "nplurals=1; plural=0;", # Tibetan
39+
# Languages with 2 plural forms: plural=(n != 1)
40+
"en": "nplurals=2; plural=(n != 1);", # English
41+
"es": "nplurals=2; plural=(n != 1);", # Spanish (all variants)
42+
"de": "nplurals=2; plural=(n != 1);", # German
43+
"el": "nplurals=2; plural=(n != 1);", # Greek
44+
"it": "nplurals=2; plural=(n != 1);", # Italian
45+
"pt": "nplurals=2; plural=(n != 1);", # Portuguese (all variants)
46+
"nl": "nplurals=2; plural=(n != 1);", # Dutch
47+
"sv": "nplurals=2; plural=(n != 1);", # Swedish
48+
"da": "nplurals=2; plural=(n != 1);", # Danish
49+
"no": "nplurals=2; plural=(n != 1);", # Norwegian
50+
"nb": "nplurals=2; plural=(n != 1);", # Norwegian Bokmål
51+
"nn": "nplurals=2; plural=(n != 1);", # Norwegian Nynorsk
52+
"fi": "nplurals=2; plural=(n != 1);", # Finnish
53+
"is": "nplurals=2; plural=(n != 1);", # Icelandic
54+
"et": "nplurals=2; plural=(n != 1);", # Estonian
55+
"lv": "nplurals=2; plural=(n != 1);", # Latvian
56+
"he": "nplurals=2; plural=(n != 1);", # Hebrew
57+
"hi": "nplurals=2; plural=(n != 1);", # Hindi
58+
"bn": "nplurals=2; plural=(n != 1);", # Bengali
59+
"gu": "nplurals=2; plural=(n != 1);", # Gujarati
60+
"kn": "nplurals=2; plural=(n != 1);", # Kannada
61+
"ml": "nplurals=2; plural=(n != 1);", # Malayalam
62+
"ta": "nplurals=2; plural=(n != 1);", # Tamil
63+
"te": "nplurals=2; plural=(n != 1);", # Telugu
64+
"or": "nplurals=2; plural=(n != 1);", # Oriya
65+
"si": "nplurals=2; plural=(n != 1);", # Sinhala
66+
"ne": "nplurals=2; plural=(n != 1);", # Nepali
67+
"mr": "nplurals=2; plural=(n != 1);", # Marathi
68+
"ur": "nplurals=2; plural=(n != 1);", # Urdu
69+
"az": "nplurals=2; plural=(n != 1);", # Azerbaijani
70+
"uz": "nplurals=2; plural=(n != 1);", # Uzbek
71+
"kk": "nplurals=2; plural=(n != 1);", # Kazakh
72+
"mn": "nplurals=2; plural=(n != 1);", # Mongolian
73+
"sq": "nplurals=2; plural=(n != 1);", # Albanian
74+
"eu": "nplurals=2; plural=(n != 1);", # Basque
75+
"ca": "nplurals=2; plural=(n != 1);", # Catalan
76+
"gl": "nplurals=2; plural=(n != 1);", # Galician
77+
"tr": "nplurals=2; plural=(n != 1);", # Turkish
78+
"af": "nplurals=2; plural=(n != 1);", # Afrikaans
79+
"fil": "nplurals=2; plural=(n != 1);", # Filipino
80+
# Languages with 2 plural forms: plural=(n > 1)
81+
"fr": "nplurals=2; plural=(n > 1);", # French
82+
"br": "nplurals=2; plural=(n > 1);", # Breton
83+
# Languages with 3 plural forms
84+
"pl": (
85+
"nplurals=3; plural=(n==1 ? 0 : n%10>=2 && n%10<=4 && "
86+
"(n%100<10 || n%100>=20) ? 1 : 2);"
87+
), # Polish
88+
"ru": (
89+
"nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && "
90+
"(n%100<10 || n%100>=20) ? 1 : 2);"
91+
), # Russian
92+
"uk": (
93+
"nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && "
94+
"(n%100<10 || n%100>=20) ? 1 : 2);"
95+
), # Ukrainian
96+
"be": (
97+
"nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && "
98+
"(n%100<10 || n%100>=20) ? 1 : 2);"
99+
), # Belarusian
100+
"sr": (
101+
"nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && "
102+
"(n%100<10 || n%100>=20) ? 1 : 2);"
103+
), # Serbian
104+
"hr": (
105+
"nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && "
106+
"(n%100<10 || n%100>=20) ? 1 : 2);"
107+
), # Croatian
108+
"bs": (
109+
"nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && "
110+
"(n%100<10 || n%100>=20) ? 1 : 2);"
111+
), # Bosnian
112+
"cs": "nplurals=3; plural=(n==1 ? 0 : (n>=2 && n<=4) ? 1 : 2);", # Czech
113+
"sk": "nplurals=3; plural=(n==1 ? 0 : (n>=2 && n<=4) ? 1 : 2);", # Slovak
114+
"lt": (
115+
"nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && "
116+
"(n%100<10 || n%100>=20) ? 1 : 2);"
117+
), # Lithuanian
118+
"hy": "nplurals=3; plural=(n==1 ? 0 : n>=2 && n<=4 ? 1 : 2);", # Armenian
119+
"ro": (
120+
"nplurals=3; plural=(n==1 ? 0 : (n==0 || (n%100 > 0 && n%100 < 20)) ? 1 : 2);"
121+
), # Romanian
122+
# Languages with 4 plural forms
123+
"cy": (
124+
"nplurals=4; plural=(n==1 ? 0 : n==2 ? 1 : (n==8 || n==11) ? 2 : 3);"
125+
), # Welsh
126+
"ga": "nplurals=4; plural=(n==1 ? 0 : n==2 ? 1 : (n>2 && n<7) ? 2 : 3);", # Irish
127+
"gd": (
128+
"nplurals=4; plural=(n==1 || n==11) ? 0 : (n==2 || n==12) ? 1 : "
129+
"(n>2 && n<20) ? 2 : 3);"
130+
), # Scottish Gaelic
131+
"mt": (
132+
"nplurals=4; plural=(n==1 ? 0 : n==0 || (n%100>=2 && n%100<=10) ? 1 : "
133+
"(n%100>=11 && n%100<=19) ? 2 : 3);"
134+
), # Maltese
135+
# Languages with 6 plural forms
136+
"ar": (
137+
"nplurals=6; plural=(n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && "
138+
"n%100<=10 ? 3 : n%100>=11 && n%100<=99 ? 4 : 5);"
139+
), # Arabic
140+
# Other languages
141+
"fa": "nplurals=2; plural=(n==0 || n==1 ? 0 : 1);", # Persian/Farsi
142+
"hu": "nplurals=2; plural=(n != 1);", # Hungarian
143+
"bg": "nplurals=2; plural=(n != 1);", # Bulgarian
144+
"am": "nplurals=2; plural=(n > 1);", # Amharic
145+
}
146+
147+
# Default plural form fallback (English-style)
148+
# Used when a language code is not found in PLURAL_FORMS
149+
DEFAULT_PLURAL_FORM = "nplurals=2; plural=(n != 1);"
150+
151+
# Typo patterns to fix in translation files
152+
TYPO_PATTERNS = [
153+
("Serch", "Search"),
154+
]
155+
156+
# Backend PO file names
157+
BACKEND_PO_FILES = ["django.po", "djangojs.po"]
158+
159+
# Backend plugin apps: (repo_dir, module_name) under translations/.
160+
# Used by sync_and_translate_language to sync/translate at
161+
# translations/<repo_dir>/<module_name>/conf/locale/<lang>/LC_MESSAGES/django.po.
162+
# When pulled in edx-platform (make pull_translations), these go to
163+
# conf/plugins-locale/plugins/<module_name>/.
164+
TRANSLATABLE_PLUGINS = [
165+
("open-edx-plugins", "ol_openedx_chat"),
166+
]
167+
168+
# PO file header metadata
169+
PO_HEADER_PROJECT_VERSION = "0.1a"
170+
PO_HEADER_BUGS_EMAIL = "openedx-translation@googlegroups.com"
171+
PO_HEADER_POT_CREATION_DATE = "2023-06-13 08:00+0000"
172+
PO_HEADER_MIME_VERSION = "1.0"
173+
PO_HEADER_CONTENT_TYPE = "text/plain; charset=UTF-8"
174+
PO_HEADER_CONTENT_TRANSFER_ENCODING = "8bit"
175+
PO_HEADER_TRANSIFEX_TEAM_BASE_URL = "https://app.transifex.com/open-edx/teams/6205"
176+
177+
# File and directory names
178+
TRANSLATION_FILE_NAMES = {
179+
"transifex_input": "transifex_input.json",
180+
"english": "en.json",
181+
"messages_dir": "messages",
182+
"i18n_dir": "i18n",
183+
"locale_dir": "locale",
184+
"lc_messages": "LC_MESSAGES",
185+
"conf_dir": "conf",
186+
"edx_platform": "edx-platform",
187+
}
188+
189+
# JSON file formatting
190+
DEFAULT_JSON_INDENT = 2
191+
192+
# Language code to human-readable name mapping
193+
# Used in PO file headers for Language-Team field
194+
LANGUAGE_MAPPING = {
195+
"ar": "Arabic",
196+
"de": "German",
197+
"el": "Greek",
198+
"es": "Spanish",
199+
"fr": "French",
200+
"hi": "Hindi",
201+
"id": "Indonesian",
202+
"ja": "Japanese",
203+
"kr": "Korean",
204+
"pt": "Portuguese",
205+
"ru": "Russian",
206+
"sq": "Albanian",
207+
"tr": "Turkish",
208+
"zh": "Chinese",
209+
}
210+
211+
# Maximum number of retries for failed translation batches
212+
MAX_RETRIES = 3
213+
214+
# Glossary parsing constants
215+
EXPECTED_GLOSSARY_PARTS = 2 # English term and translation separated by "->"
216+
217+
# HTTP Status Codes
218+
HTTP_OK = 200
219+
HTTP_CREATED = 201
220+
HTTP_NOT_FOUND = 404
221+
HTTP_TOO_MANY_REQUESTS = 429
222+
HTTP_UNPROCESSABLE_ENTITY = 422
223+
224+
# Error message length limit
225+
MAX_ERROR_MESSAGE_LENGTH = 200
226+
227+
# Maximum length for strings in log messages (truncate with "...")
228+
MAX_LOG_STRING_LENGTH = 50
229+
MAX_LOG_ICU_STRING_LENGTH = 100
230+
231+
# Plural category counts (GNU gettext nplurals)
232+
PLURAL_CATEGORIES_ARABIC = 6 # zero, one, two, few, many, other
233+
PLURAL_CATEGORIES_FOUR = 4 # one, two, few, other
234+
PLURAL_CATEGORIES_THREE = 3 # one, few, other
235+
PLURAL_CATEGORIES_TWO = 2 # one, other (most languages)

0 commit comments

Comments
 (0)