From 452084dafc373f48bcd902194282572cdf83ffed Mon Sep 17 00:00:00 2001 From: Shakti Suman Date: Tue, 12 May 2026 23:47:11 -0700 Subject: [PATCH] Add D2 diagrams to sphinx docs. Signed-off-by: Shakti Suman --- .github/workflows/docs.yaml | 32 +- README.md | 4 +- cmake/modules/BuildSphinxTarget.cmake | 15 +- docs/userguide/CMakeLists.txt | 124 ++++-- docs/userguide/_ext/eld_d2.py | 411 ++++++++++++++++++ docs/userguide/_static/d2.css | 52 +++ docs/userguide/api_docs/CMakeLists.txt | 14 +- docs/userguide/api_docs/Doxyfile.in | 3 +- docs/userguide/conf.py.in | 15 +- docs/userguide/documentation/images/Common.d2 | 94 ++++ .../images/ControlMemorySizePluginFlow.d2 | 12 + .../images/ControlMemorySizePluginFlow.dot | 15 - .../documentation/images/LinkStates.d2 | 10 + .../documentation/images/LinkStates.dot | 11 - .../documentation/images/LinkerPluginFlow.d2 | 27 ++ .../documentation/images/LinkerPluginFlow.dot | 48 -- .../documentation/images/LinkerWrapper.d2 | 9 + .../documentation/images/LinkerWrapper.dot | 9 - .../images/OutputSectionAfterLayoutFlow.d2 | 11 + .../images/OutputSectionAfterLayoutFlow.dot | 13 - .../images/OutputSectionBeforeLayoutFlow.d2 | 13 + .../images/OutputSectionBeforeLayoutFlow.dot | 16 - .../OutputSectionCreatingSectionsFlow.d2 | 12 + .../OutputSectionCreatingSectionsFlow.dot | 14 - .../images/SectionIteratorFlow.d2 | 13 + .../images/SectionIteratorFlow.dot | 16 - .../images/SectionMatcherFlow.d2 | 13 + .../images/SectionMatcherFlow.dot | 16 - .../documentation/linker_plugins/api_docs.rst | 2 +- .../linker_plugins/linker_plugins.rst | 16 +- 30 files changed, 844 insertions(+), 216 deletions(-) create mode 100644 docs/userguide/_ext/eld_d2.py create mode 100644 docs/userguide/_static/d2.css create mode 100644 docs/userguide/documentation/images/Common.d2 create mode 100644 docs/userguide/documentation/images/ControlMemorySizePluginFlow.d2 delete mode 100644 docs/userguide/documentation/images/ControlMemorySizePluginFlow.dot create mode 100644 docs/userguide/documentation/images/LinkStates.d2 delete mode 100644 docs/userguide/documentation/images/LinkStates.dot create mode 100644 docs/userguide/documentation/images/LinkerPluginFlow.d2 delete mode 100644 docs/userguide/documentation/images/LinkerPluginFlow.dot create mode 100644 docs/userguide/documentation/images/LinkerWrapper.d2 delete mode 100644 docs/userguide/documentation/images/LinkerWrapper.dot create mode 100644 docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.d2 delete mode 100644 docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.dot create mode 100644 docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.d2 delete mode 100644 docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.dot create mode 100644 docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.d2 delete mode 100644 docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.dot create mode 100644 docs/userguide/documentation/images/SectionIteratorFlow.d2 delete mode 100644 docs/userguide/documentation/images/SectionIteratorFlow.dot create mode 100644 docs/userguide/documentation/images/SectionMatcherFlow.d2 delete mode 100644 docs/userguide/documentation/images/SectionMatcherFlow.dot diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml index 346494ee4..c145e6228 100644 --- a/.github/workflows/docs.yaml +++ b/.github/workflows/docs.yaml @@ -41,9 +41,28 @@ jobs: - name: Install dependencies run: | + sudo apt-get update sudo apt-get install -y graphviz doxygen libxml2 libxml2-dev libxslt1-dev pip install -r llvm-project/llvm/tools/eld/docs/userguide/requirements.txt + - name: Install D2 diagram generator + run: | + curl -fsSL https://d2lang.com/install.sh | sh -s -- --prefix "$HOME/.local" + echo "$HOME/.local/bin" >> "$GITHUB_PATH" + + - name: Configure D2 PDF renderer + run: | + CHROME_BIN="$(command -v google-chrome || command -v google-chrome-stable || command -v chromium || command -v chromium-browser || true)" + if [ -z "$CHROME_BIN" ]; then + cd /tmp + wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb + sudo apt-get install -y ./google-chrome-stable_current_amd64.deb + CHROME_BIN="$(command -v google-chrome || command -v google-chrome-stable)" + fi + echo "D2_PDF_CHROME_BIN=$CHROME_BIN" >> "$GITHUB_ENV" + d2 --version + "$CHROME_BIN" --version + - name: Run CMake run: | mkdir obj @@ -57,12 +76,17 @@ jobs: -DELD_ENABLE_PDF_DOCS:BOOL=ON \ ../llvm-project/llvm - - name: Generate documentation + - name: Generate HTML documentation with D2 diagrams + run: | + cd obj + ninja docs-eld-userguide-html + + - name: Generate PDF documentation with D2 diagrams run: | cd obj - ninja eld-docs - mv tools/eld/docs/userguide/pdf/ELD_UserGuide.pdf \ - tools/eld/docs/userguide/html/ELD_UserGuide.pdf + ninja docs-eld-userguide-pdf + cp tools/eld/docs/userguide/pdf/ELD_UserGuide.pdf \ + tools/eld/docs/userguide/html/ELD_UserGuide.pdf - name: Deploy to docs branch run: | diff --git a/README.md b/README.md index 6b2cf7171..a2998eb54 100644 --- a/README.md +++ b/README.md @@ -207,13 +207,15 @@ ninja check-eld-vim First install the prerequisites for building documentation: - Doxygen (>=1.8.11) -- graphviz +- graphviz (for Doxygen-generated diagrams) +- D2 CLI (for Sphinx diagrams) - Sphinx and other python dependencies as specified in 'docs/userguide/requirements.txt' On an ubuntu machine, the prerequisites can be installed as: ``` sudo apt install doxygen graphviz +curl -fsSL https://d2lang.com/install.sh | sh -s -- pip3 install -r ${ELDRoot}/docs/userguide/requirements.txt ``` diff --git a/cmake/modules/BuildSphinxTarget.cmake b/cmake/modules/BuildSphinxTarget.cmake index f656fa727..2fd2473cd 100644 --- a/cmake/modules/BuildSphinxTarget.cmake +++ b/cmake/modules/BuildSphinxTarget.cmake @@ -7,13 +7,16 @@ function (build_sphinx_target builder project target_name) set(SPHINX_BUILD_DIR "${CMAKE_CURRENT_BINARY_DIR}/${builder}") set(SPHINX_DOC_TREE_DIR "${CMAKE_CURRENT_BINARY_DIR}/_doctrees-${project}-${builder}") + set(SPHINX_STAMP_FILE "${SPHINX_BUILD_DIR}/.${project}-${builder}.stamp") set(SPHINX_TARGET_NAME docs-${project}-${builder}) set(${target_name} ${SPHINX_TARGET_NAME} PARENT_SCOPE) if (NOT ARG_SOURCE_DIR) set(ARG_SOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}/source") endif() - add_custom_target(${SPHINX_TARGET_NAME} + add_custom_command( + OUTPUT "${SPHINX_STAMP_FILE}" + COMMAND ${CMAKE_COMMAND} -E make_directory "${SPHINX_BUILD_DIR}" COMMAND ${SPHINX_EXECUTABLE} -b ${builder} -Dbreathe_projects.ELD="${ARG_DOXYGEN_XML_DIR}" # This is needed to hook sphinx up with the information generated by doxygen @@ -21,8 +24,10 @@ function (build_sphinx_target builder project target_name) -q # Quiet: no output other than errors and warnings. "${ARG_SOURCE_DIR}" # Source "${SPHINX_BUILD_DIR}" # Output - DEPENDS eld-api-docs + COMMAND ${CMAKE_COMMAND} -E touch "${SPHINX_STAMP_FILE}" + DEPENDS ${ARG_DEPENDS} COMMENT - "Generating ${builder} Sphinx documentation for ${project} into \"${SPHINX_BUILD_DIR}\"") - add_dependencies(${SPHINX_TARGET_NAME} eld-api-docs ${ARG_DEPENDS}) -endfunction() \ No newline at end of file + "Generating ${builder} Sphinx documentation for ${project} into \"${SPHINX_BUILD_DIR}\"" + VERBATIM) + add_custom_target(${SPHINX_TARGET_NAME} DEPENDS "${SPHINX_STAMP_FILE}") +endfunction() diff --git a/docs/userguide/CMakeLists.txt b/docs/userguide/CMakeLists.txt index 98dab4a5a..7795a8633 100644 --- a/docs/userguide/CMakeLists.txt +++ b/docs/userguide/CMakeLists.txt @@ -3,67 +3,129 @@ add_subdirectory(api_docs) if(LLVM_ENABLE_SPHINX) include(${ELD_SOURCE_DIR}/cmake/modules/BuildSphinxTarget.cmake) set(EXHALE_OUTPUT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/api") + if(NOT ELD_D2_BIN) + if(DEFINED ENV{D2_BIN}) + set(ELD_D2_BIN + "$ENV{D2_BIN}" + CACHE FILEPATH "Path to the D2 executable used for ELD diagrams") + else() + find_program(ELD_D2_BIN NAMES d2 DOC "Path to the D2 executable used for ELD diagrams") + endif() + endif() + if(NOT ELD_D2_BIN) + message(FATAL_ERROR "D2 executable is required to build ELD userguide diagrams. Set ELD_D2_BIN or D2_BIN.") + endif() configure_file(conf.py.in ${CMAKE_CURRENT_BINARY_DIR}/source/conf.py) file(MAKE_DIRECTORY ${EXHALE_OUTPUT_DIR}) set(DOCS_BUILD_SOURCE ${CMAKE_CURRENT_BINARY_DIR}/source) + set(ELD_COPY_USERGUIDE_SOURCE_STAMP + ${DOCS_BUILD_SOURCE}/.copy-userguide-source.stamp) + file(GLOB_RECURSE ELD_USERGUIDE_SOURCE_FILES ${CMAKE_CURRENT_SOURCE_DIR}/*) + list(FILTER ELD_USERGUIDE_SOURCE_FILES EXCLUDE REGEX "/(__pycache__|api)(/|$)") # Copy the documentation sources to the build directory. We need to copy the # documentation sources to the build directory so that we have both # handwritten and automatically generated documentation sources in the same # directory. This is required because sphinx support only a single source # directory. - add_custom_target( - eld-copy-userguide-source + add_custom_command( + OUTPUT ${ELD_COPY_USERGUIDE_SOURCE_STAMP} COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_SOURCE_DIR} ${DOCS_BUILD_SOURCE} - WORKING_DIRECTORY ${CMAKE_BINARY_DIR}) + COMMAND ${CMAKE_COMMAND} -E touch ${ELD_COPY_USERGUIDE_SOURCE_STAMP} + DEPENDS ${ELD_USERGUIDE_SOURCE_FILES} + WORKING_DIRECTORY ${CMAKE_BINARY_DIR} + COMMENT "Copying ELD userguide sources" + VERBATIM) + add_custom_target(eld-copy-userguide-source + DEPENDS ${ELD_COPY_USERGUIDE_SOURCE_STAMP}) - add_custom_target(eld-linkeroptions-docs) set(LINKER_OPTS_DUMP_DIR ${DOCS_BUILD_SOURCE}/LinkerOptionsDump) + set(ELD_LINKEROPTIONS_DUMP_DIR_STAMP ${LINKER_OPTS_DUMP_DIR}/.dir.stamp) set(OPTIONS_SUPPLEMENTS_DIR ${CMAKE_CURRENT_SOURCE_DIR}/CommandLineOptionsSupplements) - add_custom_target( - eld-linkeroptions-create-json-dump-dir - COMMAND ${CMAKE_COMMAND} -E make_directory ${LINKER_OPTS_DUMP_DIR}) + file(GLOB ELD_LINKEROPTIONS_SUPPLEMENTS ${OPTIONS_SUPPLEMENTS_DIR}/*) + add_custom_command( + OUTPUT ${ELD_LINKEROPTIONS_DUMP_DIR_STAMP} + COMMAND ${CMAKE_COMMAND} -E make_directory ${LINKER_OPTS_DUMP_DIR} + COMMAND ${CMAKE_COMMAND} -E touch ${ELD_LINKEROPTIONS_DUMP_DIR_STAMP} + COMMENT "Creating ELD linker options dump directory" + VERBATIM) + add_custom_target(eld-linkeroptions-create-json-dump-dir + DEPENDS ${ELD_LINKEROPTIONS_DUMP_DIR_STAMP}) + + set(ELD_LINKEROPTIONS_DOC_OUTPUTS) + set(ELD_API_DOC_OUTPUTS ${DOXYGEN_OUTPUT_DIR}/xml/index.xml + ${DOXYGEN_OUTPUT_DIR}/html/index.html) + set(ELD_GNU_LINKEROPTIONS_JSON_DUMP + ${LINKER_OPTS_DUMP_DIR}/GnuLinkerOptionsTblGenDump.json) foreach(target Gnu ARM Hexagon RISCV) string(TOLOWER ${target} targetLC) # Generate JSON dump file of ${target}LinkerOptions.td - add_custom_target( - eld-${targetLC}linkeroptions-json-dump + set(LINKER_OPTIONS_TD + ${ELD_SOURCE_DIR}/include/eld/Driver/${target}LinkerOptions.td) + set(LINKER_OPTIONS_JSON_DUMP + ${LINKER_OPTS_DUMP_DIR}/${target}LinkerOptionsTblGenDump.json) + set(LINKER_OPTIONS_JSON_DEPENDS + llvm-tblgen + ${ELD_LINKEROPTIONS_DUMP_DIR_STAMP} + ${LINKER_OPTIONS_TD} + ${LLVM_MAIN_INCLUDE_DIR}/llvm/Option/OptParser.td) + if(NOT target MATCHES "Gnu") + list(APPEND LINKER_OPTIONS_JSON_DEPENDS + ${ELD_SOURCE_DIR}/include/eld/Driver/GnuLinkerOptions.td) + endif() + add_custom_command( + OUTPUT ${LINKER_OPTIONS_JSON_DUMP} COMMAND llvm-tblgen - ${ELD_SOURCE_DIR}/include/eld/Driver/${target}LinkerOptions.td + ${LINKER_OPTIONS_TD} -dump-json -I${LLVM_MAIN_INCLUDE_DIR} -I${ELD_SOURCE_DIR}/include/eld/Driver -o - ${LINKER_OPTS_DUMP_DIR}/${target}LinkerOptionsTblGenDump.json - DEPENDS eld-linkeroptions-create-json-dump-dir) + ${LINKER_OPTIONS_JSON_DUMP} + DEPENDS ${LINKER_OPTIONS_JSON_DEPENDS} + COMMENT "Generating ${target} linker options JSON dump" + VERBATIM) + add_custom_target(eld-${targetLC}linkeroptions-json-dump + DEPENDS ${LINKER_OPTIONS_JSON_DUMP}) # Generates ${target}LinkerOptions restructureText documentation from the # ${target}LinkerOptions JSON dump file. set(SKIP_OPTION "") + set(LINKER_OPTIONS_DOC_DEPENDS + ${LINKER_OPTIONS_JSON_DUMP} + ${ELD_COPY_USERGUIDE_SOURCE_STAMP} + ${CMAKE_CURRENT_SOURCE_DIR}/GenerateOptionsDocsFromTblGen.py + ${ELD_LINKEROPTIONS_SUPPLEMENTS}) if(NOT target MATCHES "Gnu") set(SKIP_OPTION -s - ${LINKER_OPTS_DUMP_DIR}/GnuLinkerOptionsTblGenDump.json) + ${ELD_GNU_LINKEROPTIONS_JSON_DUMP}) + list(APPEND LINKER_OPTIONS_DOC_DEPENDS ${ELD_GNU_LINKEROPTIONS_JSON_DUMP}) endif() - add_custom_target( - eld-${targetLC}linkeroptions-docs + set(LINKER_OPTIONS_DOC + ${DOCS_BUILD_SOURCE}/documentation/options/${target}LinkerOptions.rst) + add_custom_command( + OUTPUT ${LINKER_OPTIONS_DOC} + COMMAND ${CMAKE_COMMAND} -E make_directory + ${DOCS_BUILD_SOURCE}/documentation/options COMMAND python3 ${CMAKE_CURRENT_SOURCE_DIR}/GenerateOptionsDocsFromTblGen.py - ${LINKER_OPTS_DUMP_DIR}/${target}LinkerOptionsTblGenDump.json -o - ${DOCS_BUILD_SOURCE}/documentation/options/${target}LinkerOptions.rst + ${LINKER_OPTIONS_JSON_DUMP} -o + ${LINKER_OPTIONS_DOC} ${SKIP_OPTION} - -S ${OPTIONS_SUPPLEMENTS_DIR}) - - add_dependencies(eld-${targetLC}linkeroptions-docs - eld-${targetLC}linkeroptions-json-dump) - add_dependencies(eld-linkeroptions-docs eld-${targetLC}linkeroptions-docs) - if(NOT target MATCHES "Gnu") - add_dependencies(eld-${targetLC}linkeroptions-docs - eld-gnulinkeroptions-json-dump) - endif() + -S ${OPTIONS_SUPPLEMENTS_DIR} + DEPENDS ${LINKER_OPTIONS_DOC_DEPENDS} + COMMENT "Generating ${target} linker options documentation" + VERBATIM) + add_custom_target(eld-${targetLC}linkeroptions-docs + DEPENDS ${LINKER_OPTIONS_DOC}) + list(APPEND ELD_LINKEROPTIONS_DOC_OUTPUTS ${LINKER_OPTIONS_DOC}) endforeach() + add_custom_target(eld-linkeroptions-docs + DEPENDS ${ELD_LINKEROPTIONS_DOC_OUTPUTS}) + # HTML user guide build_sphinx_target( html @@ -72,8 +134,9 @@ if(LLVM_ENABLE_SPHINX) DOXYGEN_XML_DIR ${DOXYGEN_OUTPUT_DIR}/xml DEPENDS - eld-copy-userguide-source - eld-linkeroptions-docs) + ${ELD_API_DOC_OUTPUTS} + ${ELD_COPY_USERGUIDE_SOURCE_STAMP} + ${ELD_LINKEROPTIONS_DOC_OUTPUTS}) add_custom_target(eld-docs DEPENDS ${html_target_name}) @@ -86,8 +149,9 @@ if(LLVM_ENABLE_SPHINX) DOXYGEN_XML_DIR ${DOXYGEN_OUTPUT_DIR}/xml DEPENDS - eld-copy-userguide-source - eld-linkeroptions-docs) + ${ELD_API_DOC_OUTPUTS} + ${ELD_COPY_USERGUIDE_SOURCE_STAMP} + ${ELD_LINKEROPTIONS_DOC_OUTPUTS}) add_dependencies(eld-docs ${pdf_target_name}) endif() diff --git a/docs/userguide/_ext/eld_d2.py b/docs/userguide/_ext/eld_d2.py new file mode 100644 index 000000000..54784623e --- /dev/null +++ b/docs/userguide/_ext/eld_d2.py @@ -0,0 +1,411 @@ +"""Sphinx support for rendering D2 diagrams.""" + +from __future__ import annotations + +import hashlib +import math +import re +import shutil +import subprocess +from os import path +from pathlib import Path +from subprocess import PIPE, CalledProcessError +from typing import Any + +import posixpath +import sphinx +from docutils import nodes +from docutils.nodes import Node +from docutils.parsers.rst import Directive, directives +from sphinx.application import Sphinx +from sphinx.errors import SphinxError +from sphinx.locale import __, _ +from sphinx.util.docutils import SphinxDirective +from sphinx.util.i18n import search_image_for_language +from sphinx.util.nodes import set_source_info +from sphinx.util.osutil import ensuredir +from sphinx.util.typing import OptionSpec +from sphinx.writers.html import HTMLTranslator +from sphinx.writers.latex import LaTeXTranslator +from sphinx.writers.manpage import ManualPageTranslator +from sphinx.writers.text import TextTranslator + + +class D2Error(SphinxError): + category = "D2 error" + + +class d2_diagram(nodes.General, nodes.Element): + pass + + +def align_spec(argument: Any) -> str: + return directives.choice(argument, ("left", "center", "right")) + + +def figure_wrapper( + directive: Directive, node: d2_diagram, caption: str +) -> nodes.figure: + figure_node = nodes.figure("", node) + if "align" in node: + figure_node["align"] = node.attributes.pop("align") + + inline_nodes, messages = directive.state.inline_text(caption, directive.lineno) + caption_node = nodes.caption(caption, "", *inline_nodes) + caption_node.extend(messages) + set_source_info(directive, caption_node) + figure_node += caption_node + return figure_node + + +class D2Directive(SphinxDirective): + has_content = True + required_arguments = 0 + optional_arguments = 1 + final_argument_whitespace = False + option_spec: OptionSpec = { + "alt": directives.unchanged, + "align": align_spec, + "caption": directives.unchanged, + "name": directives.unchanged, + "class": directives.class_option, + } + + def run(self) -> list[Node]: + if self.arguments: + if self.content: + warning = self.state.document.reporter.warning( + __("D2 directive cannot have both content and a filename argument"), + line=self.lineno, + ) + return [warning] + + argument = search_image_for_language(self.arguments[0], self.env) + relative_filename, filename = self.env.relfn2path(argument) + self.env.note_dependency(relative_filename) + try: + with open(filename, encoding="utf-8") as diagram_file: + code = diagram_file.read() + except OSError: + warning = self.state.document.reporter.warning( + __("External D2 file %r not found or reading it failed") + % filename, + line=self.lineno, + ) + return [warning] + else: + code = "\n".join(self.content) + relative_filename = None + if not code.strip(): + warning = self.state_machine.reporter.warning( + __("Ignoring \"d2\" directive without content."), + line=self.lineno, + ) + return [warning] + + diagram_node = d2_diagram() + diagram_node["code"] = code + diagram_node["options"] = {"docname": self.env.docname} + if relative_filename: + diagram_node["filename"] = relative_filename + if "alt" in self.options: + diagram_node["alt"] = self.options["alt"] + if "align" in self.options: + diagram_node["align"] = self.options["align"] + if "class" in self.options: + diagram_node["classes"] = self.options["class"] + + if "caption" in self.options: + figure_node = figure_wrapper(self, diagram_node, self.options["caption"]) + self.add_name(figure_node) + return [figure_node] + + self.add_name(diagram_node) + return [diagram_node] + + +def render_d2( + translator: Any, + code: str, + options: dict[str, Any], + output_format: str, + filename: str | None = None, +) -> tuple[str, str]: + d2_bin = translator.builder.config.d2_bin + d2_args = list(translator.builder.config.d2_args) + if output_format == "txt" and not any( + arg == "--ascii-mode" or arg.startswith("--ascii-mode=") for arg in d2_args + ): + d2_args.extend(["--ascii-mode", "standard"]) + hash_inputs = [code, str(d2_bin), str(d2_args), output_format] + if output_format == "png": + hash_inputs.extend( + [ + str(translator.builder.config.d2_pdf_chrome_bin), + str(translator.builder.config.d2_pdf_raster_scale), + ] + ) + hash_key = "\n".join(hash_inputs).encode() + output_name = "d2-%s.%s" % (hashlib.sha1(hash_key).hexdigest(), output_format) + relative_output = posixpath.join(translator.builder.imgpath, output_name) + output_path = path.join( + translator.builder.outdir, translator.builder.imagedir, output_name + ) + + if path.isfile(output_path): + return relative_output, output_path + + ensuredir(path.dirname(output_path)) + + if output_format == "png": + svg_path = render_d2( + translator, code, options, "svg", filename + )[1] + rasterize_svg_for_pdf(translator, svg_path, output_path) + return relative_output, output_path + + command = [d2_bin, *d2_args] + input_bytes = None + if filename: + input_path = path.join(translator.builder.srcdir, filename) + working_dir = path.dirname(input_path) + command.extend([input_path, output_path]) + else: + docname = options.get("docname", "index") + working_dir = path.dirname(path.join(translator.builder.srcdir, docname)) + command.extend(["-", output_path]) + input_bytes = code.encode() + + try: + result = subprocess.run( + command, + input=input_bytes, + stdout=PIPE, + stderr=PIPE, + cwd=working_dir, + check=True, + ) + except OSError as error: + raise D2Error( + __("d2 command %r cannot be run; check the d2_bin setting") % d2_bin + ) from error + except CalledProcessError as error: + raise D2Error( + __("d2 exited with error:\n[stderr]\n%r\n[stdout]\n%r") + % (error.stderr, error.stdout) + ) from error + + if not path.isfile(output_path): + raise D2Error( + __("d2 did not produce an output file:\n[stderr]\n%r\n[stdout]\n%r") + % (result.stderr, result.stdout) + ) + return relative_output, output_path + + +def svg_viewbox_size(svg_path: str) -> tuple[int, int]: + with open(svg_path, encoding="utf-8") as svg_file: + svg_start = svg_file.read(4096) + + viewbox_match = re.search(r'viewBox="\s*[-\d.]+\s+[-\d.]+\s+([\d.]+)\s+([\d.]+)', svg_start) + if viewbox_match: + return ( + max(1, math.ceil(float(viewbox_match.group(1)))), + max(1, math.ceil(float(viewbox_match.group(2)))), + ) + + svg_match = re.search(r"]*", svg_start) + if svg_match: + width_match = re.search(r'\bwidth="([\d.]+)', svg_match.group(0)) + height_match = re.search(r'\bheight="([\d.]+)', svg_match.group(0)) + if width_match and height_match: + return ( + max(1, math.ceil(float(width_match.group(1)))), + max(1, math.ceil(float(height_match.group(1)))), + ) + + return (1200, 900) + + +def find_chrome_bin(config: Any) -> str: + configured_chrome = config.d2_pdf_chrome_bin + candidates = [configured_chrome] if configured_chrome else [] + candidates.extend(["google-chrome", "chromium", "chromium-browser", "chrome"]) + for candidate in candidates: + if not candidate: + continue + resolved = shutil.which(candidate) + if resolved: + return resolved + if path.isfile(candidate): + return candidate + + raise D2Error( + _( + "D2 PDF rendering requires Google Chrome or Chromium to rasterize " + "SVG diagrams. Set d2_pdf_chrome_bin or D2_PDF_CHROME_BIN." + ) + ) + + +def rasterize_svg_for_pdf(translator: Any, svg_path: str, png_path: str) -> None: + chrome_bin = find_chrome_bin(translator.builder.config) + width, height = svg_viewbox_size(svg_path) + scale = translator.builder.config.d2_pdf_raster_scale + command = [ + chrome_bin, + "--headless", + "--disable-gpu", + "--hide-scrollbars", + "--no-sandbox", + f"--force-device-scale-factor={scale}", + f"--window-size={width},{height}", + f"--screenshot={png_path}", + Path(svg_path).resolve().as_uri(), + ] + + try: + result = subprocess.run(command, stdout=PIPE, stderr=PIPE, check=True) + except OSError as error: + raise D2Error( + _("Chrome command %r cannot be run; check d2_pdf_chrome_bin") + % chrome_bin + ) from error + except CalledProcessError as error: + raise D2Error( + _("Chrome failed to rasterize D2 SVG:\n[stderr]\n%r\n[stdout]\n%r") + % (error.stderr, error.stdout) + ) from error + + if not path.isfile(png_path): + raise D2Error( + _("Chrome did not produce a D2 PNG:\n[stderr]\n%r\n[stdout]\n%r") + % (result.stderr, result.stdout) + ) + + +def make_d2_image_node( + translator: Any, + node: d2_diagram, + output_format: str, + absolute_uri: bool = False, +) -> nodes.image: + relative_output, output_path = render_d2( + translator, node["code"], node["options"], output_format, node.get("filename") + ) + + image_node = nodes.image(uri=output_path if absolute_uri else relative_output) + if "alt" in node: + image_node["alt"] = node["alt"] + if "align" in node: + image_node["align"] = node["align"] + if "classes" in node: + image_node["classes"] = node["classes"] + for attribute in ("ids", "names", "dupnames", "backrefs"): + if attribute in node: + image_node[attribute] = node[attribute] + return image_node + + +def html_visit_d2_diagram(translator: HTMLTranslator, node: d2_diagram) -> None: + relative_output = render_d2( + translator, node["code"], node["options"], "svg", node.get("filename") + )[0] + + alt = node.get("alt", _("D2 diagram")) + classes = " ".join(["d2-diagram", *node.get("classes", [])]) + if "align" in node: + translator.body.append( + '
' % (node["align"], node["align"]) + ) + translator.body.append('
') + translator.body.append( + '\n' + % (relative_output, classes) + ) + translator.body.append('

%s

' % translator.encode(alt)) + translator.body.append("
\n") + if "align" in node: + translator.body.append("
\n") + raise nodes.SkipNode + + +def latex_visit_d2_diagram(translator: LaTeXTranslator, node: d2_diagram) -> None: + image_node = make_d2_image_node( + translator, node, translator.builder.config.d2_latex_output_format + ) + translator.visit_image(image_node) + raise nodes.SkipNode + + +def rst2pdf_visit_d2_diagram(translator: Any, node: d2_diagram) -> None: + if translator.builder.config.d2_pdf_output_format == "txt": + output_path = render_d2( + translator, + node["code"], + node["options"], + "txt", + node.get("filename"), + )[1] + with open(output_path, encoding="utf-8") as text_diagram_file: + text_diagram = text_diagram_file.read() + literal_node = nodes.literal_block(text_diagram, text_diagram) + node.parent.replace(node, literal_node) + raise nodes.SkipNode + + image_node = make_d2_image_node( + translator, + node, + translator.builder.config.d2_pdf_output_format, + absolute_uri=True, + ) + node.parent.replace(node, image_node) + raise nodes.SkipNode + + +def rst2pdf_depart_d2_diagram(translator: Any, node: d2_diagram) -> None: + pass + + +def text_visit_d2_diagram(translator: TextTranslator, node: d2_diagram) -> None: + translator.add_text(_("[D2 diagram: %s]") % node.get("alt", "")) + raise nodes.SkipNode + + +def man_visit_d2_diagram(translator: ManualPageTranslator, node: d2_diagram) -> None: + translator.body.append(_("[D2 diagram: %s]") % node.get("alt", "")) + raise nodes.SkipNode + + +def install_rst2pdf_support() -> None: + try: + from rst2pdf.pdfbuilder import PDFTranslator + except Exception: + return + + if getattr(PDFTranslator, "_eld_d2_support", False): + return + + PDFTranslator.visit_d2_diagram = rst2pdf_visit_d2_diagram + PDFTranslator.depart_d2_diagram = rst2pdf_depart_d2_diagram + PDFTranslator._eld_d2_support = True + + +def setup(app: Sphinx) -> dict[str, Any]: + install_rst2pdf_support() + app.add_node( + d2_diagram, + html=(html_visit_d2_diagram, None), + latex=(latex_visit_d2_diagram, None), + text=(text_visit_d2_diagram, None), + man=(man_visit_d2_diagram, None), + ) + app.add_directive("d2", D2Directive) + app.add_config_value("d2_bin", "d2", "html") + app.add_config_value("d2_args", [], "html") + app.add_config_value("d2_pdf_output_format", "png", "html") + app.add_config_value("d2_pdf_chrome_bin", "", "html") + app.add_config_value("d2_pdf_raster_scale", 2, "html") + app.add_config_value("d2_latex_output_format", "pdf", "html") + app.add_css_file("d2.css") + return {"version": sphinx.__display_version__, "parallel_read_safe": True} diff --git a/docs/userguide/_static/d2.css b/docs/userguide/_static/d2.css new file mode 100644 index 000000000..4c1843afb --- /dev/null +++ b/docs/userguide/_static/d2.css @@ -0,0 +1,52 @@ +.d2 { + display: flex; + justify-content: center; + margin: 2rem 0; + padding: 1rem; + overflow-x: auto; + border-radius: 18px; + background: + radial-gradient(circle at top left, rgb(37 99 235 / 12%), transparent 32rem), + linear-gradient(135deg, #f8fafc 0%, #eef2ff 100%); +} + +.d2 object { + width: min(100%, 1120px); + max-width: 100%; + border: 1px solid #dbeafe; + border-radius: 16px; + background: #fff; + box-shadow: 0 18px 45px rgb(15 23 42 / 14%); + transition: + box-shadow 160ms ease, + transform 160ms ease; +} + +.d2 object:hover { + box-shadow: 0 24px 60px rgb(15 23 42 / 18%); + transform: translateY(-2px); +} + +@media (prefers-color-scheme: dark) { + .d2 { + background: + radial-gradient(circle at top left, rgb(56 189 248 / 18%), transparent 32rem), + linear-gradient(135deg, #0f172a 0%, #111827 100%); + } + + .d2 object { + border-color: rgb(96 165 250 / 35%); + background: #0f172a; + box-shadow: 0 18px 45px rgb(0 0 0 / 30%); + } +} + +@media (prefers-reduced-motion: reduce) { + .d2 object { + transition: none; + } + + .d2 object:hover { + transform: none; + } +} diff --git a/docs/userguide/api_docs/CMakeLists.txt b/docs/userguide/api_docs/CMakeLists.txt index 50b636c4c..18b0cf9d8 100644 --- a/docs/userguide/api_docs/CMakeLists.txt +++ b/docs/userguide/api_docs/CMakeLists.txt @@ -7,14 +7,22 @@ if(LLVM_ENABLE_SPHINX) ${CMAKE_CURRENT_BINARY_DIR} CACHE INTERNAL "") set(DOXYGEN_INDEX_FILE ${DOXYGEN_OUTPUT_DIR}/xml/index.xml) + set(DOXYGEN_HTML_INDEX_FILE ${DOXYGEN_OUTPUT_DIR}/html/index.html) set(DOXYGEN_FILE_IN ${CMAKE_CURRENT_SOURCE_DIR}/Doxyfile.in) set(DOXYGEN_FILE_OUT ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile) configure_file(${DOXYGEN_FILE_IN} ${DOXYGEN_FILE_OUT} @ONLY) file(MAKE_DIRECTORY ${DOXYGEN_OUTPUT_DIR}) - add_custom_target( - eld-api-docs ALL + file(GLOB_RECURSE ELD_API_DOC_INPUTS ${ELD_SOURCE_DIR}/include/eld/PluginAPI/*) + + add_custom_command( + OUTPUT ${DOXYGEN_INDEX_FILE} ${DOXYGEN_HTML_INDEX_FILE} COMMAND ${DOXYGEN_EXECUTABLE} ${DOXYGEN_FILE_OUT} - COMMENT "Generating API documentation with Doxygen") + DEPENDS ${DOXYGEN_FILE_OUT} ${ELD_API_DOC_INPUTS} + COMMENT "Generating API documentation with Doxygen" + VERBATIM) + + add_custom_target(eld-api-docs ALL DEPENDS ${DOXYGEN_INDEX_FILE} + ${DOXYGEN_HTML_INDEX_FILE}) endif() diff --git a/docs/userguide/api_docs/Doxyfile.in b/docs/userguide/api_docs/Doxyfile.in index 19c61e602..cc52c3ed9 100644 --- a/docs/userguide/api_docs/Doxyfile.in +++ b/docs/userguide/api_docs/Doxyfile.in @@ -772,8 +772,7 @@ WARN_LOGFILE = # spaces. See also FILE_PATTERNS and EXTENSION_MAPPING # Note: If this tag is empty the current directory is searched. -INPUT = "@ELD_SOURCE_DIR@/include/eld/PluginAPI" \ - "@CMAKE_BINARY_DIR@/tools/mclinker/include" +INPUT = "@ELD_SOURCE_DIR@/include/eld/PluginAPI" # This tag can be used to specify the character encoding of the source files # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses diff --git a/docs/userguide/conf.py.in b/docs/userguide/conf.py.in index bdd11f653..0d68776c6 100644 --- a/docs/userguide/conf.py.in +++ b/docs/userguide/conf.py.in @@ -17,6 +17,7 @@ THIS_DIR = Path(__file__).resolve().parent REPO_ROOT = (THIS_DIR / ".." / "..").resolve() sys.path.insert(0, str(REPO_ROOT)) +sys.path.insert(0, str(THIS_DIR / "_ext")) project = "ELD" @@ -26,7 +27,7 @@ extensions = [ "sphinx.ext.autodoc", "sphinx.ext.todo", "sphinx.ext.ifconfig", - "sphinx.ext.graphviz", + "eld_d2", ] _BREATHE_XML_DIR = os.environ.get("ELD_BREATHE_XML_DIR") @@ -75,9 +76,15 @@ html_meta = { ), } -# Graphviz configuration -graphviz_dot = "/usr/bin/dot" -graphviz_output_format = "svg" +# D2 diagram configuration +_configured_d2_bin = "@ELD_D2_BIN@" +if not _configured_d2_bin or _configured_d2_bin.startswith("@"): + _configured_d2_bin = "d2" +d2_bin = os.environ.get("D2_BIN", _configured_d2_bin) +d2_args = [] +d2_pdf_output_format = "png" +d2_pdf_chrome_bin = os.environ.get("D2_PDF_CHROME_BIN", "") +d2_pdf_raster_scale = int(os.environ.get("D2_PDF_RASTER_SCALE", "2")) def _read_env_breathe_projects() -> dict[str, str]: """ diff --git a/docs/userguide/documentation/images/Common.d2 b/docs/userguide/documentation/images/Common.d2 new file mode 100644 index 000000000..3fab42725 --- /dev/null +++ b/docs/userguide/documentation/images/Common.d2 @@ -0,0 +1,94 @@ +vars: { + d2-config: { + sketch: true + layout-engine: elk + theme-id: 101 + } +} + +*.style.font-size: 32 + +classes: { + state: { + shape: rectangle + style: { + fill: "#EFF6FF" + stroke: "#2563EB" + stroke-width: 2 + border-radius: 12 + shadow: true + font-color: "#1E3A8A" + } + } + phase: { + shape: rectangle + style: { + fill: "#F8FAFC" + stroke: "#2563EB" + stroke-width: 2 + border-radius: 12 + shadow: true + font-color: "#0F172A" + } + } + component: { + shape: rectangle + style: { + fill: "#F8FAFC" + stroke: "#2563EB" + stroke-width: 2 + border-radius: 12 + shadow: true + font-color: "#0F172A" + } + } + adapter: { + shape: rectangle + style: { + fill: "#ECFEFF" + stroke: "#0891B2" + stroke-width: 2 + border-radius: 12 + shadow: true + font-color: "#164E63" + } + } + callback: { + shape: rectangle + style: { + fill: "#F5F3FF" + stroke: "#7C3AED" + stroke-width: 2 + border-radius: 12 + shadow: true + font-color: "#312E81" + } + } + hook: { + shape: oval + style: { + fill: "#F5F3FF" + stroke: "#7C3AED" + stroke-width: 2 + shadow: true + font-color: "#312E81" + } + } + flow: { + style: { + stroke: "#2563EB" + stroke-width: 3 + animated: true + font-size: 32 + } + } + hook-flow: { + style: { + stroke: "#7C3AED" + stroke-width: 3 + stroke-dash: 5 + animated: true + font-size: 32 + } + } +} diff --git a/docs/userguide/documentation/images/ControlMemorySizePluginFlow.d2 b/docs/userguide/documentation/images/ControlMemorySizePluginFlow.d2 new file mode 100644 index 000000000..1364bc190 --- /dev/null +++ b/docs/userguide/documentation/images/ControlMemorySizePluginFlow.d2 @@ -0,0 +1,12 @@ +...@Common + +direction: right + +create: "Creating\noutput\nsections" { class: phase } +init: "UserPlugin::\nInit" { class: callback } +add_blocks: "UserPlugin::\nAddBlocks" { class: callback } +run: "UserPlugin::\nRun" { class: callback } +destroy: "UserPlugin::\nDestroy" { class: callback } +finalize: "Finalize output\nimage layout" { class: phase } + +create -> init -> add_blocks -> run -> destroy -> finalize { class: flow } diff --git a/docs/userguide/documentation/images/ControlMemorySizePluginFlow.dot b/docs/userguide/documentation/images/ControlMemorySizePluginFlow.dot deleted file mode 100644 index 339b25ce4..000000000 --- a/docs/userguide/documentation/images/ControlMemorySizePluginFlow.dot +++ /dev/null @@ -1,15 +0,0 @@ -digraph G { - node [shape=rectangle] - rankdir="LR"; - A [label="Creating \n output \n sections"] - B [label="UserPlugin::\nInit"] - C [label="UserPlugin:: \n AddBlocks"] - D [label="UserPlugin::\nRun"] - E [label="UserPlugin::\n Destroy"] - F [label="Finalize output \n image layout"] - A -> B - B -> C - C -> D - D -> E - E -> F -} \ No newline at end of file diff --git a/docs/userguide/documentation/images/LinkStates.d2 b/docs/userguide/documentation/images/LinkStates.d2 new file mode 100644 index 000000000..d06ad06f5 --- /dev/null +++ b/docs/userguide/documentation/images/LinkStates.d2 @@ -0,0 +1,10 @@ +...@Common + +direction: right + +initializing: "Initializing" { class: state } +before_layout: "BeforeLayout" { class: state } +creating_sections: "CreatingSections" { class: state } +after_layout: "AfterLayout" { class: state } + +initializing -> before_layout -> creating_sections -> after_layout { class: flow } diff --git a/docs/userguide/documentation/images/LinkStates.dot b/docs/userguide/documentation/images/LinkStates.dot deleted file mode 100644 index 196d09932..000000000 --- a/docs/userguide/documentation/images/LinkStates.dot +++ /dev/null @@ -1,11 +0,0 @@ -digraph G { - rankdir="LR"; - node [shape=rectangle] - A [label="Initializing"] - B [label="BeforeLayout"] - C [label="CreatingSections"] - D [label="AfterLayout"] - A -> B - B -> C - C -> D -} \ No newline at end of file diff --git a/docs/userguide/documentation/images/LinkerPluginFlow.d2 b/docs/userguide/documentation/images/LinkerPluginFlow.d2 new file mode 100644 index 000000000..235b65de0 --- /dev/null +++ b/docs/userguide/documentation/images/LinkerPluginFlow.d2 @@ -0,0 +1,27 @@ +...@Common + +direction: down + +setup: "Setup" { class: phase } +load_plugins: "Load linker\nplugins" { class: phase } +read_inputs: "Read inputs" { class: phase } +rule_matching: "Rule matching" { class: phase } +section_merging: "Section merging" { class: phase } +perform_layout: "Perform layout" { class: phase } +write_output: "Write output" { class: phase } + +init: "Init" { class: hook } +visit_sections: "Visit sections\nand symbols" { class: hook } +before_rule_matching: "BeforeRuleMatching" { class: hook } +before_section_merging: "BeforeSectionMerging" { class: hook } +before_layout: "BeforePerformingLayout" { class: hook } +before_output: "BeforeWritingOutput" { class: hook } + +setup -> load_plugins -> read_inputs -> rule_matching -> section_merging -> perform_layout -> write_output { class: flow } + +load_plugins -> init: "plugin hook" { class: hook-flow } +read_inputs -> visit_sections: "plugin hook" { class: hook-flow } +read_inputs -> before_rule_matching: "plugin hook" { class: hook-flow } +rule_matching -> before_section_merging: "plugin hook" { class: hook-flow } +section_merging -> before_layout: "plugin hook" { class: hook-flow } +perform_layout -> before_output: "plugin hook" { class: hook-flow } diff --git a/docs/userguide/documentation/images/LinkerPluginFlow.dot b/docs/userguide/documentation/images/LinkerPluginFlow.dot deleted file mode 100644 index 399590e72..000000000 --- a/docs/userguide/documentation/images/LinkerPluginFlow.dot +++ /dev/null @@ -1,48 +0,0 @@ -digraph G { - node [shape=rect]; - rankdir=TB; - splines=ortho; - - Setup -> LoadLinkerPlugins; - LoadLinkerPlugins -> in1 [arrowhead=none]; - - in1 [label="", shape=point] - in1 -> ReadInputs; - - ReadInputs -> in3; - in3 [label="", shape=point] - in3 -> RuleMatching; - - RuleMatching -> in4; - in4 [label="", shape=point] - in4 -> SectionMerging; - - SectionMerging -> in5; - in5 [label="", shape=point] - in5 -> PerformLayout; - - PerformLayout -> in6; - in6 [label="", shape=point] - in6 -> WriteOutput; - - node [shape=ellipse]; - edge [minlen=5]; - - {rank=same; in1 Init} - in1 -> Init; - - {rank=same; ReadInputs VisitSectionsAndSymbols} - ReadInputs -> VisitSectionsAndSymbols [minlen=3]; - - {rank=same; in3 BeforeRuleMatching} - in3 -> BeforeRuleMatching; - - {rank=same; in4 BeforeSectionMerging} - in4 -> BeforeSectionMerging; - - {rank=same; in5 BeforePerformingLayout} - in5 -> BeforePerformingLayout; - - {rank=same; in6 BeforeWritingOutput} - in6 -> BeforeWritingOutput; -} \ No newline at end of file diff --git a/docs/userguide/documentation/images/LinkerWrapper.d2 b/docs/userguide/documentation/images/LinkerWrapper.d2 new file mode 100644 index 000000000..b13ec3afe --- /dev/null +++ b/docs/userguide/documentation/images/LinkerWrapper.d2 @@ -0,0 +1,9 @@ +...@Common + +direction: right + +plugin: "Plugin" { class: component } +wrapper: "LinkerWrapper" { class: adapter } +linker: "Linker" { class: component } + +plugin -> wrapper -> linker { class: flow } diff --git a/docs/userguide/documentation/images/LinkerWrapper.dot b/docs/userguide/documentation/images/LinkerWrapper.dot deleted file mode 100644 index c035200f7..000000000 --- a/docs/userguide/documentation/images/LinkerWrapper.dot +++ /dev/null @@ -1,9 +0,0 @@ -digraph G { - rankdir="LR"; - node [shape=rectangle] - A [label="Plugin"] - B [label="LinkerWrapper"] - C [label="Linker"] - A -> B - B -> C -} \ No newline at end of file diff --git a/docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.d2 b/docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.d2 new file mode 100644 index 000000000..c83d6e6a5 --- /dev/null +++ b/docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.d2 @@ -0,0 +1,11 @@ +...@Common + +direction: right + +finalize_layout: "Finalize output\nimage layout" { class: phase } +init: "UserPlugin::Init" { class: callback } +process_output: "Call\nUserPlugin::ProcessOutputSection\nfor each output section" { class: callback } +destroy: "Call\nUserPlugin::Destroy" { class: callback } +emit_image: "Finalize relocations,\nsymbol values and then\nemit output image." { class: phase } + +finalize_layout -> init -> process_output -> destroy -> emit_image { class: flow } diff --git a/docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.dot b/docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.dot deleted file mode 100644 index 6e17226d1..000000000 --- a/docs/userguide/documentation/images/OutputSectionAfterLayoutFlow.dot +++ /dev/null @@ -1,13 +0,0 @@ -digraph G { - node [shape=rectangle] - rankdir="LR"; - A [label="Finalize output \n image layout"] - B [label="UserPlugin::Init"] - C [label="Call \n UserPlugin::ProcessOutputSection \n for each output section"] - D [label="Call \n UserPlugin::Destroy"] - E [label="Finalize relocations, \n symbol values and then \n emit output image."] - A -> B - B -> C - C -> D - D -> E -} \ No newline at end of file diff --git a/docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.d2 b/docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.d2 new file mode 100644 index 000000000..3a9d592c2 --- /dev/null +++ b/docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.d2 @@ -0,0 +1,13 @@ +...@Common + +direction: right + +initialize: "Initialize plugin" { class: phase } +process_input: "Process input\nsections" { class: phase } +init: "UserPlugin::Init(...)" { class: callback } +process_output: "Call UserPlugin::ProcessOutputSection(...)\nfor each output\nsection" { class: callback } +run: "UserPlugin::Run(...)" { class: callback } +destroy: "UserPlugin::Destroy(...)" { class: callback } +merge_sections: "Merge input sections\nbased on rules and\ncreate missing output\nsections" { class: phase } + +initialize -> process_input -> init -> process_output -> run -> destroy -> merge_sections { class: flow } diff --git a/docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.dot b/docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.dot deleted file mode 100644 index cd4fc4843..000000000 --- a/docs/userguide/documentation/images/OutputSectionBeforeLayoutFlow.dot +++ /dev/null @@ -1,16 +0,0 @@ -digraph G { - node [shape=rectangle] - A [label="Initialize plugin"] - B [label="Process input \n sections"] - C [label="UserPlugin::Init(...)"] - D [label="call UserPlugin::ProcessOutputSection(...) \n for each output \n section"] - E [label="UserPlugin::Run(...)"] - F [label="UserPlugin::Destroy(...)"] - G [label="Merge input sections \n based on rules and \n create missing output \n sections"] - A -> B - B -> C - C -> D - D -> E - E -> F - F -> G -} diff --git a/docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.d2 b/docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.d2 new file mode 100644 index 000000000..c05ecddba --- /dev/null +++ b/docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.d2 @@ -0,0 +1,12 @@ +...@Common + +direction: right + +merge_sections: "Merge input sections\nbased on rules and create\nmissing output sections" { class: phase } +init: "UserPlugin::Init(...)" { class: callback } +process_output: "Call\nUserPlugin::ProcessOutputSection(...)\nfor each input section" { class: callback } +run: "UserPlugin::run(...)" { class: callback } +destroy: "UserPlugin::Destroy(...)" { class: callback } +complete: "Complete processing\nof output sections" { class: phase } + +merge_sections -> init -> process_output -> run -> destroy -> complete { class: flow } diff --git a/docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.dot b/docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.dot deleted file mode 100644 index 221354536..000000000 --- a/docs/userguide/documentation/images/OutputSectionCreatingSectionsFlow.dot +++ /dev/null @@ -1,14 +0,0 @@ -digraph G { - node [shape=rectangle] - A [label="Merge input sections \n based on rules and create \n missing output sections"] - B [label="UserPlugin::Init(...)"] - C [label="Call \n UserPlugin::ProcessOutputSection(...) \n for each input section"] - D [label="UserPlugin::run(...)"] - E [label="UserPlugin::Destroy(...)"] - F [label="Complete processing \n of output sections"] - A -> B - B -> C - C -> D - D -> E - E -> F -} diff --git a/docs/userguide/documentation/images/SectionIteratorFlow.d2 b/docs/userguide/documentation/images/SectionIteratorFlow.d2 new file mode 100644 index 000000000..520a52cf0 --- /dev/null +++ b/docs/userguide/documentation/images/SectionIteratorFlow.d2 @@ -0,0 +1,13 @@ +...@Common + +direction: down + +initialize: "Initialize the plugin" { class: phase } +process_files: "Process input files, and perform garbage\ncollection on sections" { class: phase } +init: "UserPlugin::Init(...)" { class: callback } +process_section: "Call\nUserPlugin::ProcessSection(...)\nfor each input section that is\nnot garbage-collected" { class: callback } +run: "UserPlugin::Run(...)" { class: callback } +destroy: "UserPlugin::Destroy(...)" { class: callback } +continue_link: "Continue the linking\nprocess" { class: phase } + +initialize -> process_files -> init -> process_section -> run -> destroy -> continue_link { class: flow } diff --git a/docs/userguide/documentation/images/SectionIteratorFlow.dot b/docs/userguide/documentation/images/SectionIteratorFlow.dot deleted file mode 100644 index 923e77738..000000000 --- a/docs/userguide/documentation/images/SectionIteratorFlow.dot +++ /dev/null @@ -1,16 +0,0 @@ -digraph G { - node [shape=rectangle] - A [label="Initialize the plugin"] - B [label="Process input files, and perform garbage \n collection on sections"] - C [label="UserPlugin::Init(...)"] - D [label="call \n UserPlugin::ProcessSection(...) \n for each input section that is \n not garbage-collected"] - E [label="UserPlugin::Run(...)"] - F [label="UserPlugin::Destroy(...)"] - G [label="Continue the linking \n Process"] - A -> B - B -> C - C -> D - D -> E - E -> F - F -> G -} diff --git a/docs/userguide/documentation/images/SectionMatcherFlow.d2 b/docs/userguide/documentation/images/SectionMatcherFlow.d2 new file mode 100644 index 000000000..1f2e0476a --- /dev/null +++ b/docs/userguide/documentation/images/SectionMatcherFlow.d2 @@ -0,0 +1,13 @@ +...@Common + +direction: right + +rule_matching: "Perform section\nrule-matching" { class: phase } +init: "UserPlugin::Init(...)" { class: callback } +process_section: "Call\nUserPlugin::ProcessSection(...)\nfor each input section" { class: callback } +run: "UserPlugin::run(...)" { class: callback } +destroy: "UserPlugin::Destroy(...)" { class: callback } +garbage_collection: "Perform garbage\ncollection" { class: phase } +continue_link: "Continue the linking\nprocess" { class: phase } + +rule_matching -> init -> process_section -> run -> destroy -> garbage_collection -> continue_link { class: flow } diff --git a/docs/userguide/documentation/images/SectionMatcherFlow.dot b/docs/userguide/documentation/images/SectionMatcherFlow.dot deleted file mode 100644 index 9fb2729f9..000000000 --- a/docs/userguide/documentation/images/SectionMatcherFlow.dot +++ /dev/null @@ -1,16 +0,0 @@ -digraph G { - node [shape=rectangle] - A [label="Perform Section \n rule-matching"] - B [label="UserPlugin::Init(...)"] - C [label="Call \n UserPlugin::ProcessSection(...) \n for each input section"] - D [label="UserPlugin::run(...)"] - E [label="UserPlugin::Destroy(...)"] - F [label="Perform garbage \n collection"] - G [label="Continue the linking \n Process"] - A -> B - B -> C - C -> D - D -> E - E -> F - F -> G -} diff --git a/docs/userguide/documentation/linker_plugins/api_docs.rst b/docs/userguide/documentation/linker_plugins/api_docs.rst index a9da50fa6..0071be2bf 100644 --- a/docs/userguide/documentation/linker_plugins/api_docs.rst +++ b/docs/userguide/documentation/linker_plugins/api_docs.rst @@ -4,7 +4,7 @@ Linker Plugin API Usage and Reference Linker Wrapper ---------------- -.. graphviz:: ../images/LinkerWrapper.dot +.. d2:: ../images/LinkerWrapper.d2 :alt: LinkerWrapper flow. Plugins interacts with the linker using LinkerWrapper. diff --git a/docs/userguide/documentation/linker_plugins/linker_plugins.rst b/docs/userguide/documentation/linker_plugins/linker_plugins.rst index ac3e00759..7d0f1d444 100644 --- a/docs/userguide/documentation/linker_plugins/linker_plugins.rst +++ b/docs/userguide/documentation/linker_plugins/linker_plugins.rst @@ -462,7 +462,7 @@ A sample trace output is shown below:: Link States ================== -.. graphviz:: ../images/LinkStates.dot +.. d2:: ../images/LinkStates.d2 :alt: Link states sequence The link process has different run states, also known as link states. The @@ -557,7 +557,7 @@ For example, :code:`VisitSymbol(eld::plugin::Symbol S)` is called immediately after the symbol is created. -.. graphviz:: ../images/LinkerPluginFlow.dot +.. d2:: ../images/LinkerPluginFlow.d2 :alt: LinkerPlugin flow Section Matcher Interface @@ -579,7 +579,7 @@ cannot be accessed during :code:`SectionMatcherPlugin` run. Link state throughout the plugin run is :code:`eld::plugin::LinkerWrapper::BeforeLayout`. -.. graphviz:: ../images/SectionMatcherFlow.dot +.. d2:: ../images/SectionMatcherFlow.d2 :alt: SectionMatcherPlugin flow. :code:`UserPlugin::Init` is called before :code:`UserPlugin::ProcessSection`, @@ -612,7 +612,7 @@ matching and garbage collection information is accessible during :code:`SectionIteratorPlugin` run. Link state throughout the plugin run is :code:`eld::plugin::LinkerWrapper::BeforeLayout`. -.. graphviz:: ../images/SectionIteratorFlow.dot +.. d2:: ../images/SectionIteratorFlow.d2 :alt: SectionIteratorPlugin flow :code:`UserPlugin::Init` is called before :code:`ProcessSection`, @@ -662,7 +662,7 @@ Different phases of :code:`OutputSectionIteratorPlugin` are described below: BeforeLayout -- Phase 1 ^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. graphviz:: ../images/OutputSectionBeforeLayoutFlow.dot +.. d2:: ../images/OutputSectionBeforeLayoutFlow.d2 :alt: OutputSectionIteratorPlugin flow in BeforeLayout link state This phase is called into action after the linker has performed section @@ -671,7 +671,7 @@ rule-matching and before the linker has performed section merging. CreatingSections -- Phase 2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. graphviz:: ../images/OutputSectionCreatingSectionsFlow.dot +.. d2:: ../images/OutputSectionCreatingSectionsFlow.d2 :alt: OutputSectionIteratorPlugin flow in CreatingSections link state This phase is called into action after the linker has performed section @@ -680,7 +680,7 @@ merging. AfterLayout -- Phase 3 ^^^^^^^^^^^^^^^^^^^^^^^^^ -.. graphviz:: ../images/OutputSectionAfterLayoutFlow.dot +.. d2:: ../images/OutputSectionAfterLayoutFlow.d2 :alt: OutputSectionIteratorPlugin flow in AfterLayout link state. This phase is called into action after the linker has finalized the image @@ -699,7 +699,7 @@ this is an output section plugin, :code:`init` and :code:`destroy` functions are called for each output section the plugin runs on. This plugin is called in the `LinkerWrapper::CreatingSections` step of the linker. -.. graphviz:: ../images/ControlMemorySizePluginFlow.dot +.. d2:: ../images/ControlMemorySizePluginFlow.d2 :alt: ControlMemorySizePlugin Flow :code:`init`, :code:`AddBlocks`, :code:`Run`, :code:`GetBlocks` and :code:`Destroy`