Skip to content

Update release policy page #688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
HyukjinKwon wants to merge 1 commit into
base: asf-site
Choose a base branch
from
+107 −39
Draft

Update release policy page #688

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 67 additions & 20 deletions site/versioning-policy.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,19 +164,31 @@ <h3>Spark versions</h3>
<p>Each Spark release will be versioned: <code class="language-plaintext highlighter-rouge">[MAJOR].[FEATURE].[MAINTENANCE]</code></p>

<ul>
<li><strong>MAJOR</strong>: All releases with the same major version number will have API compatibility.
Major version numbers will remain stable over long periods of time. For instance, 1.X.Y may last
1 year or more.</li>
<li><strong>FEATURE</strong>: Feature releases will typically contain new features, improvements, and bug fixes.
Each feature release will have a merge window where new patches can be merged, a QA window when
<li><strong>MAJOR</strong>: Major releases occur annually, third-party dependency
upgrades, and major code refactoring. All releases with the same major version number will have
Copy link
Copy Markdown
Contributor

@cloud-fan cloud-fan May 14, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

major code refactoring does not need to wait for the yearly major release, we should mention breaking changes here instead.

API compatibility.</li>
<li><strong>FEATURE</strong>: Feature releases occur quarterly (every 3 months) and contain new features, performance
improvements, API additions, and bug fixes. To ensure safe and predictable upgrades for downstream
projects, feature releases have the following compatibility:
<ul>
<li>No third-party dependency upgrades (e.g. Parquet, Arrow, ORC, Hadoop, Netty).</li>
<li>No behavior or semantic changes (e.g. SQL semantics, execution behavior, optimizer behavior,
configuration defaults). Exceptions might occur case by case (e.g., security issues).</li>
<li>Public APIs may be added but not changed or removed.</li>
</ul>
</li>
</ul>

<p>Each feature release will have a merge window where new patches can be merged, a QA window when
only fixes can be merged, then a final period where voting occurs on release candidates. These
windows will be announced immediately after the previous feature release to give people plenty
of time, and over time, we might make the whole release process more regular (similar to Ubuntu).</li>
<li><strong>MAINTENANCE</strong>: Maintenance releases will occur more frequently and depend on specific patches
introduced (e.g. bug fixes) and their urgency. In general these releases are designed to patch bugs.
However, higher level libraries may introduce small features, such as a new algorithm, provided
they are entirely additive and isolated from existing code paths. Spark core may not introduce
any features.</li>
of time.</p>
<ul>
<li><strong>MAINTENANCE</strong>: Maintenance releases will occur on an ad hoc basis depending on specific patches
introduced (e.g. critical bug fixes and security patches) and their urgency. In general these releases
are designed to patch bugs. However, higher level libraries may introduce small features, such as a
Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • First, what is the definition of higher level libraries? In the context, this could mean every module except Spark Core. In other words, SQL, MLLIB, GRAPHX and so on.

  • Second, why do we need to allow this in the maintenance releases? I believe we had better keep the policy simple. In other words, no new features at all.

However, higher level libraries may introduce small features, such as a new algorithm, provided they are entirely additive and isolated from existing code paths.

new algorithm, provided they are entirely additive and isolated from existing code paths. Spark core
may not introduce any features.</li>
</ul>

<h3>Alpha components</h3>
Expand Down Expand Up @@ -274,9 +286,10 @@ <h4>Alternatives to breaking an API</h4>

<h2>Release cadence</h2>

<p>The branch is cut every January and July, so feature (&#8220;minor&#8221;) releases occur about every 6 months in general.
Hence, Spark 2.3.0 would generally be released about 6 months after 2.2.0. Maintenance releases happen as needed
in between feature releases. Major releases do not happen according to a fixed schedule.</p>
<p>Starting with Spark 4.3, feature releases occur quarterly (every 3 months), containing new features,
improvements, and bug fixes. Major releases occur annually (every 12 months), allowing breaking
changes and dependency upgrades. Maintenance releases happen as needed in between for critical
bug fixes and security patches.</p>

<h3>Spark 4.2 release window</h3>

Expand Down Expand Up @@ -305,14 +318,48 @@ <h3>Spark 4.2 release window</h3>

<h2>Maintenance releases and EOL</h2>

<p>Feature release branches will, generally, be maintained with bug fix releases for a period of 18 months.
For example, branch 2.3.x is no longer considered maintained as of September 2019, 18 months after the release
of 2.3.0 in February 2018. No more 2.3.x releases should be expected after that point, even for bug fixes.</p>
<p>The following table summarizes the maintenance window for each release type:</p>

<table>
<thead>
<tr>
<th>Release Type</th>
<th>Cadence</th>
<th>Maintenance Window</th>
</tr>
</thead>
<tbody>
<tr>
<td>Feature (x.y)</td>
<td>Every 3 months</td>
<td>6 months</td>
</tr>
<tr>
<td>LTS (final feature of a major)</td>
Copy link
Copy Markdown
Contributor

@cloud-fan cloud-fan May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we make it explicit that every x.3 release is the LTS?

Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this PR already mentions two exceptions like the following, it would be great to mention more specifically by giving examples like 5.3 or 6.3 in addition to every x.3 release.

As an exception from the normal versioning policy, version 3.5.x has an ...

For example, Spark 4.5 (the final 4.x feature release) would be maintained for 18 months from its release date.

<td>Every 12 months</td>
<td>18 months</td>
</tr>
<tr>
<td>Maintenance (x.y.z)</td>
<td>Ad hoc</td>
<td>N/A (patches only)</td>
</tr>
</tbody>
</table>

<p>Non-LTS feature release branches will, generally, be maintained with bug fix releases for a period of
6 months.</p>

<p>The final feature release within a major release will be designated as the &#8220;LTS&#8221; (Long-Term Support) release
and will be maintained for 18 months. For example, Spark 4.5 (the final 4.x feature release) would be
maintained for 18 months from its release date. LTS releases provide a stable target for ecosystem
projects and downstream vendors to standardize around.</p>

<p>The last minor release within a major release will typically be maintained for longer as an &#8220;LTS&#8221; release.
For example, 3.5.0 was released on September 13th 2023 and would normally be maintained for 31 months until April 12th 2026.</p>
<p>Critical security patches will be backported to all actively maintained branches. Critical bug fixes
(e.g., correctness issues) that may introduce behavior changes will be evaluated by the community
on a case-by-case basis.</p>

<p>As an exception from the normal versioning policy, version 3.5.x has an &#8220;extended&#8221; LTS period to allow for migrations to be completed. This extended LTS period will end <em>November 2027</em>. During the 3.5.x extended LTS period, we will only include security fixes. This extend LTS only applies to the primary Apache Spark project/repo and does not apply to sub projects with separate repos/releases (namely: Spark Connect for Swift/Rust/Go and Spark Kubernetes operator). Additionally, as Java 8 support may be removed from other projects (including Hadoop), should a dependency have a security fix that is not backported to a Java 8 compatible version we may decide to mark that vulnerability as a won&#8217;t fix or release the new version without Java 8 support.</p>
<p>As an exception from the normal versioning policy, version 3.5.x has an &#8220;extended&#8221; LTS period to allow for migrations to be completed. This extended LTS period will end <em>November 2027</em>. During the 3.5.x extended LTS period, we will only include security fixes. This extended LTS only applies to the primary Apache Spark project/repo and does not apply to sub projects with separate repos/releases (namely: Spark Connect for Swift/Rust/Go and Spark Kubernetes operator). Additionally, as Java 8 support may be removed from other projects (including Hadoop), should a dependency have a security fix that is not backported to a Java 8 compatible version we may decide to mark that vulnerability as a won&#8217;t fix or release the new version without Java 8 support.</p>

</div>
<div class="col-12 col-md-3">
Expand Down
59 changes: 40 additions & 19 deletions versioning-policy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,26 @@ These small differences account for Spark's nature as a multi-module project.

Each Spark release will be versioned: `[MAJOR].[FEATURE].[MAINTENANCE]`

- **MAJOR**: All releases with the same major version number will have API compatibility.
Major version numbers will remain stable over long periods of time. For instance, 1.X.Y may last
1 year or more.
- **FEATURE**: Feature releases will typically contain new features, improvements, and bug fixes.
- **MAJOR**: Major releases occur annually, third-party dependency
upgrades, and major code refactoring. All releases with the same major version number will have
API compatibility.
- **FEATURE**: Feature releases occur quarterly (every 3 months) and contain new features, performance
improvements, API additions, and bug fixes. To ensure safe and predictable upgrades for downstream
projects, feature releases have the following compatibility:
- No third-party dependency upgrades (e.g. Parquet, Arrow, ORC, Hadoop, Netty).
- No behavior or semantic changes (e.g. SQL semantics, execution behavior, optimizer behavior,
configuration defaults). Exceptions might occur case by case (e.g., security issues).
- Public APIs may be added but not changed or removed.

Each feature release will have a merge window where new patches can be merged, a QA window when
only fixes can be merged, then a final period where voting occurs on release candidates. These
windows will be announced immediately after the previous feature release to give people plenty
of time, and over time, we might make the whole release process more regular (similar to Ubuntu).
- **MAINTENANCE**: Maintenance releases will occur more frequently and depend on specific patches
introduced (e.g. bug fixes) and their urgency. In general these releases are designed to patch bugs.
However, higher level libraries may introduce small features, such as a new algorithm, provided
they are entirely additive and isolated from existing code paths. Spark core may not introduce
any features.
of time.
- **MAINTENANCE**: Maintenance releases will occur on an ad hoc basis depending on specific patches
introduced (e.g. critical bug fixes and security patches) and their urgency. In general these releases
are designed to patch bugs. However, higher level libraries may introduce small features, such as a
new algorithm, provided they are entirely additive and isolated from existing code paths. Spark core
may not introduce any features.

<h3>Alpha components</h3>

Expand Down Expand Up @@ -96,9 +103,10 @@ In cases where there is a "Bad API", but where the cost of removal is also high,

<h2>Release cadence</h2>

The branch is cut every January and July, so feature ("minor") releases occur about every 6 months in general.
Hence, Spark 2.3.0 would generally be released about 6 months after 2.2.0. Maintenance releases happen as needed
in between feature releases. Major releases do not happen according to a fixed schedule.
Starting with Spark 4.3, feature releases occur quarterly (every 3 months), containing new features,
improvements, and bug fixes. Major releases occur annually (every 12 months), allowing breaking
changes and dependency upgrades. Maintenance releases happen as needed in between for critical
bug fixes and security patches.

<h3>Spark 4.2 release window</h3>

Expand All @@ -110,11 +118,24 @@ in between feature releases. Major releases do not happen according to a fixed s

<h2>Maintenance releases and EOL</h2>

Feature release branches will, generally, be maintained with bug fix releases for a period of 18 months.
For example, branch 2.3.x is no longer considered maintained as of September 2019, 18 months after the release
of 2.3.0 in February 2018. No more 2.3.x releases should be expected after that point, even for bug fixes.
The following table summarizes the maintenance window for each release type:

| Release Type | Cadence | Maintenance Window |
| ----- | ----- | ----- |
Copy link
Copy Markdown
Contributor

@peter-toth peter-toth May 14, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need Major type with version (x.0.0) here?

| Feature (x.y) | Every 3 months | 6 months |
Copy link
Copy Markdown
Contributor

@peter-toth peter-toth May 14, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we use version (x.1+.0) for Feature and (x.y.1+) for Maintenance?

| LTS (final feature of a major) | Every 12 months | 18 months |
| Maintenance (x.y.z) | Ad hoc | N/A (patches only) |

Non-LTS feature release branches will, generally, be maintained with bug fix releases for a period of
6 months.

The final feature release within a major release will be designated as the "LTS" (Long-Term Support) release
and will be maintained for 18 months. For example, Spark 4.5 (the final 4.x feature release) would be
maintained for 18 months from its release date. LTS releases provide a stable target for ecosystem
projects and downstream vendors to standardize around.

The last minor release within a major release will typically be maintained for longer as an "LTS" release.
For example, 3.5.0 was released on September 13th 2023 and would normally be maintained for 31 months until April 12th 2026.
Critical security patches will be backported to all actively maintained branches. Critical bug fixes
(e.g., correctness issues) that may introduce behavior changes will be evaluated by the community
on a case-by-case basis.

As an exception from the normal versioning policy, version 3.5.x has an "extended" LTS period to allow for migrations to be completed. This extended LTS period will end *November 2027*. During the 3.5.x extended LTS period, we will only include security fixes. This extend LTS only applies to the primary Apache Spark project/repo and does not apply to sub projects with separate repos/releases (namely: Spark Connect for Swift/Rust/Go and Spark Kubernetes operator). Additionally, as Java 8 support may be removed from other projects (including Hadoop), should a dependency have a security fix that is not backported to a Java 8 compatible version we may decide to mark that vulnerability as a won't fix or release the new version without Java 8 support.
As an exception from the normal versioning policy, version 3.5.x has an "extended" LTS period to allow for migrations to be completed. This extended LTS period will end *November 2027*. During the 3.5.x extended LTS period, we will only include security fixes. This extended LTS only applies to the primary Apache Spark project/repo and does not apply to sub projects with separate repos/releases (namely: Spark Connect for Swift/Rust/Go and Spark Kubernetes operator). Additionally, as Java 8 support may be removed from other projects (including Hadoop), should a dependency have a security fix that is not backported to a Java 8 compatible version we may decide to mark that vulnerability as a won't fix or release the new version without Java 8 support.
Loading