Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions versioned_docs/version-2.x/build/build-based-on-docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,27 @@ docker run --name cbdb-dev -it --rm -h cdw --shm-size=2gb apache/incubator-cloud
docker run --name cbdb-dev -it --rm -h cdw --shm-size=2gb apache/incubator-cloudberry:cbdb-build-rocky9-latest
```
</TabItem>
<TabItem value="rocky-linux-10" label="Rocky Linux 10">
```bash
docker run --name cbdb-dev -it --rm -h cdw --shm-size=2gb apache/incubator-cloudberry:cbdb-build-rocky10-latest
```
</TabItem>
<TabItem value="ubuntu-22" label="Ubuntu 22.04">
```bash
docker run --name cbdb-dev -it --rm -h cdw --shm-size=2gb apache/incubator-cloudberry:cbdb-build-ubuntu22.04-latest
```
</TabItem>
<TabItem value="ubuntu-24" label="Ubuntu 24.04">
```bash
docker run --name cbdb-dev -it --rm -h cdw --shm-size=2gb apache/incubator-cloudberry:cbdb-build-ubuntu24.04-latest
```
</TabItem>
</Tabs>

:::note
Rocky Linux 10 and Ubuntu 24.04 are supported starting from Apache Cloudberry 2.2.
:::

Explaination:
- `--name=cbdb-dev` will set the name of the container to `cbdb-dev`. You can change it to any name you prefer.
- `-it` will run the container in interactive mode, allowing you to execute commands inside it.
Expand Down Expand Up @@ -114,6 +128,10 @@ cd /home/gpadmin/cloudberry
export SRC_DIR=/home/gpadmin/cloudberry
mkdir -p ${SRC_DIR}/build-logs

# Since Cloudberry 2.2
# Pre-stage the Python packages for `--with-pythonsrc-ext` before building
make -C gpMgmt/bin download-python-deps

## Configure, build, and install Cloudberry
./devops/build/automation/cloudberry/scripts/configure-cloudberry.sh
./devops/build/automation/cloudberry/scripts/build-cloudberry.sh
Expand Down
110 changes: 109 additions & 1 deletion versioned_docs/version-2.x/build/configure.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,33 @@ title: Configure Apache Cloudberry Build
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

## Pre-stage the Python packages for `--with-pythonsrc-ext`

:::note
This step is only required starting from Apache Cloudberry 2.2.
:::

The Python source packages used by `--with-pythonsrc-ext` are not bundled in the repository. They have been removed since Apache Cloudberry 2.0 to comply with the Apache release policy, and used to be downloaded on the fly during the build. Starting from Apache Cloudberry 2.2, they are downloaded ahead of the build instead, so pre-stage them before you run `configure`.

Building PyYAML requires Cython earlier than 3.0. On Ubuntu, install the matching distribution package first:

```bash
# For Ubuntu 22.04, cython3 is 0.29.x
sudo apt install -y cython3

# For Ubuntu 24.04, cython3 is 3.x and cannot build PyYAML, so use cython3-legacy instead
sudo apt install -y cython3-legacy
```

On Rocky Linux 8 and 9, `python3-Cython` has already been installed in [Install required packages](./install-required-packages). On Rocky Linux 10, the distribution only ships Cython 3.x, so the following command installs a compatible version through `pip3` for you.

Then download the packages:

```bash
cd ~/cloudberry
make -C gpMgmt/bin download-python-deps
```

## Configure the build process

<Tabs>
Expand All @@ -28,6 +55,44 @@ sudo chown -R gpadmin:gpadmin /usr/local/cloudberry-db

The `configure` command sets up the build environment for Apache Cloudberry. This configuration includes several development features and extensions.

:::note
Starting from Apache Cloudberry 2.2, new extensions are available, including `diskquota`, `gp_stats_collector`, and `yezzey`. Use the corresponding `configure` options to enable them.
:::

<Tabs>
<TabItem value="cloudberry-2.2" label="Cloudberry 2.2" default>
```bash
cd ~/cloudberry
export LD_LIBRARY_PATH=/usr/local/cloudberry-db/lib:${LD_LIBRARY_PATH:-""}
./configure --prefix=/usr/local/cloudberry-db \
--disable-external-fts \
--enable-gpcloud \
--enable-ic-proxy \
--enable-mapreduce \
--enable-orafce \
--enable-orca \
--enable-pax \
--disable-pxf \
--enable-tap-tests \
--with-diskquota \
--with-gp-stats-collector \
--with-gssapi \
--with-ldap \
--with-libxml \
--with-lz4 \
--with-pam \
--with-perl \
--with-pgport=5432 \
--with-python \
--with-pythonsrc-ext \
--with-ssl=openssl \
--with-uuid=e2fs \
--with-yezzey \
--with-includes=/usr/local/xerces-c/include \
--with-libraries=/usr/local/cloudberry-db/lib
```
</TabItem>
<TabItem value="cloudberry-2.0-2.1" label="Cloudberry 2.0/2.1">
```bash
cd ~/cloudberry
export LD_LIBRARY_PATH=/usr/local/cloudberry-db/lib:${LD_LIBRARY_PATH:-""}
Expand Down Expand Up @@ -56,7 +121,9 @@ export LD_LIBRARY_PATH=/usr/local/cloudberry-db/lib:${LD_LIBRARY_PATH:-""}
--with-libraries=/usr/local/cloudberry-db/lib
```
</TabItem>
<TabItem value="ubuntu-linux" label="For Ubuntu 20.04+" default>
</Tabs>
</TabItem>
<TabItem value="ubuntu-linux" label="For Ubuntu 22.04+">

### Prepare environment

Expand All @@ -73,6 +140,12 @@ sudo chown -R gpadmin:gpadmin /usr/local/cloudberry-db

The `configure` command sets up the build environment for Apache Cloudberry. This configuration includes several development features and extensions.

:::note
Starting from Apache Cloudberry 2.2, new extensions are available, including `diskquota`, `gp_stats_collector`, and `yezzey`. Use the corresponding `configure` options to enable them.
:::

<Tabs>
<TabItem value="cloudberry-2.2" label="Cloudberry 2.2" default>
```bash
cd ~/cloudberry
./configure --prefix=/usr/local/cloudberry-db \
Expand All @@ -85,6 +158,8 @@ cd ~/cloudberry
--enable-pax \
--disable-pxf \
--enable-tap-tests \
--with-diskquota \
--with-gp-stats-collector \
--with-gssapi \
--with-ldap \
--with-libxml \
Expand All @@ -96,9 +171,39 @@ cd ~/cloudberry
--with-pythonsrc-ext \
--with-ssl=openssl \
--with-uuid=e2fs \
--with-yezzey \
--with-includes=/usr/include/xercesc
```
</TabItem>
<TabItem value="cloudberry-2.0-2.1" label="Cloudberry 2.0/2.1">
```bash
cd ~/cloudberry
./configure --prefix=/usr/local/cloudberry-db \
--disable-external-fts \
--enable-gpcloud \
--enable-ic-proxy \
--enable-mapreduce \
--enable-orafce \
--enable-orca \
--enable-pax \
--disable-pxf \
--enable-tap-tests \
--with-gssapi \
--with-ldap \
--with-libxml \
--with-lz4 \
--with-pam \
--with-perl \
--with-pgport=5432 \
--with-python \
--with-pythonsrc-ext \
--with-ssl=openssl \
--with-uuid=e2fs \
--with-includes=/usr/include/xercesc
```
</TabItem>
</Tabs>
</TabItem>
</Tabs>

## `configure` options
Expand Down Expand Up @@ -144,3 +249,6 @@ Also, some packages names vary between different Linux distributions.
| `--with-libxml` | Build with libxml2, enabling SQL/XML support.|This requires libxml2 to be installed.|
| `--with-lz4` | Build with LZ4 compression support |This allows the use of LZ4 for compression of table data and lz4 library is required to be installed.|
| `--with-ssl=LIB` | Build with support for SSL (encrypted) connections. | The only LIBRARY supported is openssl, so `--with-ssl=openssl` is used in this guide. This requires the OpenSSL package to be installed. |
| `--with-diskquota` | Build with diskquota extension. | Diskquota is an extension that provides disk usage enforcement for database objects in Apache Cloudberry. **Since Cloudberry 2.2**|
| `--with-gp-stats-collector` | Build with stats collector extension. | An extension for collecting query execution metrics and reporting them to an external agent. **Since Cloudberry 2.2**|
| `--with-yezzey` | Build with Yezzey extension. | Yezzey is an extension for offloading data from Cloudberry to S3-compatible external storage. This option builds the extension only. You do not need a proxy if there are only a few requests to S3, but many parallel data exchange streams can exhaust CPU and network on the cluster host. For better performance under heavy traffic, additionally deploy [YProxy](https://github.com/open-gpdb/yproxy), which pools connections to S3 and schedules the requests. **Since Cloudberry 2.2**|
4 changes: 2 additions & 2 deletions versioned_docs/version-2.x/build/create-gpadmin-user.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ If `sudo` & `git` command is not already installed, run the following command to
dnf install -y sudo git
```
</TabItem>
<TabItem value="ubuntu" label="For Ubuntu 20.04+" default>
<TabItem value="ubuntu" label="For Ubuntu 22.04+" default>
```bash
apt update && apt install -y sudo git
```
Expand Down Expand Up @@ -63,7 +63,7 @@ This script performs three main tasks as the `gpadmin` user:
The script uses a heredoc (EOF) block to execute multiple commands under the `gpadmin` user context. This will be used multiple time throughout these instructions.

<Tabs>
<TabItem value="cloudberry-2.1.0" label="Apache Cloudberry 2.1.0" default>
<TabItem value="cloudberry-2.1.0" label="Apache Cloudberry 2.1.0+" default>
```bash
sudo -u gpadmin bash <<'EOF'
# Add Cloudberry environment setup to .bashrc
Expand Down
35 changes: 19 additions & 16 deletions versioned_docs/version-2.x/build/download-source-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,44 +19,40 @@ git clone https://github.com/apache/cloudberry.git ~/cloudberry
cd ~/cloudberry
git fetch --tags

# For Apache Cloudberry 2.0.0
git checkout tags/2.0.0-incubating
# For Apache Cloudberry 2.2.0
git checkout tags/2.2.0-incubating

# For Apache Cloudberry 2.1.0
git checkout tags/2.1.0-incubating

# For Apache Cloudberry 2.0.0
git checkout tags/2.0.0-incubating

git submodule update --init --recursive
```

:::note
The command `git submodule update --init --recursive` is used to initialize submodules for building with PAX support in this guide. If you don't plan to build with PAX support, you can skip this step.
The command `git submodule update --init --recursive` initializes the submodules that the build depends on. In Apache Cloudberry 2.0 and 2.1, these are only needed for building with PAX support, so you can skip this step if you don't plan to enable PAX. Starting from Apache Cloudberry 2.2, other components are shipped as submodules too, such as `gpcontrib/yezzey` for `--with-yezzey`, so run this command unless you are sure you need none of them.
:::

:::caution

In the Ubuntu container, you may encounter the following error when clone the source code:
In the Ubuntu container, you may encounter the following error when cloning the source code: `error: git-remote-https died of signal 4`.

```
error: git-remote-https died of signal 4
```

You can set the following environment variable to avoid this error:
You can set the following environment variable to avoid this error: `export GNUTLS_CPUID_OVERRIDE=0x1`.

```bash
export GNUTLS_CPUID_OVERRIDE=0x1
```
:::

## Download the source code archive

Alternatively, you can download the source code archive from the [Apache Cloudberry releases page](/releases).


- For Apache Cloudberry 2.0.0
- For Apache Cloudberry 2.2.0

```bash
tar xvzf apache-cloudberry-2.0.0-incubating-src.tar.gz
mv apache-cloudberry-2.0.0-incubating cloudberry
tar xvzf apache-cloudberry-2.2.0-incubating-src.tar.gz
mv apache-cloudberry-2.2.0-incubating cloudberry
```

- For Apache Cloudberry 2.1.0
Expand All @@ -66,6 +62,13 @@ tar xvzf apache-cloudberry-2.1.0-incubating-src.tar.gz
mv apache-cloudberry-2.1.0-incubating cloudberry
```

- For Apache Cloudberry 2.0.0

```bash
tar xvzf apache-cloudberry-2.0.0-incubating-src.tar.gz
mv apache-cloudberry-2.0.0-incubating cloudberry
```

:::note
The submodules are already included in the latest 2.x.0 release source code archive, so you don't need to download the submodules manually after extracting the archive.
:::
:::
18 changes: 11 additions & 7 deletions versioned_docs/version-2.x/build/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,20 +35,24 @@ For those new to Cloudberry or PostgreSQL, we recommend starting with a PostgreS

## Prerequisites

### Provision a Rocky Linux 8+ / Ubuntu 20.04+ Environment
### Provision a Rocky Linux 8+ / Ubuntu 22.04+ Environment

:::note
Since Apache Cloudberry 2.2, Ubuntu 20.04 support has been dropped, and support for Rocky Linux 10 and Ubuntu 24.04 has been added.
:::

- Use any platform to create a virtual machine or container:

- **Cloud providers**: You can use the Rocky Linux 8+ or Ubuntu 20.04+ images provided by the cloud providers, such as AWS, Google Cloud, Microsoft Azure, and more.
- **VirtualBox**: Use the official [Rocky Linux 8+](https://rockylinux.org/download) / [Ubuntu 20.04+](https://ubuntu.com/download) ISO or Vagrant boxes.
- **Docker**: These instructions were validated under Rocky Linux 8+ and Ubuntu 20.04 official base docker images, but should work with any of their based container.
- **Cloud providers**: You can use the Rocky Linux 8+ or Ubuntu 22.04+ images provided by the cloud providers, such as AWS, Google Cloud, Microsoft Azure, and more.
- **VirtualBox**: Use the official [Rocky Linux 8+](https://rockylinux.org/download) / [Ubuntu 22.04+](https://ubuntu.com/download) ISO or Vagrant boxes.
- **Docker**: These instructions were validated under Rocky Linux 8+ and Ubuntu 22.04 official base docker images, but should work with any of their based container.
- For example, you can run the following command to start a Rocky Linux 8 container:

```bash
docker run -it --shm-size=2gb -h cdw rockylinux/rockylinux:8

# Start a Ubuntu 20.04 container:
# docker run -it --shm-size=2gb -h cdw ubuntu:20.04
# Start a Ubuntu 22.04 container:
# docker run -it --shm-size=2gb -h cdw ubuntu:22.04
```

The hostname `cdw` (Coordinator Data Warehouse) is just an example of how we started the container for testing.
Expand Down Expand Up @@ -76,7 +80,7 @@ Minimum requirements for development environment:

## Build Apache Cloudberry from source code

The following steps guide you through building Apache Cloudberry from source code on Rocky Linux 8+ or Ubuntu 20.04+. The process is similar for both operating systems, with minor differences in package management, dependencies and software versions between these two distributions.
The following steps guide you through building Apache Cloudberry from source code on Rocky Linux 8+ or Ubuntu 22.04+. The process is similar for both operating systems, with minor differences in package management, dependencies and software versions between these two distributions.

Just go ahead and follow the steps below to build Apache Cloudberry from source code:

Expand Down
Loading
Loading