feat: add support for tag tracking

documentation/docs/getting-started/configuration.mdx

@@ -13,6 +13,7 @@ OPAL supports multiple ways to configure your deployment:
 Let's get OPAL running! You'll need to set a few environment variables that tell OPAL where to find your policies and how to connect everything together.
 
 #### OPAL Server
+
 ```bash
 # Where your policies are stored (Git repo)
 export OPAL_POLICY_REPO_URL=https://github.com/your-org/policy-repo.git
@@ -33,6 +34,7 @@ export OPAL_BROADCAST_URI=postgres://postgres:postgres@localhost:5432/postgres
 ```
 
 #### OPAL Client
+
 ```bash
 # Connect to your OPAL server
 export OPAL_SERVER_URL=http://localhost:7002
@@ -646,9 +648,17 @@ Set if OPAL server should use a fixed clone path (and reuse if it already exists
 
 #### OPAL_POLICY_REPO_MAIN_BRANCH
 
-Default: `master`
+Default: `None`
 
-The main branch to track for policy files.
+The git branch to track for policy files. If neither `OPAL_POLICY_REPO_MAIN_BRANCH` nor `OPAL_POLICY_REPO_TAG` is set, defaults to `master`. Cannot be used together with `OPAL_POLICY_REPO_TAG`.
+
+For more information, see [tracking a Git repository](/tutorials/track_a_git_repo).
+
+#### OPAL_POLICY_REPO_TAG
+
+Default: `None`
+
+The git tag to track for policy files. When set, OPAL will track the specified tag instead of a branch. This is useful for pinning policies to a specific release. Cannot be used together with `OPAL_POLICY_REPO_MAIN_BRANCH`.
 
 For more information, see [tracking a Git repository](/tutorials/track_a_git_repo).
 
@@ -783,6 +793,7 @@ Enforce webhook events only from a specific branch.
 #### OPAL_POLICY_REPO_WEBHOOK_PARAMS
 
 Default:
+
 ```json
 {
   "secret_header_name": "x-hub-signature-256",
@@ -846,7 +857,7 @@ Default: `[]`
 **DEPRECATED** - Use [OPAL_BUNDLE_IGNORE](#opal_bundle_ignore) instead.
 :::
 
-Paths to omit from policy bundle. List of glob-style paths, or paths without wildcards but ending with "/**" indicating a parent path (ignoring all under it).
+Paths to omit from policy bundle. List of glob-style paths, or paths without wildcards but ending with "/\*\*" indicating a parent path (ignoring all under it).
 
 _Added in OPAL v0.5.0_
 
@@ -885,13 +896,16 @@ Default route for data callbacks (exists as a sane default in case the user did
 #### OPAL_DATA_CONFIG_SOURCES
 
 Default:
+
 ```json
 {
   "config": {
-    "entries": [{
-      "url": "http://localhost:7002/policy-data",
-      "topics": ["data"]
-    }]
+    "entries": [
+      {
+        "url": "http://localhost:7002/policy-data",
+        "topics": ["data"]
+      }
+    ]
   }
 }
 ```
@@ -1098,7 +1112,7 @@ Retry options when connecting to the policy store (i.e., the agent that handles
 
 Default: `[]`
 
-Which policy paths pushed to the client should be ignored. List of glob-style paths, or paths without wildcards but ending with "/**" indicating a parent path (ignoring all under it). Supports paths starting with '!' to force not ignoring them: a negated path always takes precedence.
+Which policy paths pushed to the client should be ignored. List of glob-style paths, or paths without wildcards but ending with "/\*\*" indicating a parent path (ignoring all under it). Supports paths starting with '!' to force not ignoring them: a negated path always takes precedence.
 
 #### OPAL_OPA_HEALTH_CHECK_POLICY_ENABLED
 
@@ -1223,6 +1237,7 @@ For more information, see [tracking a Git repository](/tutorials/track_a_git_rep
 #### OPAL_POLICY_UPDATER_CONN_RETRY
 
 Default:
+
 ```json
 {
   "wait_strategy": "random_exponential",
@@ -1275,6 +1290,7 @@ For more information, see [monitoring OPAL](/tutorials/monitoring_opal).
 #### OPAL_DEFAULT_UPDATE_CALLBACK_CONFIG
 
 Default:
+
 ```json
 {
   "method": "POST",
@@ -1300,6 +1316,7 @@ For more information, see [monitoring OPAL](/tutorials/monitoring_opal).
 #### OPAL_DATA_UPDATER_CONN_RETRY
 
 Default:
+
 ```json
 {
   "wait_strategy": "random_exponential",
@@ -1324,6 +1341,7 @@ _Added in OPAL v0.7.5_
 ## Advanced Configuration Options
 
 #### CLI Help
+
 ```bash
 # View all options
 opal-server --help
@@ -1335,12 +1353,15 @@ opal-client run --help
 ```
 
 #### Kubernetes Deployment
+
 For Kubernetes deployments, see [OPAL Helm Chart for Kubernetes](/tutorials/helm-chart-for-kubernetes).
 
 #### Proxy Configuration
+
 For proxy configurations, see [Setup OPAL Behind a Proxy](/tutorials/setup_opal_behind_proxy).
 
 #### Source Code
+
 - [Common config](https://github.com/permitio/opal/blob/master/packages/opal-common/opal_common/config.py) - Shared variables
 - [Server config](https://github.com/permitio/opal/blob/master/packages/opal-server/opal_server/config.py) - Server-specific variables
 - [Client config](https://github.com/permitio/opal/blob/master/packages/opal-client/opal_client/config.py) - Client-specific variables

documentation/docs/getting-started/running-opal/as-python-package/opal-server-setup.mdx

@@ -90,9 +90,11 @@ a [Github SSH key here](https://docs.github.com/en/github/authenticating-to-gith
 
 The value you pass for the `POLICY_REPO_SSH_KEY` can either be a file path, or the contents of the SSH-key - with newlines replaced with `\_`.
 
-#### `OPAL_POLICY_REPO_CLONE_PATH` & `OPAL_POLICY_REPO_MAIN_BRANCH`
+#### `OPAL_POLICY_REPO_CLONE_PATH`, `OPAL_POLICY_REPO_MAIN_BRANCH` & `OPAL_POLICY_REPO_TAG`
 
-These will allow you to control how the repo is cloned.
+These will allow you to control how the repo is cloned. By default OPAL will track the `master` branch of the repo, you may optionally track another branch or a tag in the repo.
+
+You must choose between tracking a branch or a tag, OPAL will fail if you try to supply both `OPAL_POLICY_REPO_MAIN_BRANCH` and `OPAL_POLICY_REPO_TAG`.
 
 ### Simple run with Data source configuration
 

documentation/docs/getting-started/running-opal/as-python-package/overview.mdx

@@ -185,7 +185,9 @@ The value you pass for the `POLICY_REPO_SSH_KEY` can either be a file path, or t
 
 ##### `OPAL_POLICY_REPO_CLONE_PATH` & `OPAL_POLICY_REPO_MAIN_BRANCH`
 
-These will allow you to control how the repo is cloned.
+These will allow you to control how the repo is cloned. By default OPAL will track the `master` branch of the repo, you may optionally track another branch or a tag in the repo.
+
+You must choose between tracking a branch or a tag, OPAL will fail if you try to supply both `OPAL_POLICY_REPO_MAIN_BRANCH` and `OPAL_POLICY_REPO_TAG`.
 
 #### Simple run with Data source configuration
 

documentation/docs/getting-started/running-opal/run-opal-server/policy-repo-location.mdx

@@ -87,7 +87,13 @@ For these config vars, in most cases you are good with the default values:
     <tr>
       <td valign="top">OPAL_POLICY_REPO_MAIN_BRANCH</td>
       <td>
-        Name of the git branch to track for policy files (default: `master`)
+        Name of the git branch to track for policy files (default: `master`, unless `OPAL_POLICY_REPO_TAG` is set)
+      </td>
+    </tr>
+    <tr>
+      <td valign="top">OPAL_POLICY_REPO_TAG</td>
+      <td>
+        Name of the git tag to track for policy files (default: `None`).
       </td>
     </tr>
   </tbody>

packages/opal-common/opal_common/git_utils/branch_tracker.py

@@ -1,7 +1,7 @@
 from functools import partial
 from typing import Optional, Tuple
 
-from git import GitCommandError, Head, Remote, Repo
+from git import GitCommandError, Head, Reference, Remote, Repo
 from git.objects.commit import Commit
 from opal_common.git_utils.env import provide_git_ssh_environment
 from opal_common.git_utils.exceptions import GitFailed
@@ -135,6 +135,10 @@ def tracked_branch(self) -> Head:
             )
             raise GitFailed(e)
 
+    @property
+    def tracked_reference(self) -> Reference:
+        return self.tracked_branch
+
     @property
     def tracked_remote(self) -> Remote:
         """Returns the tracked remote object (of type git.Remote) or throws if

packages/opal-common/opal_common/git_utils/repo_cloner.py

@@ -139,7 +139,8 @@ def __init__(
         self,
         repo_url: str,
         clone_path: str,
-        branch_name: str = "master",
+        branch_name: Optional[str] = "master",
+        tag_name: Optional[str] = None,
         retry_config=None,
         ssh_key: Optional[str] = None,
         ssh_key_file_path: Optional[str] = None,
@@ -151,6 +152,10 @@ def __init__(
             repo_url (str): the url to the remote repo we want to clone
             clone_path (str): the target local path in our file system we want the
                 repo to be cloned to
+            branch_name (str, optional): branch to clone. git clone --branch accepts
+                both branch and tag names.
+            tag_name (str, optional): tag to clone. Used as the --branch value when
+                branch_name is not set.
             retry_config (dict): Tenacity.retry config (@see https://tenacity.readthedocs.io/en/latest/api.html#retry-main-api)
             ssh_key (str, optional): private ssh key used to gain access to the cloned repo
             ssh_key_file_path (str, optional): local path to save the private ssh key contents
@@ -160,7 +165,8 @@ def __init__(
 
         self.url = repo_url
         self.path = os.path.expanduser(clone_path)
-        self.branch_name = branch_name
+        # git clone --branch accepts both branch and tag names
+        self.clone_ref = branch_name or tag_name
         self._ssh_key = ssh_key
         self._ssh_key_file_path = (
             ssh_key_file_path or opal_common_config.GIT_SSH_KEY_FILE
@@ -178,10 +184,10 @@ async def clone(self) -> CloneResult:
         - finds a cloned repo locally and does not clone from remote.
         """
         logger.info(
-            "Cloning repo from '{url}' to '{to_path}' (branch: '{branch}')",
+            "Cloning repo from '{url}' to '{to_path}' (ref: '{ref}')",
             url=self.url,
             to_path=self.path,
-            branch=self.branch_name,
+            ref=self.clone_ref,
         )
         loop = asyncio.get_running_loop()
         return await loop.run_in_executor(None, self._attempt_clone_from_url)
@@ -204,9 +210,10 @@ def _attempt_clone_from_url(self) -> CloneResult:
 
     def _clone(self, env) -> Repo:
         try:
-            return Repo.clone_from(
-                url=self.url, to_path=self.path, branch=self.branch_name, env=env
-            )
+            kwargs = dict(url=self.url, to_path=self.path, env=env)
+            if self.clone_ref is not None:
+                kwargs["branch"] = self.clone_ref
+            return Repo.clone_from(**kwargs)
         except (GitError, GitCommandError) as e:
             logger.error("cannot clone policy repo: {error}", error=e)
             raise

packages/opal-common/opal_common/git_utils/tag_tracker.py

@@ -0,0 +1,113 @@
+from functools import partial
+from typing import Optional, Tuple
+
+from git import GitCommandError, Reference, Repo, Tag
+from git.objects.commit import Commit
+from opal_common.git_utils.branch_tracker import BranchTracker
+from opal_common.git_utils.env import provide_git_ssh_environment
+from opal_common.git_utils.exceptions import GitFailed
+from opal_common.logger import logger
+from tenacity import retry, stop_after_attempt, wait_fixed
+
+
+class TagTracker(BranchTracker):
+    """Tracks the state of a git tag (hash the tag is pointing at).
+
+    Can detect if the tag has been moved to point at a different commit.
+    """
+
+    def __init__(
+        self,
+        repo: Repo,
+        tag_name: str,
+        remote_name: str = "origin",
+        retry_config=None,
+        ssh_key: Optional[str] = None,
+    ):
+        """Initializes the TagTracker.
+
+        Args:
+            repo (Repo): a git repo in which we want to track the specific commit a tag is pointing to
+            tag_name (str): the tag we want to track
+            remote_name (str): the remote in which the tag is located
+            retry_config (dict): Tenacity.retry config
+            ssh_key (Optional[str]): SSH key for private repositories
+        """
+        self._tag_name = tag_name
+        super().__init__(
+            repo,
+            branch_name=None,
+            remote_name=remote_name,
+            retry_config=retry_config,
+            ssh_key=ssh_key,
+        )
+
+    def checkout(self):
+        """Checkouts the repository at the current tag."""
+        checkout_func = partial(self._repo.git.checkout, self._tag_name)
+        attempt_checkout = retry(**self._retry_config)(checkout_func)
+        try:
+            return attempt_checkout()
+        except GitCommandError as e:
+            tags = [tag.name for tag in self._repo.tags]
+            logger.error(
+                "did not find tag: {tag_name}, instead found: {tags_found}, got error: {error}",
+                tag_name=self._tag_name,
+                tags_found=tags,
+                error=str(e),
+            )
+            raise GitFailed(e)
+
+    def _fetch(self):
+        """Fetch updates including tags with force option."""
+
+        def _inner_fetch(*args, **kwargs):
+            env = provide_git_ssh_environment(self.tracked_remote.url, self._ssh_key)
+            with self.tracked_remote.repo.git.custom_environment(**env):
+                self.tracked_remote.repo.git.fetch("--tags", "--force", *args, **kwargs)
+
+        attempt_fetch = retry(**self._retry_config)(_inner_fetch)
+        return attempt_fetch()
+
+    @property
+    def latest_commit(self) -> Commit:
+        """The commit of the tracked tag."""
+        return self.tracked_tag.commit
+
+    @property
+    def tracked_tag(self) -> Tag:
+        """Returns the tracked tag reference (of type git.Reference) or throws
+        if such tag does not exist on the repo."""
+        try:
+            return getattr(self._repo.tags, self._tag_name)
+        except AttributeError as e:
+            tags = [{"path": tag.path} for tag in self._repo.tags]
+            logger.exception(
+                "did not find main branch: {error}, instead found: {tags_found}",
+                error=e,
+                tags_found=tags,
+            )
+            raise GitFailed(e)
+
+    @property
+    def tracked_reference(self) -> Reference:
+        return self.tracked_tag
+
+    def pull(self) -> Tuple[bool, Commit, Commit]:
+        """Overrides the pull method to handle tag updates.
+
+        Returns:
+            pull_result (bool, Commit, Commit): a tuple consisting of:
+                has_changes (bool): whether the tag has been moved to a different commit
+                prev (Commit): the previous commit the tag was pointing to
+                latest (Commit): the new commit the tag is currently pointing to
+        """
+        self._fetch()
+        self.checkout()
+
+        if self.prev_commit.hexsha == self.latest_commit.hexsha:
+            return False, self.prev_commit, self.prev_commit
+        else:
+            prev = self._prev_commit
+            self._save_latest_commit_as_prev_commit()
+            return True, prev, self.latest_commit

packages/opal-common/opal_common/git_utils/tests/conftest.py

@@ -73,6 +73,15 @@ def create_rename_file_commit(
         repo.index.move([filename, new_filename])
         repo.index.commit(commit_msg, author=author)
 
+    @staticmethod
+    def create_new_tag(repo: Repo, tag_name: str):
+        repo.create_tag(tag_name)
+
+    @staticmethod
+    def update_tag_to_head(repo: Repo, tag_name: str):
+        repo.delete_tag(tag_name)
+        repo.create_tag(tag_name)
+
 
 @pytest.fixture
 def helpers() -> Helpers:
@@ -140,6 +149,9 @@ def local_repo(tmp_path, helpers: Helpers) -> Repo:
 
     # create a "delete" commit
     helpers.create_delete_file_commit(repo, root / "deleted.rego")
+
+    # create a test tag
+    helpers.create_new_tag(repo, "test_tag")
     return repo