Skip to content

IQGeo/utils-data-validation

Repository files navigation

Pre-Import Data Validation Tool

Validate your CDIF data packages before importing them into IQGeo.
This tool runs a series of checks to catch CDIF data issues early, helping prevent failed imports and downstream errors — surfacing actionable feedback so problems can be corrected before the import process begins.

Data Privacy & Security
This tool runs entirely on your local machine. Your data never leaves your environment — no files are uploaded, no external services are contacted, and no network connectivity is required. It is safe to use with sensitive or proprietary datasets.


Prerequisites

  • Python 3.11+ installed (initial release tested with 3.11-3.14.)
  • A CDIF data package (zipped or unzipped)

Getting Started

1. Set up the environment

Create and activate a Python virtual environment, then install dependencies:

python -m venv venv
source venv/bin/activate        # On Windows: venv\Scripts\activate
pip install -r requirements.txt

To exit the virtual environment when you're done, run deactivate.

2. Run the validator

Pass the path to your CDIF directory as the argument. If you have a zipped package, use the -z flag and pass the path to the zip file instead.

Option A: Run with Python

python cdif_validator_pre_import.py <CDIF_path>

Option B: Run as an executable (Linux/macOS)

First, make the script executable (only needs to be done once):

chmod +x cdif_validator_pre_import.py 

Then run it directly:

./cdif_validator_pre_import.py <CDIF_path>

3. Review the output

The tool will report any validation errors and warnings found in the CDIF data. All reported issues should be addressed before proceeding with an IQGeo import.

For full details on all checks and warnings, use the --verbose flag (see Options below).


Options:

> python cdif_validator_pre_import.py --help
usage: cdif_validator_pre_import.py [-h] [--model MODEL] [--verbose] [--unzip] root_dir

Validate CDIF package This tool is used to validate CDIF data packages before they are imported into IQGeo.

positional arguments:
  root_dir              Path to the CDIF package directory

options:
  -h, --help            show this help message and exit
  --model MODEL, -m MODEL
                        Path to user data model file (default: data_model.yml).
  --verbose, -v         Enable verbose output.
  --report, -r          Create JSON format error report.
  --unzip, -z           Treat root_dir as a zip file and unzip it into a temporary directory.

Reporting Issues

If a file named return_to_sender_please.txt or serious_crash_log.txt is created during a run, please submit a support ticket at https://partnersupport.iqgeo.com. Including these files in your ticket will help the team investigate and improve the tool.

Copyright

Copyright (c) 2025-2026 IQGeo Group Plc. Use subject to conditions at license.txt

About

DevRel: data validation tool

Resources

License

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors