Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions .github/workflows/ci-development.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,41 +3,41 @@ name: Continuous Integration for development
on: pull_request
jobs:
lint:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- name: Install python
uses: actions/setup-python@v4
uses: actions/setup-python@v6
with:
python-version: "3.11"
python-version: "3.12"
- name: Install lint dependencies
run: |
pip install pre-commit black
- name: Install and run pre-commit
uses: pre-commit/action@v3.0.0
uses: pre-commit/action@v3.0.1
test:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
needs:
- lint
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]
python-version: ["3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- name: Set up testing python ${{ matrix.config.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.config.python-version }}
- name: Install poetry
uses: abatilo/actions-poetry@v2.3.0
uses: abatilo/actions-poetry@v4
- name: Setup a local virtual environment for caching
run: |
poetry config virtualenvs.create true --local
poetry config virtualenvs.in-project true --local
- name: Define a cache for the virtual environment based on the dependencies lock file to speed up later runs
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}
Expand Down
24 changes: 12 additions & 12 deletions .github/workflows/ci-production.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,26 +11,26 @@ on:

jobs:
test:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]
python-version: ["3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- name: Set up testing python ${{ matrix.config.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.config.python-version }}
- name: Install poetry
uses: abatilo/actions-poetry@v2.3.0
uses: abatilo/actions-poetry@v4
- name: Setup a local virtual environment for caching
run: |
poetry config virtualenvs.create true --local
poetry config virtualenvs.in-project true --local
- name: Define a cache for the virtual environment based on the dependencies lock file to speed up later runs
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}
Expand All @@ -41,17 +41,17 @@ jobs:

build-and-publish:
name: Build and publish Python distributions 📦 to PyPI and TestPyPI
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
needs:
- test
steps:
- uses: actions/checkout@v4
- name: Set up base Python 3.11
uses: actions/setup-python@v4
- uses: actions/checkout@v5
- name: Set up base Python 3.12
uses: actions/setup-python@v6
with:
python-version: 3.11
python-version: 3.12
- name: Python Poetry Action
uses: abatilo/actions-poetry@v2.3.0
uses: abatilo/actions-poetry@v4
- name: Publish distribution 📦 with test.pypi.org
if: startsWith(github.ref, 'refs/tags')
run: |
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -138,3 +138,6 @@ setup.py

# project files
**/ohsome_log/

# default testing script file
playground.py
10 changes: 5 additions & 5 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
repos:
- repo: https://github.com/ambv/black
rev: 23.11.0
rev: 25.9.0
hooks:
- id: black
- repo: https://github.com/pycqa/flake8
rev: 6.1.0
rev: 7.3.0
hooks:
- id: flake8
args:
Expand All @@ -13,9 +13,9 @@ repos:
- "--ignore=E501,W503,E203,F401,D400,D205,D401,D202,T001,D105"
additional_dependencies:
[
"flake8-bugbear==23.9.16",
"flake8-bugbear==24.12.12",
"flake8-coding==1.3.2",
"flake8-comprehensions==3.14.0",
"flake8-comprehensions==3.17.0",
"flake8-debugger==4.1.2",
"flake8-deprecated==2.2.1",
"flake8-pep3101==2.1.0",
Expand All @@ -25,7 +25,7 @@ repos:
"flake8-docstrings==1.7.0",
]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v6.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
Expand Down
58 changes: 36 additions & 22 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

## [Unreleased](https://github.com/GIScience/ohsome-py/compare/v0.3.3..master)

### Removed

- support for python 3.10 (it probably still works, we just don't assert it anymore)

### Added

- functionality to filter the output geometry types ([#178](https://github.com/GIScience/ohsome-py/issues/178))
- support for python 3.13

## [0.4.0](https://github.com/GIScience/ohsome-py/releases/tag/v0.4.0)

### Changed
Expand Down Expand Up @@ -35,55 +44,60 @@

### Changed

- if tags are supplied for explosion in `response.as_dataframe`, the respective column will always be present in the resulting Geodataframe, even if the tags were not part of the result. In that case the column will be all-None ([#149](https://github.com/GIScience/ohsome-py/issues/149)).

- if tags are supplied for explosion in `response.as_dataframe`, the respective column will always be present in the
resulting Geodataframe, even if the tags were not part of the result. In that case the column will be
all-None ([#149](https://github.com/GIScience/ohsome-py/issues/149)).

## [0.3.1](https://github.com/GIScience/ohsome-py/releases/tag/v0.3.1)

### Fixed

- prevent an exception if the `log_dir` for the `OhsomeClient` was set to `None`
- removed time-dependency of unit tests that would cause them to fail at any time after the cassettes were recorded
- prevent an exception if the `log_dir` for the `OhsomeClient` was set to `None`
- removed time-dependency of unit tests that would cause them to fail at any time after the cassettes were recorded

### Changed

- relaxed dependency requirement for `urllib3` to >=2.0.2 to prevent ohsome-py from becoming a 'diamond-dependency'
- improved and sped up testing (first steps towards [#139](https://github.com/GIScience/ohsome-py/issues/139))
- move metadata property from singleton to `cached_property`
- relaxed dependency requirement for `urllib3` to >=2.0.2 to prevent ohsome-py from becoming a 'diamond-dependency'
- improved and sped up testing (first steps towards [#139](https://github.com/GIScience/ohsome-py/issues/139))
- move metadata property from singleton to `cached_property`

## [0.3.0](https://github.com/GIScience/ohsome-py/releases/tag/v0.3.0)

### Added

- support for python 3.12
- custom [retry](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry) configuration
- start and end timestamp meta information of the client are now datetime objects
- accept shapely Polygon and MultiPolygon for `bpolys` input parameter
- if a request fails a bash script containing the respective `curl` command is logged (if possible). This allows for easier debugging and sharing of failed requests.
- timestamps are converted without timezone information. Deviates from Ohsome API [(Issue #318)](https://github.com/GIScience/ohsome-api/issues/318)
- support for python 3.12
- custom [retry](https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry) configuration
- start and end timestamp meta information of the client are now datetime objects
- accept shapely Polygon and MultiPolygon for `bpolys` input parameter
- if a request fails a bash script containing the respective `curl` command is logged (if possible). This allows for
easier debugging and sharing of failed requests.
- timestamps are converted without timezone information. Deviates from Ohsome
API [(Issue #318)](https://github.com/GIScience/ohsome-api/issues/318)

### Changed

- breaking: geodataframes now contain a `@other_tags` colum containing all OSM tags. This behaviour can be adapted using the `explode_tags` parameter that allows to specify tags that should be in a separate column or to disable the feature completely. The latter will result in a potentially wide but sparse data frame.
- breaking: geodataframes now contain a `@other_tags` colum containing all OSM tags. This behaviour can be adapted using
the `explode_tags` parameter that allows to specify tags that should be in a separate column or to disable the feature
completely. The latter will result in a potentially wide but sparse data frame.

### Removed

- support for python < 3.10
- support for geopandas < 0.14
- support for pandas < 2.1
- support for urllib3 < 2.1
- support for python < 3.10
- support for geopandas < 0.14
- support for pandas < 2.1
- support for urllib3 < 2.1

## [0.2.0](https://github.com/GIScience/ohsome-py/releases/tag/v0.2.0)

### Added

- support for python 3.11
- support for geopandas up to v0.12.0
- support for python 3.11
- support for geopandas up to v0.12.0

### Removed

- support for python 3.7
- support for python 3.7

### Fixed

- wrong formatting of list parameters for ohsome requests if not given as string
- wrong formatting of list parameters for ohsome requests if not given as string
6 changes: 3 additions & 3 deletions ohsome/helper.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ def format_bcircles(
dict,
gpd.GeoDataFrame,
pd.DataFrame,
]
],
) -> str:
"""
Formats bcircles parameter to comply with ohsome API
Expand Down Expand Up @@ -152,7 +152,7 @@ def format_bboxes(
List[float],
List[List[str]],
List[List[float]],
]
],
) -> str:
"""
Formats bboxes parameter to comply with ohsome API
Expand Down Expand Up @@ -206,7 +206,7 @@ def format_bboxes(
def format_bpolys(
bpolys: Union[
gpd.GeoDataFrame, gpd.GeoSeries, shapely.Polygon, shapely.MultiPolygon, str
]
],
) -> str:
"""
Formats bpolys parameter to comply with ohsome API
Expand Down
19 changes: 15 additions & 4 deletions ohsome/response.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ def __init__(self, data: dict, url: str = None):
self.url = url

def as_dataframe(
self, multi_index: Optional[bool] = True, explode_tags: Optional[tuple] = ()
self,
multi_index: Optional[bool] = True,
explode_tags: Optional[tuple] = (),
geometry_filter: Optional[list[str]] = None,
) -> Union[pd.DataFrame, gpd.GeoDataFrame]:
"""
Converts the ohsome response to a pandas.DataFrame or a geopandas.GeoDataFrame if the
Expand All @@ -31,12 +34,14 @@ def as_dataframe(
:param explode_tags: By default, tags of extracted features are stored in a single dict-column. You can specify
a tuple of tags that should be popped from this column. To disable it completely, pass None. Yet, be aware that
you may get a large but sparse data frame.
:param geometry_filter: Limit the output geometries to the specified types to prevent
https://github.com/GIScience/ohsome-api/issues/339
:return: pandas.DataFrame or geopandas.GeoDataFrame
"""
if "features" not in self.data.keys():
return self._as_dataframe(multi_index)
else:
return self._as_geodataframe(multi_index, explode_tags)
return self._as_geodataframe(multi_index, explode_tags, geometry_filter)

def _as_dataframe(self, multi_index=True) -> pd.DataFrame:
groupby_names = []
Expand Down Expand Up @@ -70,7 +75,10 @@ def _as_dataframe(self, multi_index=True) -> pd.DataFrame:
return result_df.sort_index()

def _as_geodataframe(
self, multi_index: Optional[bool] = True, explode_tags: Optional[tuple] = ()
self,
multi_index: Optional[bool] = True,
explode_tags: Optional[tuple] = (),
geom_filter: Optional[list[str]] = None,
) -> gpd.GeoDataFrame:
if len(self.data["features"]) == 0:
return gpd.GeoDataFrame(
Expand All @@ -88,7 +96,7 @@ def _as_geodataframe(
for feature in self.data["features"]:
properties = feature["properties"]
tags = {}
new_properties = {k: None for k in explode_tags}
new_properties = dict.fromkeys(explode_tags, None)
for k in properties.keys():
if (
(k.startswith("@"))
Expand All @@ -108,6 +116,9 @@ def _as_geodataframe(
"This result type cannot be converted to a GeoPandas GeoDataFrame object."
)

if geom_filter is not None:
features = features[features.geom_type.isin(geom_filter)]

time_columns = ["@validFrom", "@validTo", "@snapshotTimestamp", "@timestamp"]
existing_time_columns = features.columns.intersection(time_columns)
features[existing_time_columns] = features[existing_time_columns].apply(
Expand Down
46 changes: 46 additions & 0 deletions ohsome/test/data/mixed_geometries.geojson
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
{
"attribution": {
"url": "https://ohsome.org/copyrights",
"text": "\u00a9 OpenStreetMap contributors"
},
"apiVersion": "1.10.4",
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"coordinates": [
[
12.30001864618336,
48.220125234580934
],
[
12.30022996627983,
48.22026810127991
]
],
"type": "LineString"
},
"properties": {
"@osmId": "way/26936052",
"@snapshotTimestamp": "2016-01-01T00:00:00Z",
"c": "d"
}
},
{
"type": "Feature",
"geometry": {
"coordinates": [
12.3003682566372,
48.22021116170123
],
"type": "Point"
},
"properties": {
"@osmId": "way/26936051",
"@snapshotTimestamp": "2016-01-01T00:00:00Z",
"a": "b"
}
}
]
}
Loading
Loading