Skip to content

t.list: Add support for listing multiple dataset types #7731

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
saket0187 wants to merge 8 commits into
base: main
Choose a base branch
from
Open

t.list: Add support for listing multiple dataset types #7731

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
2bf81a0
Add multiple types input option
saket0187 Jul 16, 2026
9380913
Add changes and improvements
saket0187 Jul 17, 2026
91035e8
Support both str and list
saket0187 Jul 17, 2026
ac3d6dd
Add changes
saket0187 Jul 20, 2026
eaf31d5
Add bug fixes and design changes
saket0187 Jul 30, 2026
0957f85
Fix plain format
saket0187 Jul 30, 2026
50aef44
Fix issues
saket0187 Aug 6, 2026
35ba70a
Merge branch 'main' into Add-multiple-types-input-option-t.list
saket0187 Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 33 additions & 26 deletions python/grass/temporal/list_stds.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def get_dataset_list(
This method returns a dictionary, the keys are the available mapsets,
the values are the rows from the SQL database query.

:param type: The type of the datasets (strds, str3ds, stvds, raster,
:param type: A list of dataset types (strds, str3ds, stvds, raster,
Comment thread
petrasovaa marked this conversation as resolved.
Outdated
raster_3d, vector)
:param temporal_type: The temporal type of the datasets (absolute,
relative)
Expand Down Expand Up @@ -76,15 +76,15 @@ def get_dataset_list(
... )
>>> mapset = tgis.get_current_mapset()
>>> stds_list = tgis.list_stds.get_dataset_list(
... "strds", "absolute", columns="name"
... ["strds"], "absolute", columns="name"
... )
>>> rows = stds_list[mapset]
>>> for row in rows:
... if row["name"] == name:
... print(True)
True
>>> stds_list = tgis.list_stds.get_dataset_list(
... "strds",
... ["strds"],
... "absolute",
... columns="name,mapset",
... where="mapset = '%s'" % (mapset),
Expand All @@ -101,31 +101,38 @@ def get_dataset_list(

result = {}

for mapset in dbif.tgis_mapsets:
if temporal_type == "absolute":
table = type + "_view_abs_time"
else:
table = type + "_view_rel_time"

if columns and columns.find("all") == -1:
sql = "SELECT " + str(columns) + " FROM " + table
else:
sql = "SELECT * FROM " + table

if where:
sql += " WHERE " + where
sql += " AND mapset = '%s'" % (mapset)
else:
sql += " WHERE mapset = '%s'" % (mapset)

if order:
sql += " ORDER BY " + order
for dtype in type:
for mapset in dbif.tgis_mapsets:
if temporal_type == "absolute":
table = dtype + "_view_abs_time"

@ninsbl ninsbl Jul 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the intention is to be able to list all STDS in one go, should we not support also multiple / list input for temporal_type?
Sorry for bringing that up somewhat late...

@petrasovaa petrasovaa Jul 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree and t.list already allows that, just not the function.

@ninsbl ninsbl Jul 19, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That probably means the temporal_type should always go into the JSON and CSV output too, no?

@petrasovaa petrasovaa Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The temporal_type is already in CSV and JSON with columns=all, it could be added explicitly as one of the options, but I am not sure it's worth it.

else:
table = dtype + "_view_rel_time"

dbif.execute(sql, mapset=mapset)
rows = dbif.fetchall(mapset=mapset)
if columns and columns.find("all") == -1:
sql = "SELECT " + columns + " FROM " + table
else:
sql = "SELECT * FROM " + table

if rows:
result[mapset] = rows
if where:
sql += " WHERE " + where
sql += " AND mapset = '%s'" % (mapset)
else:
sql += " WHERE mapset = '%s'" % (mapset)

if order:
sql += " ORDER BY " + order

dbif.execute(sql, mapset=mapset)
rows = dbif.fetchall(mapset=mapset)

if rows:
if mapset not in result:
result[mapset] = []
for row in rows:
row_dict = dict(row)
if len(type) > 1:
row_dict["type"] = dtype

@ninsbl ninsbl Jul 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Back to the question if the keys should be consistent, regardless if t.list is run with multiple or single type (or possibly single or multiple temporal_type input? I tend to say: make the dict (JSON) output structure consistent / predictable. But no strong opinion. It should just be deliberate.

Also, could this block be solved simpler with a dict-comprehension and/or a dict-update? (consult ruff --preview --select ALL)

@petrasovaa petrasovaa Jul 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSON should not have mixed items (each item in an array should have the same set of keys), which is the case now because we don't mix maps and datasets. In the comment above I was thinking to add the type key always for JSON and CSV. I hope I am not missing anything here.

@ninsbl ninsbl Jul 19, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but e.g semantic_label is only available for STRDS. So, should columns that do not exist for every dataset type just be empty? Just me thinking out loud...

@petrasovaa petrasovaa Jul 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented on this below, I think we need to select the intersection of the different columns.

result[mapset].append(row_dict)

if connection_state_changed:
dbif.close()
Expand Down
44 changes: 42 additions & 2 deletions temporal/t.list/t.list.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

*t.list* lists any dataset that is registered in the temporal database.
Datasets are raster, 3D raster and vector maps as well as their
corresponding space time datasets (STRDS, STR3DS and STVDS). The type of
the dataset can be specified using the *type* option, default is STRDS.
corresponding space time datasets (STRDS, STR3DS and STVDS). The type of the
dataset can be specified using the *type* option (default is STRDS).
Multiple comma-separated types can be provided, such as *type=strds,stvds*.
By default all datasets with relative and absolute time are listed.
However, the user has the ability to specify a single temporal type with
the *temporaltype* option. The user can define the columns that should
Expand Down Expand Up @@ -149,7 +150,46 @@ t.list type=raster format=json columns=id,start_time
"start_time": "2012-12-01 00:00:00"
}
]
```

To list multiple dataset types at once, specify a comma-separated list of
types.

**Note:** When multiple types are requested (e.g., `type=strds,stvds`), the
`type` column is automatically included in **csv** and **json** outputs to
clearly identify the dataset type of each record:

```sh
t.list type=strds,stvds columns=id format=json
```

```json
[
{
"id": "lst_daily@PERMANENT",
"type": "strds"
},
{
"id": "mini_set@PERMANENT",
"type": "strds"
},
{
"id": "nc_lst_daily@PERMANENT",
"type": "strds"
},
{
"id": "precip_abs1@PERMANENT",
"type": "strds"
},
{
"id": "prec_observer@PERMANENT",
"type": "stvds"
},
{
"id": "schools_stds@PERMANENT",
"type": "stvds"
}
]
```

## SEE ALSO
Expand Down
125 changes: 69 additions & 56 deletions temporal/t.list/t.list.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
# % description: Type of the space time dataset or map, default is strds
# % guisection: Selection
# % required: no
# % multiple: yes
# % options: strds, str3ds, stvds, raster, raster_3d, vector
# % answer: strds
# %end
Expand Down Expand Up @@ -112,7 +113,7 @@
def main():

# Get the options
stds_type = options["type"]
stds_type = options["type"].split(",")
temporal_type = options["temporaltype"]
columns = options["columns"]
order = options["order"]
Expand Down Expand Up @@ -143,6 +144,8 @@ def main():
elif output_format == "line":
if colhead:
gs.fatal(_("Column names are not allowed with line format"))
if len(stds_type) > 1:
gs.fatal(_("Only one type is allowed for line format"))
if not separator:
separator = ","
columns_list = columns.split(",") if columns else []
Expand All @@ -156,6 +159,18 @@ def main():
elif not separator: # output_format == "plain"
separator = "|"

if set(stds_type) & {"raster", "raster_3d", "vector"} and set(stds_type) & {
"strds",
"str3ds",
"stvds",
}:
gs.fatal(
_(
"Combinations across space time datasets and time stamped maps "
"(e.g., raster and strds) are not allowed"
)
)

# Lazy import and initialize TGIS
import grass.temporal as tgis

Expand Down Expand Up @@ -186,9 +201,6 @@ def main():
line_output = []
first = True

if gs.verbosity() > 0 and not outpath and output_format == "plain":
sys.stderr.write("----------------------------------------------\n")

# Replace separate "if outpath" and "else" blocks with a unified context manager:
with (
open(outpath, "w")
Expand All @@ -204,66 +216,67 @@ def main():
for mapset in dbif.tgis_mapsets:
rows = stds_list.get(mapset)
if rows:
if (
gs.verbosity() > 0
and (not outpath or outpath == "-")
and output_format == "plain"
):
if stds_type in {"raster", "raster_3d", "vector"}:
if len(stds_type) > 1 and output_format == "plain":
rows_by_type = {}
for r in rows:
rows_by_type.setdefault(r.get("type"), []).append(r)
target_types = rows_by_type.items()
else:
target_types = [(stds_type[0], rows)]

for type_label, current_rows in target_types:
if (
gs.verbosity() > 0
and (not outpath or outpath == "-")
and output_format == "plain"
):
sys.stderr.write(
_(
"Time stamped %s maps with %s available in mapset "
"<%s>:\n"
)
% (stds_type, time, mapset)
"----------------------------------------------\n"
)
else:
sys.stderr.write(
_(
"Space time %s datasets with %s available in "
"mapset <%s>:\n"
if type_label in {"raster", "raster_3d", "vector"}:
sys.stderr.write(
_(
"Time stamped maps of type <%s> with %s "
Comment thread
saket0187 marked this conversation as resolved.
Outdated
"available in mapset <%s>:\n"
)
% (type_label, time, mapset)
)
% (
stds_type,
time,
mapset,
else:
sys.stderr.write(
_(
"Space time datasets of type <%s> with %s "
"available in mapset <%s>:\n"
)
% (type_label, time, mapset)
)
)

if output_format == "json":
for row in rows:
json_output.append(dict(row))
if output_format == "json":
json_output.extend(current_rows)

elif output_format == "line":
line_output.extend([str(row[0]) for row in rows])
elif output_format == "line":
line_output.extend(
[str(v) for row in current_rows for v in row.values()]
)

else:
if (colhead or output_format == "csv") and first:
output = ""
count = 0
for col_key in rows[0].keys():
output += (separator if count > 0 else "") + str(
col_key
else:
if (colhead or output_format == "csv") and first:
output = separator.join(
str(k)
for k in current_rows[0].keys()
if output_format == "csv" or k != "type"
)
count += 1
out_file.write(f"{output}\n")
first = False

for row in rows:
output = ""
count = 0
for col in row:
# If the database value is None, make it an empty string for csv
if col is None:
cell_value = (
"" if output_format == "csv" else "None"
)
else:
cell_value = str(col)

output += (separator if count > 0 else "") + cell_value
count += 1
out_file.write("{st}\n".format(st=output))
out_file.write(f"{output}\n")
first = False

for row in current_rows:
output = separator.join(
("" if output_format == "csv" else "None")
if v is None
else str(v)
for k, v in row.items()
if output_format == "csv" or k != "type"
)
out_file.write(f"{output}\n")

# Dump the collected JSON and line data
if output_format == "json":
Expand Down
Loading
Loading