Skip to content

Try to use the same guards for the same type of template instantiation. #2051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
hchen99 wants to merge 10 commits into
base: master
Choose a base branch
from
Draft

Try to use the same guards for the same type of template instantiation. #2051

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
6ada4fe
Try to use the same guards for the same type of template instantiation.
hchen99 Feb 23, 2026
34e4372
Merge branch 'master' into template_instantiation_duplication
hchen99 Feb 24, 2026
0f02aee
Updated to always use tpye based guards for user defined %template or…
hchen99 Feb 26, 2026
9fab482
Updated comments.
hchen99 Feb 27, 2026
bea8111
Merge branch 'master' into template_instantiation_duplication
hchen99 Feb 27, 2026
cd90be1
Merge branch 'master' into template_instantiation_duplication
hchen99 Mar 30, 2026
5f0fd90
Merge branch 'master' into template_instantiation_duplication
hchen99 Apr 6, 2026
9519809
Added to generate a report for all template instantiations so user ca…
hchen99 Apr 6, 2026
c2825a8
Merge branch 'master' into template_instantiation_duplication
hchen99 May 19, 2026
c5dbf18
Merge branch 'master' into template_instantiation_duplication
hchen99 Jun 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
285 changes: 277 additions & 8 deletions libexec/trick/convert_swig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,21 @@ my %out_of_date ;
my ($version, $thread, $year) ;
my %processed_templates ;

# Prefix for template instantiation guards
my $template_guard_prefix = "TRICK_SWIG_TEMPLATE_" ;

# Current file being processed (set in process_file loop, used by sub-functions)
my $current_file = '' ;

# Global report of all template instantiations encountered across all processed files.
# Key: normalized type (no whitespace). Value: arrayref of {name, file, kind} hashrefs.
# kind is 'user' for explicit %template directives, 'auto' for Trick-generated ones.
my %all_templates_report ;

# Set of header files actually processed in this convert_swig run (after skip guards).
# Used by write_template_report to evict stale entries from previous runs for those files.
my %files_processed_this_run ;

my $typedef_def = qr/typedef\s+ # the word typedef
(?:[_A-Za-z][\s\w]*\s*) # resolved type
(?:[_A-Za-z]\w*) # new type
Expand Down Expand Up @@ -173,6 +188,7 @@ if ( scalar keys %out_of_date == 0 ) {
##

process_file() ;
write_template_report() ;

##
## ================================================================================
Expand Down Expand Up @@ -203,6 +219,10 @@ sub process_file() {
next if ( $f =~ /^$ENV{TRICK_HOME}\/include/ ) ;
next if ( $f =~ /^$ENV{TRICK_HOME}\/trick_source/ ) ;

# Track the current file so subroutines can record template instantiations.
$current_file = $f ;
$files_processed_this_run{$f} = 1 ;

# clear the processed templates per each file
undef %processed_templates ;

Expand Down Expand Up @@ -285,6 +305,88 @@ sub process_file() {
}
}

## Scan for user defined %template directives and wrap in #ifndef guards if not already in a #ifndef block.
## The guard is based on the normalized template type to prevent duplicate template instantiations across SWIG interface files.
## Also check that the template is not within a %define block, as SWIG does not allow preprocessor directives within macro definitions.
my @templates_to_wrap;
while ( $contents =~ /^(.*?)(\%template\s*\(([^)]+)\)\s*([^;]+)\s*;)/gm ) {
my $leading_text = $1;
my $full_match = $2;
my $template_name = $3;
my $template_type = $4;
# Normalize the type (remove all whitespace)
my $template_type_no_sp = $template_type;
$template_type_no_sp =~ s/\s+//g;

# Check if this template is already wrapped in #ifndef
my $check_pos = pos($contents) // 0;
my $before_template = substr($contents, 0, $check_pos - length($full_match) - length($leading_text));

# Check if this template is inside a %define...%enddef block
my $define_count = () = $before_template =~ /\%define/g;
my $enddef_count = () = $before_template =~ /\%enddef/g;
my $inside_macro = ($define_count > $enddef_count);

# Record every user-defined %template directive in the global report, regardless
# of whether its guard already exists or it is inside a macro. This gives users a
# complete view of what names were requested for each C++ template type.
push @{$all_templates_report{$template_type_no_sp}}, {
name => $template_name,
file => $current_file,
kind => 'user',
};

# Only wrap if not already in a #ifndef TRICK_SWIG_TEMPLATE block and not inside a macro
if ( !$inside_macro && $before_template !~ /#ifndef\s+${template_guard_prefix}\w+\s*$/ ) {
# Generate unique guard name based on normalized type
my $guard_name = $template_guard_prefix . $template_type_no_sp;
$guard_name =~ s/[^a-zA-Z0-9_]/_/g; # Make it a valid C identifier

# Generate deterministic canonical name from type (same across all files)
my $canonical_name = $template_type_no_sp;
$canonical_name =~ s/[^a-zA-Z0-9_]/_/g; # Replace all special chars with _
$canonical_name =~ s/_+/_/g; # Collapse multiple underscores
$canonical_name =~ s/_$//; # Remove trailing underscore
# Add _ptr suffix if template contains pointer types
if ($template_type =~ /\*/) {
$canonical_name .= "_ptr";
}

push @templates_to_wrap, {
leading => $leading_text,
template => $full_match,
user_name => $template_name,
canonical_name => $canonical_name,
template_type => $template_type,
guard => $guard_name,
type => $template_type_no_sp
};

# Mark as processed
$processed_templates{$template_type_no_sp} = 1;
}
}

# Wrap templates in #ifndef guards (in reverse order to preserve positions)
foreach my $tpl (reverse @templates_to_wrap) {
my $leading = $tpl->{leading};
my $template = $tpl->{template};
my $user_name = $tpl->{user_name};
my $template_type = $tpl->{template_type};
my $guard = $tpl->{guard};
my $escaped = quotemeta($template);

# Just wrap user's template in guard - no canonical names or aliases
# Guard prevents duplicate %template instantiations across SWIG interface files
# User defined name is preserved in the SWIG interface file
my $wrapped = "$leading#ifndef $guard\n";
$wrapped .= "$leading#define $guard\n";
$wrapped .= "$leading$template\n";
$wrapped .= "$leading#endif\n";

$contents =~ s/(\Q$leading\E)$escaped/$wrapped/;
}

## Process the contents of the out_of_date header file to create the corresponding SWIG interface.

process_contents( \$contents , \$new_contents , "" , \@class_names ) ;
Expand Down Expand Up @@ -362,7 +464,6 @@ sub process_file() {

print "Writing $out_file\n" if not verbose_build() ;
}

}

sub usage() {
Expand All @@ -371,7 +472,131 @@ sub usage() {
}

## ================================================================================
## process_contents
## write_template_report
##
## Synopsis
##
## Write a human-readable report of all SWIG %template instantiations encountered
## during the current convert_swig run to "build/convert_swig_template_instantiations".
##
## For each unique C++ type the report lists every Python alias name that was
## registered (user-defined or auto-generated). When more than one name exists for
## the same type, the SWIG preprocessor guard (#ifndef TRICK_SWIG_TEMPLATE_...)
## ensures that only the first name encountered in the SWIG include chain is
## effective. The report marks any additional names as "(suppressed by guard)" so
## that users know which name to use when a Python type cannot be found.
##
sub write_template_report() {

my $report_file = "build/convert_swig_template_instantiations" ;

# Build the merged report data by:
# - Read any entries from a previous run that belong to files NOT being
# processed in this invocation (preserve cross-file accumulation).
# - Discard entries whose source file IS in %files_processed_this_run
# (those are stale — we have fresh data for those files now).
# - Add all entries recorded in %all_templates_report for this run.
my %merged ; # type_no_sp -> [ {name, file, kind}, ... ]

if ( -e $report_file ) {
open my $in, '<', $report_file
or warn "convert_swig: could not read existing report $report_file: $!\n" ;
if ( $in ) {
while ( <$in> ) {
next if /^\s*#/ ; # skip comment/header lines
next if /^\s*$/ ; # skip blank lines
# Data line format (space-padded, pipe-separated):
# type | name | kind | file [ # note]
my ( $type_col, $name_col, $kind_col, $rest ) =
split /\s*\|\s*/, $_, 4 ;
next unless defined $rest ;
( my $file_col = $rest ) =~ s/\s*#.*$// ; # strip trailing note
$file_col =~ s/^\s+|\s+$//g ; # trim whitespace
for my $v ( $type_col, $name_col, $kind_col ) {
$v =~ s/^\s+|\s+$//g if defined $v ;
}
# Only preserve entries for files we are NOT rebuilding now.
unless ( $files_processed_this_run{$file_col} ) {
push @{ $merged{$type_col} }, {
name => $name_col,
file => $file_col,
kind => $kind_col,
} ;
}
}
close $in ;
}
}

# Merge in the fresh entries collected during this run, deduplicating within
# each (name, file, kind) triple to suppress repeated auto-generated entries
# for multiple variables of the same template type within one class.
for my $type_no_sp ( keys %all_templates_report ) {
my %seen_key ;
for my $r ( @{ $all_templates_report{$type_no_sp} } ) {
my $key = "$r->{name}\t$r->{file}\t$r->{kind}" ;
next if $seen_key{$key}++ ;
push @{ $merged{$type_no_sp} }, $r ;
}
}

return unless %merged ;

open my $rfh, '>', $report_file or do {
warn "convert_swig: could not write template report to $report_file: $!\n" ;
return ;
} ;

print $rfh "# Trick SWIG Template Instantiations Report\n" ;
print $rfh "# Generated by convert_swig — accumulates across incremental builds.\n" ;
print $rfh "#\n" ;
print $rfh "# This file lists every C++ template instantiation that convert_swig registered\n" ;
print $rfh "# (or encountered) for Python during this build.\n" ;
print $rfh "#\n" ;
print $rfh "# When the same C++ type appears in multiple SWIG interface files with different\n" ;
print $rfh "# Python names, only the first name the SWIG preprocessor encounters is\n" ;
print $rfh "# effective. All subsequent names are blocked by the\n" ;
print $rfh "# #ifndef TRICK_SWIG_TEMPLATE_<type>\n" ;
print $rfh "# guard.\n" ;
print $rfh "#\n" ;
print $rfh "# If a Python name is not found in an input file, look it up here to discover\n" ;
print $rfh "# the effective name for the same C++ type.\n" ;
print $rfh "#\n" ;
print $rfh "# Columns: C++ type (no whitespace) | Python name | kind | source file\n" ;
print $rfh "# kind: 'user' = explicit %template directive written by the user\n" ;
print $rfh "# 'auto' = %template directive generated automatically by Trick\n" ;
print $rfh "# 'skipped' = type used as a member but no %template generated\n" ;
print $rfh "# (STL type filtered without -s flag, or wstring)\n" ;
print $rfh "# Look for a 'user' or 'auto' entry to find the effective name\n" ;
print $rfh "#\n" ;

foreach my $type_no_sp ( sort keys %merged ) {
my @regs = @{ $merged{$type_no_sp} } ;

my $first_active = 1 ; # first non-skipped entry is the effective one
for my $r ( @regs ) {
my $note ;
if ( $r->{kind} eq 'skipped' ) {
$note = 'no %template generated here (STL/filtered type)' ;
} elsif ( $first_active ) {
$note = '' ;
$first_active = 0 ;
} else {
$note = 'may be suppressed by guard — check the first active entry for the effective name' ;
}
printf $rfh "%-60s | %-35s | %-7s | %s%s\n",
$type_no_sp,
$r->{name},
$r->{kind},
$r->{file},
( $note ? " # $note" : '' ) ;
}
}

close $rfh ;
print "[34mWriting[0m $report_file\n" if not verbose_build() ;
}

##
## Synopsis
##
Expand Down Expand Up @@ -675,27 +900,61 @@ sub process_class($$$$$) {

$template_full_type =~ /([_A-Za-z][:\w]*)\s*</ ;
my ($template_type) = $1 ;

# Compute normalized type once — used both in the active and skipped branches.
my ($template_type_no_sp) = $template_full_type ;
$template_type_no_sp =~ s/\s//g ;

# ignore some STL types and types that involve std::wstring
if ( (( $stls and ! exists $stl_names{$template_type})
or ( !$stls and ! exists $all_stl_names{$template_type} ))
and $template_full_type !~ /(std::)?wstring/ ) {

my ($template_type_no_sp) = $template_full_type ;
$template_type_no_sp =~ s/\s//g ;
# Generate deterministic canonical name from type (always, so it
# can be recorded in the report even if the %template guard
# prevents emission in this file).
my $canonical_name = $template_type_no_sp;
$canonical_name =~ s/[^a-zA-Z0-9_]/_/g; # Replace all special chars with _
$canonical_name =~ s/_+/_/g; # Collapse multiple underscores
$canonical_name =~ s/_$//; # Remove trailing underscore
# Add _ptr suffix if template contains pointer types
if ($template_full_type =~ /\*/) {
$canonical_name .= "_ptr";
}

# Record every occurrence in the global report — including ones
# where the %template directive will be suppressed by the guard at
# SWIG compile time (i.e. the same type was already emitted by an
# earlier-included .i file). This gives users a complete picture of
# which files reference each template type and which Python name each
# file would have used.
if ($isSwigExcludeBlock == 0) {
push @{$all_templates_report{$template_type_no_sp}}, {
name => $canonical_name,
file => $current_file,
kind => 'auto',
};
}

#print "*** template_type_no_sp = $template_type_no_sp ***\n" ;
if ( ! exists $processed_templates{$template_type_no_sp} ) {

my $sanitized_namespace = $curr_namespace =~ s/:/_/gr ;
my $identifier = "${sanitized_namespace}${class_name}_${var_name}" ;
my $trick_swig_template = "TRICK_SWIG_TEMPLATE_$identifier" ;
my $trick_swig_template;

# Use type based guard
$trick_swig_template = $template_guard_prefix . $template_type_no_sp;
$trick_swig_template =~ s/[^a-zA-Z0-9_]/_/g;

# Mark as processed in this file
$processed_templates{$template_type_no_sp} = 1;

# Insert template directive immediately before instance
# This is required as of SWIG 4
my $typedef = "\n#ifndef $trick_swig_template\n" ;
$typedef .= "#define $trick_swig_template\n" ;
$typedef .= "\%template($identifier) $template_full_type;\n" ;
$typedef .= "\%template($canonical_name) $template_full_type;\n" ;
$typedef .= "#endif\n" ;
# Auto-generated templates just use canonical name, no aliases

# A SWIG %template directive must:
# 1. Appear before each template instantiation
Expand Down Expand Up @@ -736,6 +995,16 @@ sub process_class($$$$$) {

$processed_templates{$template_type_no_sp} = 1 ;
}
} elsif ( $isSwigExcludeBlock == 0 ) {
# This template type is in the STL skip list (or is wstring) so no
# %template directive is auto-generated here. Record every occurrence
# so users can see which files reference this type even when it is
# filtered out.
push @{$all_templates_report{$template_type_no_sp}}, {
name => '(skipped-stl)',
file => $current_file,
kind => 'skipped',
} ;
}
}
}
Expand Down
Loading