Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

S3method(as.matrix,quantile_pred)
S3method(as_tibble,quantile_pred)
S3method(factor_key,default_formula_blueprint)
S3method(factor_key,default_recipe_blueprint)
S3method(factor_key,default_xy_blueprint)
S3method(factor_key,formula_blueprint)
S3method(factor_key,recipe_blueprint)
S3method(factor_key,terms)
S3method(factor_key,xy_blueprint)
S3method(forge,data.frame)
S3method(forge,default)
S3method(forge,matrix)
Expand Down Expand Up @@ -94,6 +101,7 @@ export(extract_recipe)
export(extract_spec_parsnip)
export(extract_tailor)
export(extract_workflow)
export(factor_key)
export(fct_encode_one_hot)
export(forge)
export(frequency_weights)
Expand Down
2 changes: 2 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# hardhat (development version)

* Added `factor_key()` to create mappings between original factor variables and their binary indicator columns in model matrices. This function helps understand how factors are encoded with different contrast methods, including support for interactions and nested effects.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Added `factor_key()` to create mappings between original factor variables and their binary indicator columns in model matrices. This function helps understand how factors are encoded with different contrast methods, including support for interactions and nested effects.
* Added `factor_key()` to create mappings between original factor variables and their binary indicator columns in model matrices. This function helps understand how factors are encoded with different contrast methods, including support for interactions and nested effects (#304).


# hardhat 1.4.3

* `mold()` no longer throws warnings about `strings_as_factors` argument when used on recipe objects (#284).
Expand Down
359 changes: 359 additions & 0 deletions R/factor-key.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,359 @@
#' Map factor variables to their numeric features
#'
#' @description
#' `factor_key()` creates a mapping between original factor variables and
#' their corresponding numeric features (e.g., binary indicator columns, etc.)
#' generated in a model matrix.
#'
#' @param x An object. For the default method, a terms object.
#' @param ... Arguments passed to methods.
#'
#' @return
#' A tibble with two columns:
#' \describe{
#' \item{`source`}{The name of the original factor variable}
#' \item{`derived`}{The name of the generated model matrix column}
#' }
#'
#' For interaction terms, multiple rows are returned with each source
#' factor mapped to the interaction column.
#'
#' Returns an empty tibble with the correct columns if no factors are present
#' in the terms.
#'
#' @details
#'
#' This function helps you understand how R's model matrix construction
#' converts factor variables into multiple features. It handles:
#'
#' - **Contrast types**: Different contrast methods (treatment, sum, helmert,
#' polynomial, one-hot) produce different column naming patterns
#' - **Ordered factors**: These typically use polynomial contrasts, creating
#' columns with `.L` (linear), `.Q` (quadratic), `.C` (cubic) suffixes
#' - **Interactions**: Interaction terms like `A:B` will have multiple rows in
#' the output, one for each contributing factor
#' - **Nested effects**: Terms like `A/B` (which expands to `A + A:B`) are
#' properly handled, with the nested interaction mapped to both source factors
#'
#' The function uses the same model matrix generation as [model_matrix()],
#' ensuring consistency with how your models will actually be fit.
#'
#' @examples
#' if (rlang::is_installed("modeldata")) {
#' library(modeldata)
#'
#' # ---------------------------------------------------------------------------
#' # Simple factor with treatment contrasts (default)
#'
#' framed <- model_frame(bill_length_mm ~ species, modeldata::penguins)
#' factor_key(framed$terms, framed$data)
#'
#' # ---------------------------------------------------------------------------
#' # Multiple factors
#'
#' framed <- model_frame(Income ~ Home + Job, modeldata::credit_data)
#' factor_key(framed$terms, framed$data)
#'
#' # ---------------------------------------------------------------------------
#' # Interaction between two factors
#'
#' framed <- model_frame(bill_length_mm ~ species * island, modeldata::penguins)
#' factor_key(framed$terms, framed$data)
#'
#' # ---------------------------------------------------------------------------
#' # Nested effects (Job nested within Home)
#'
#' framed <- model_frame(Income ~ Home / Job, modeldata::credit_data)
#' # This expands to: Income ~ Home + Home:Job
#' factor_key(framed$terms, framed$data)
#'
#' # ---------------------------------------------------------------------------
#' # No factors returns empty tibble
#'
#' framed <- model_frame(compressive_strength ~ cement + water, modeldata::concrete)
#' factor_key(framed$terms, framed$data)
#'
#' # ---------------------------------------------------------------------------
#' # Custom contrasts
#'
#' penguins2 <- modeldata::penguins
#' species_sum <- penguins2$species
#' contrasts(species_sum) <- contr.sum(3)
#' penguins2$species <- species_sum
#'
#' framed <- model_frame(bill_length_mm ~ species, penguins2)
#' factor_key(framed$terms, framed$data)
#'
#' # ---------------------------------------------------------------------------
#' # Using with blueprints from mold()
#'
#' # Formula blueprint
#' molded <- mold(bill_length_mm ~ species + island, penguins)
#' factor_key(molded$blueprint, penguins)
#'
#' # XY blueprint (returns empty tibble since no terms/factors)
#' bp_xy <- default_xy_blueprint()
#' molded_xy <- mold(modeldata::penguins[c("species", "island")],
#' modeldata::penguins["bill_length_mm"],
#' blueprint = bp_xy)
#' factor_key(molded_xy$blueprint)
#' }
#'
#' # ---------------------------------------------------------------------------
#' # Ordered factor with polynomial contrasts
#'
#' data(mtcars)
#' mtcars2 <- mtcars
#' mtcars2$gear_ord <- ordered(mtcars2$gear)
#' framed <- model_frame(mpg ~ gear_ord, mtcars2)
#' factor_key(framed$terms, framed$data)
#'
#' @seealso
#' - [model_matrix()] for generating the design matrix
#' - [get_levels()] for extracting factor levels
#' - [stats::contrasts()] for setting contrast methods
#'
#' @export
factor_key <- function(x, ...) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's rename this to get_factor_key(), like other exported get_*() functions in hardhat, e.g., get_levels(), get_outcome_levels()

UseMethod("factor_key")
}
Comment on lines +117 to +119

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add the standard .default() method to error with the input type? hardhat has that in other places as well.


#' @param data A data frame or tibble containing the variables in `x`.
#' @inheritParams validate_column_names
#'
#' @rdname factor_key
#' @export

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to export this or rather treat it has an internal function and the exported methods are for the blueprints? Having worked through the rest, I now think we do indeed want this to be internal.

factor_key.terms <- function(x, data, ..., call = current_env()) {
check_dots_empty0(...)
check_terms(x, call = call)
check_data_frame_or_matrix(data, call = call)
data <- coerce_to_tibble(data)

# Get the data classes from the terms to identify factors
data_classes <- attr(x, "dataClasses")

# If no dataClasses attribute, try to infer from the data
if (is.null(data_classes)) {
# Get variable names from the terms
term_vars <- all.vars(x)
data_classes <- vapply(
term_vars,
get_first_class,
character(1),
data = data
)
names(data_classes) <- term_vars
}

# Identify factor and ordered variables
# Note: "ordered" is a subclass of "factor" but stored separately in dataClasses
factor_vars <- names(data_classes)[
data_classes %in% c("factor", "ordered", "character")
]

# If no factors, return empty tibble with correct structure
if (length(factor_vars) == 0) {
return(tibble::tibble(source = character(), derived = character()))
}

# Generate the model matrix to get actual column names
# Use with_na_pass to handle missing values properly
# Also wrap in tryCatch to handle single-level factors gracefully

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we say above

#' The function uses the same model matrix generation as [model_matrix()],
#' ensuring consistency with how your models will actually be fit.

I would expect us to error here, too.

If you need us to return the empty tibble, I would try to avoid grepping on the text of the error message since that's one that might get translated if someone's using R in a non-English locale. We could look for single-level factors via stats::.getXlevels(terms, data) and also warn alongside the empty tibble.

mm <- tryCatch(
{
with_na_pass(model.matrix(x, data))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation above (line 38) says we're using model_matrix(), so I suggest we align those two.

},
error = function(e) {
# Check if it's the single-level factor error
if (
grepl(
"contrasts can be applied only to factors with 2 or more levels",
e$message
)
) {
# Return NULL to indicate no model matrix could be created
return(NULL)
}
# Re-throw other errors
stop(e)
}
)

# If model matrix couldn't be created (e.g., single-level factors), return empty tibble
if (is.null(mm)) {
return(tibble::tibble(source = character(), derived = character()))
}

# Get the assign attribute which maps columns to term indices
assign_attr <- attr(mm, "assign")
mm_colnames <- colnames(mm)

Comment on lines +187 to +190

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this down below the two blocks on factors_matrix since we don't use it immediately?

# Get the factors matrix from terms (shows which variables contribute to each term)
factors_matrix <- attr(x, "factors")

# If no factors matrix (e.g., intercept-only model), return empty tibble
if (is.null(factors_matrix)) {
return(tibble::tibble(source = character(), derived = character()))
}

# Get term labels
term_labels <- attr(x, "term.labels")

# Build mapping data
mapping_list <- list()

for (i in seq_along(mm_colnames)) {
col_name <- mm_colnames[i]
term_index <- assign_attr[i]

# Skip intercept (term_index == 0)
if (term_index == 0) {
next
}

# Get the term label
term_label <- term_labels[term_index]

# Find which variables contribute to this term
contributing_vars <- rownames(factors_matrix)[
factors_matrix[, term_index] > 0
]

# Filter to only factor variables
factor_contributors <- intersect(contributing_vars, factor_vars)

# If this column has factor contributors, add to mapping
if (length(factor_contributors) > 0) {
for (factor_var in factor_contributors) {
mapping_list[[length(mapping_list) + 1]] <- data.frame(
source = factor_var,
derived = col_name,
stringsAsFactors = FALSE
)
}
}
}

# Combine all mappings into a single data frame
if (length(mapping_list) > 0) {
result <- do.call(rbind, mapping_list)
result <- tibble::as_tibble(result)
} else {
result <- tibble::tibble(source = character(), derived = character())
}

result
Comment on lines +199 to +245

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Get term labels
term_labels <- attr(x, "term.labels")
# Build mapping data
mapping_list <- list()
for (i in seq_along(mm_colnames)) {
col_name <- mm_colnames[i]
term_index <- assign_attr[i]
# Skip intercept (term_index == 0)
if (term_index == 0) {
next
}
# Get the term label
term_label <- term_labels[term_index]
# Find which variables contribute to this term
contributing_vars <- rownames(factors_matrix)[
factors_matrix[, term_index] > 0
]
# Filter to only factor variables
factor_contributors <- intersect(contributing_vars, factor_vars)
# If this column has factor contributors, add to mapping
if (length(factor_contributors) > 0) {
for (factor_var in factor_contributors) {
mapping_list[[length(mapping_list) + 1]] <- data.frame(
source = factor_var,
derived = col_name,
stringsAsFactors = FALSE
)
}
}
}
# Combine all mappings into a single data frame
if (length(mapping_list) > 0) {
result <- do.call(rbind, mapping_list)
result <- tibble::as_tibble(result)
} else {
result <- tibble::tibble(source = character(), derived = character())
}
result
# For each model matrix column, find the factor variables contributing to it
# (the intercept maps to no factors, so it contributes an empty character)
contributors <- map(assign_attr, function(term_index) {
if (term_index == 0) {
return(character())
}
contributing_vars <- rownames(factors_matrix)[
factors_matrix[, term_index] > 0
]
intersect(contributing_vars, factor_vars)
})
tibble::tibble(
source = unlist(contributors),
derived = rep(mm_colnames, lengths(contributors))
)

This is Claude's suggestion when I asked it to use hardhat's pattern of "assemble columns, build once" which also avoids growing that list. Tests passed.

}

# Helper function to get the first class of a variable
get_first_class <- function(var, data) {
if (var %in% names(data)) {
return(class(data[[var]])[1])
} else {
return("unknown")
}
}

# ------------------------------------------------------------------------------
# Blueprint methods

#' @param data A data frame or tibble containing the variables used during
#' model fitting. Required for blueprint methods since blueprints don't
#' store the original data.
#'
#' @rdname factor_key
#' @export
factor_key.default_formula_blueprint <- function(
x,
data,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need data because the column names depend only on the factor levels (not the data values) and the blueprint stores blueprint$ptypes$predictors, a 0-row tibble, with the levels preserved in the factors of the tibble. We could mock up a data frame with that information to pass to the .terms() method.

Another reason I think we should avoid the data argument here is that if we are running factor_key() on a blueprint, we likely want to know what the translation was for exactly that blueprint, i.e., if we take a data input, the factor levels in that data might not be exactly the same as in the blueprint.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

library(hardhat)

train <- data.frame(
  y = 1:6,
  f1 = factor(rep(c("a", "b", "c"), 2)),
  f2 = factor(rep(c("x", "y"), 3))
)

molded <- mold(y ~ f1 + f2, train)
bp <- molded$blueprint

baseline <- factor_key(bp, train)

# alternative: use the ptypes from the blueprint
ptype <- bp$ptypes$predictors

# 0-row tibble, but keeps <fct> levels
ptype
#> # A tibble: 0 × 2
#> # ℹ 2 variables: f1 <fct>, f2 <fct>

rows_needed <- max(vapply(ptype, nlevels, integer(1)))
df_mocked_from_blueprint <-
  purrr::map(ptype, function(col) {
    factor(rep(levels(col), length.out = rows_needed), levels = levels(col))
  }) |>
  as.data.frame()

identical(factor_key(bp, df_mocked_from_blueprint), baseline)
#> [1] TRUE


df_without_level_c <- data.frame(
  y = 1:4,
  f1 = factor(rep(c("a", "b"), 2)),
  f2 = factor(rep(c("x", "y"), 2))
)

factor_key(bp, df_without_level_c) |>
  dplyr::filter(source == "f1")
#> # A tibble: 2 × 2
#>   source derived
#>   <chr>  <chr>  
#> 1 f1     f1a    
#> 2 f1     f1b

baseline |>
  dplyr::filter(source == "f1")
#> # A tibble: 3 × 2
#>   source derived
#>   <chr>  <chr>  
#> 1 f1     f1a    
#> 2 f1     f1b    
#> 3 f1     f1c

Created on 2026-06-09 with reprex v2.1.1

...,
call = current_env()
) {
check_dots_empty0(...)

# Extract the predictors terms from the blueprint
terms_obj <- x$terms$predictors

if (is.null(terms_obj)) {
cli::cli_abort(
"Blueprint does not contain terms for predictors.",
call = call
)
}

# Call the terms method
factor_key.terms(terms_obj, data, call = call)
}

#' @rdname factor_key
#' @export
factor_key.formula_blueprint <- function(x, data, ..., call = current_env()) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove the methods for the <interface>_blueprint parent classes? The default_<interface>_blueprint classes inherit from the parent classes but hardhat only implements methods for the default_* classes. The reason for this is that the parent classes are the extensibility point of hardhat: someone else could subclass, e.g., formula_blueprint and then provide their own run_mold() etc methods. If some method is missing, the .default() method kicks in.

# Fallback for non-default formula blueprints
# Try to extract terms if available, otherwise error
check_dots_empty0(...)

if (!is.null(x$terms) && !is.null(x$terms$predictors)) {
factor_key.terms(x$terms$predictors, data, call = call)
} else {
cli::cli_abort(
"Cannot extract factor mappings from this formula blueprint type.",
call = call
)
}
}

#' @rdname factor_key
#' @export
factor_key.default_recipe_blueprint <- function(
x,
data = NULL,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use data so could remove it here

...,
call = current_env()
) {
check_dots_empty0(...)

# Recipes handle factors differently - they may encode them during prep
# For now, return a message indicating this isn't directly supported
cli::cli_abort(
c(
"factor_key() is not yet implemented for recipe blueprints.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "yet" is making me curious. If we plan on doing this, it'd be good to add it to this PR. If we don't, we can remove it and fall back on the (to-be added) .default() method. If the point is the error message about using a formula blueprint, we should remove the "yet".

"i" = "Recipes handle factor encoding during the prep step.",
"i" = "Consider using a formula blueprint if you need factor mappings."
),
call = call
)
}

#' @rdname factor_key
#' @export
factor_key.recipe_blueprint <- function(
x,
data = NULL,
...,
call = current_env()
) {
# Fallback for non-default recipe blueprints
factor_key.default_recipe_blueprint(x, data, ..., call = call)
}

#' @rdname factor_key
#' @export
factor_key.default_xy_blueprint <- function(
x,
data = NULL,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use data so could remove it here

...,
call = current_env()
) {
check_dots_empty0(...)

# XY blueprints don't use terms or formulas, so there's no factor encoding to map
# Return empty tibble to be consistent with no-factors case
tibble::tibble(source = character(), derived = character())
}

#' @rdname factor_key
#' @export
factor_key.xy_blueprint <- function(x, data = NULL, ..., call = current_env()) {
# Fallback for non-default XY blueprints
factor_key.default_xy_blueprint(x, data, ..., call = call)
}
Loading
Loading