canvod.config API reference¶
Settings: loading the settings file, Pydantic validation, and defaults.
Loading¶
Configuration management for canvodpy.
This package provides: - Pydantic models for type-safe configuration - YAML-based configuration loading - CLI for configuration management - Validation and error reporting
Examples:
>>> from canvod.config import load_config
>>> config = load_config()
>>> print(config.nasa_earthdata_acc_mail)
>>> print(config.processing.aux_data.agency)
CanvodConfig
¶
Bases: BaseSettings
Complete canvodpy configuration.
Loaded from YAML files by ConfigLoader; individual fields can be
overridden via environment variables using the CANVOD__ prefix
and __ as the nested delimiter. Environment variables take
priority over YAML-file values.
Examples:
Override a single nested field without touching the YAML. VAR=value
must appear on the same command line as canvodpy — it only sets a
variable for the current shell otherwise, and canvodpy (a separate
process) never sees it::
CANVOD__PROCESSING__PARAMS__DAYS_PER_BATCH=7 canvodpy run ...
CANVOD__PROCESSING__CREDENTIALS__NASA_EARTHDATA_ACC_MAIL=me@x.com canvodpy run ...
To reuse an override across several commands, export it once instead::
export CANVOD__PROCESSING__PARAMS__DAYS_PER_BATCH=7
canvodpy config show # confirms the override took effect
canvodpy run ...
nasa_earthdata_acc_mail
property
¶
Return the configured NASA Earthdata email for CDDIS authentication.
Returns:
| Type | Description |
|---|---|
str | None
|
NASA Earthdata email address. |
MetadataConfig
¶
Bases: _StrictModel
Metadata to be written to processed files.
Notes
This is a Pydantic model for configuration validation.
to_attrs_dict()
¶
Convert to a dictionary for xarray attributes.
Returns:
| Type | Description |
|---|---|
dict[str, str]
|
Metadata as xarray-compatible attributes. |
ProcessingConfig
¶
Bases: _StrictModel
Complete processing configuration.
SiteConfig
¶
Bases: _StrictModel
Research site configuration.
validate_paired_canopies_targets()
¶
Validate that paired_canopies entries reference existing canopy receivers.
get_base_path()
¶
Get gnss_site_data_root as a Path.
Returns:
| Type | Description |
|---|---|
Path
|
Site data root directory as a Path object. |
get_canopy_receiver_names()
¶
Get names of all canopy receivers.
Returns:
| Type | Description |
|---|---|
list[str]
|
Canopy receiver names. |
receiver_of_group(group)
¶
The receiver whose data a GNSS store group holds.
A canopy receiver's group is its name; a reference receiver has one
group per paired canopy (:func:reference_store_group).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
group
|
str
|
Group name in the GNSS store, e.g. |
required |
Returns:
| Type | Description |
|---|---|
str or None
|
The receiver's name in |
resolve_paired_canopies(receiver_name)
¶
Resolve paired_canopies for a reference receiver to a list of canopy names.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
receiver_name
|
str
|
Name of the reference receiver. |
required |
Returns:
| Type | Description |
|---|---|
list[str]
|
List of canopy receiver names paired with this reference. |
validate_same_file_format()
¶
Reject a canopy and reference that record different file formats.
get_reference_canopy_pairs()
¶
Expand paired_canopies into (reference_name, canopy_name) pairs.
Returns:
| Type | Description |
|---|---|
list[tuple[str, str]]
|
List of (reference_name, canopy_name) pairs. |
SitesConfig
¶
Bases: _StrictModel
All research sites.
An empty sites: {} is a valid, unwarned state: config loading is a
foundation-layer concern used by every canvod-* package, most of
which never touch sites at all (see CLAUDE.md's "package-standalone
usage" goal). The actionable error for a missing site belongs at the
point where a site is actually looked up by name --
GnssResearchSite.__init__ (canvod-store/manager.py) already raises
a KeyError listing available sites there.
SidsConfig
¶
Bases: _StrictModel
Signal ID configuration.
validate_preset_when_mode_preset(v, info)
classmethod
¶
Ensure preset is set when mode is preset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
v
|
str | None
|
Preset name. |
required |
info
|
ValidationInfo
|
Pydantic validation info. |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
Preset value if valid. |
get_sids()
¶
Get the effective SID list.
Returns:
| Type | Description |
|---|---|
list[str] | None
|
None if mode is "all" (keep all SIDs), otherwise a SID list. |
load_config(config_dir=None, config_file=None)
¶
Load configuration from YAML files.
This is the main entry point for loading configuration.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config_dir
|
Path | None
|
Directory containing config files. If None, automatically finds monorepo root and uses {monorepo_root}/config. |
None
|
config_file
|
Path | None
|
Optional overlay YAML file applied on top of the main config.
Can also be set via the |
None
|
Returns:
| Type | Description |
|---|---|
CanvodConfig
|
Validated configuration object. |
Raises:
| Type | Description |
|---|---|
ConfigValidationError
|
If configuration YAML is invalid. |
FileNotFoundError
|
If |
Notes
CANVOD_CONFIG_DIR/CANVOD_CONFIG_FILE are resolved here, before
the cached call, not inside it. The actual caching happens in
_load_config_cached, keyed on the fully-resolved paths. Resolving
the env vars inside the cached function itself (as this used to do)
made the cache key blind to them: whichever call happened first in a
process -- e.g. canvodpy.logging.logging_config's module-level
LOGGER = configure_logging(), which called load_config() bare at
import time until 2026-10, before a CLI's --config/CANVOD_CONFIG_FILE is ever
set -- would cache the config under the no-arg key, and every later
bare load_config() call in that process (e.g. store-path/chunk-
strategy resolution deep in canvod-store) would silently get that
stale, pre-overlay config back, no matter what the env var said by
then. See canvodpy/tests/test_cli_store.py's _clear_config_cache
fixture for a hand-rolled workaround of the same symptom that predates
this fix.
Examples:
>>> from canvod.config import load_config
>>> config = load_config()
>>> print(config.nasa_earthdata_acc_mail)
>>> print(config.processing.aux_data.agency)
Models¶
Pydantic models for canvodpy configuration.
These models provide: - Type validation for all configuration values - Serialization support (YAML/JSON/dict) - API-ready data transfer objects - IDE autocomplete and type hints
Split across focused submodules by config section —
this __init__.py re-exports everything, so from canvod.config.models
import X keeps working exactly as it did when this was a single file.
ProcessingParams
¶
Bases: _StrictModel
Processing parameters.
Notes
This is a Pydantic model for configuration validation.
validate_resource_mode()
¶
Validate resource_mode constraints.
In 'manual' mode, n_max_threads must be set.
In 'auto' mode, n_max_threads is ignored with a warning if set.
resolve_resources()
¶
Resolve effective resource settings based on resource_mode.
Returns:
| Type | Description |
|---|---|
dict
|
Resolved resource values with keys: |
ReceiverConfig
¶
Bases: _StrictModel
Receiver configuration.
validate_paired_canopies()
¶
Validate paired_canopies is required for reference, forbidden for canopy.
PreprocessingConfig
¶
Bases: _StrictModel
Operations applied before the data are written to the GNSS store.
Each operation is applied only if its section is set (and enabled).
Temporal aggregation runs first, then the grid cell assignment.
TemporalAggregationConfig
¶
Bases: _StrictModel
Temporal aggregation of the observations into time bins.
GridAssignmentConfig
¶
Bases: _StrictModel
Assignment of each observation to a cell of a hemispherical grid.
StorageConfig
¶
Bases: _StrictModel
Storage strategy configuration.
Notes
This is a Pydantic model for configuration validation.
validate_stores_dir(v)
classmethod
¶
Expand ~, keep an S3 URL, reject placeholders and other URLs.
Checked before the value becomes a Path, which would turn
s3://bucket/prefix into the local path s3:/bucket/prefix.
An S3 URL stays a string, as s3://bucket/prefix without a
trailing slash (#350).
get_gnss_store_path(site_name)
¶
Get the GNSS store path for a site.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
site_name
|
str
|
Site name. |
required |
Returns:
| Type | Description |
|---|---|
Path | str
|
Path to the site's GNSS store, or its |
get_vod_store_path(site_name)
¶
Get the VOD store path for a site.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
site_name
|
str
|
Site name. |
required |
Returns:
| Type | Description |
|---|---|
Path | str
|
Path to the site's VOD store, or its |
get_statistics_store_path(site_name)
¶
Get the statistics store path for a site.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
site_name
|
str
|
Site name. |
required |
Returns:
| Type | Description |
|---|---|
Path | str
|
Path to the site's statistics store, or its |
get_rollup_store_path(site_name)
¶
Get the hemigrid rollup store path for a site.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
site_name
|
str
|
Site name. |
required |
Returns:
| Type | Description |
|---|---|
Path | str
|
Path to the site's hemigrid rollup store, or its |
get_aux_data_dir()
¶
Get the directory for auxiliary data files.
Returns:
| Type | Description |
|---|---|
Path
|
Aux data directory (configured or system temp). |
get_shared_aux_cache_dir()
¶
Get the network-wide shared aux cache directory, if configured.
Returns:
| Type | Description |
|---|---|
Path | None
|
The configured shared cache root, created if missing, or
|
Setting help¶
What each setting means, from the descriptions on the models: the text of
canvodpy config explain and of the Settings reference.
What each setting of canvod-settings.yaml means, from the models.
The descriptions on the pydantic models are the one source of the help
texts: canvodpy config explain, the ? answer of
canvodpy config init --interactive, the settings reference page
(scripts/settings_docs.py) and the links in the settings template all
come from here.
A setting is named by its path in the file, with a placeholder for the
names you choose: sites.<site>.receivers.<receiver>.directory. An item
of a list is named with []: processing.references.funding[].funder.
SettingHelp
dataclass
¶
One setting, or one section of settings, and what it means.
default is the value as written in the file (null for none);
it is None when the setting has to be set, and for a section.
UnknownSettingError
¶
Bases: KeyError
No setting has the path asked for.
explain(path)
¶
What the setting at path means, as plain text.
Raises:
| Type | Description |
|---|---|
UnknownSettingError
|
If path names no setting (see :func: |
find_setting(path)
¶
The setting at path, as written in the file or ending the path.
sites.my_site.receivers.reference_01.directory finds
sites.<site>.receivers.<receiver>.directory, and gnss_store.epoch
finds processing.icechunk.chunk_strategies.gnss_store.epoch.
Raises:
| Type | Description |
|---|---|
UnknownSettingError
|
If no setting, or more than one, ends with path; the message names the closest settings. |
all_settings()
¶
Every setting and section, in the order of the models.