Skip to content

canvod.readers API reference

RINEX observation file parsing with validation and GNSS signal specifications.

Package

GNSS data format readers.

This package provides readers for various GNSS data formats, all implementing a common interface for seamless integration with processing pipelines.

Supported formats: - RINEX v3.04 observations (Rnxv3Obs) - RINEX v2.11 observations (Rnxv2Obs) - Septentrio Binary Format, SBF (SbfReader) - NMEA (NmeaObs)

Quick Start

 
from canvod.readers import Rnxv3Obs

# Read RINEX v3 file
reader = Rnxv3Obs(fpath="station.24o")
dataset = reader.to_ds()

Or use the canvodpy factory, which picks the reader from the file's content (RINEX v2, RINEX v3 or NMEA): 

from canvodpy import ReaderFactory

reader = ReaderFactory.create_from_file("station.24o")
dataset = reader.to_ds()

GNSSDataReader

Bases: BaseModel, ABC

Abstract base class for all GNSS data format readers.

All readers must: 1. Inherit from this class 2. Implement all abstract methods 3. Return xarray.Dataset that passes :func:validate_dataset 4. Provide file hash for deduplication

This ensures compatibility with: - canvod-vod: VOD calculation - canvod-store: MyIcechunkStore storage - canvod-grids: Grid projection operations

Subclasses may override model_config to set frozen, extra, etc. The base class provides arbitrary_types_allowed=True which is needed by readers that use pint.Quantity or similar third-party types.

Examples:

>>> class Rnxv3Obs(GNSSDataReader):
...     def to_ds(self, **kwargs) -> xr.Dataset:
...         # Implementation
...         return dataset
...
>>> reader = Rnxv3Obs(fpath="station.24o")
>>> ds = reader.to_ds()
>>> validate_dataset(ds)

source_format abstractmethod property

Return the format identifier for this reader (e.g. "rinex3", "sbf").

A run puts it on each file's dataset (attribute "Source Format") and the GNSS store records it as its source_format: the root attribute and identity.source_format of the store's metadata. A run refuses files of different formats in one store.

file_hash abstractmethod property

Return SHA256 hash of file for deduplication.

Used by MyIcechunkStore to avoid duplicate ingestion. Must be deterministic and reproducible.

Returns:

Type Description
str

Short hash (16 chars) or full hash of file content

start_time abstractmethod property

Return start time of observations.

Returns:

Type Description
datetime

First observation timestamp in the file.

end_time abstractmethod property

Return end time of observations.

Returns:

Type Description
datetime

Last observation timestamp in the file.

systems abstractmethod property

Return list of GNSS systems in file.

Returns:

Type Description
list of str

System identifiers: 'G', 'R', 'E', 'C', 'J', 'S', 'I'

num_epochs property

Return number of epochs in file.

Default implementation iterates epochs. Subclasses may override with a faster approach.

Returns:

Type Description
int

Total number of observation epochs.

num_satellites abstractmethod property

Return total number of unique satellites observed.

Returns:

Type Description
int

Count of unique satellite vehicles across all systems.

to_ds(keep_data_vars=None, **kwargs) abstractmethod

Convert data to xarray.Dataset.

Must return Dataset with structure: - Dims: (epoch, sid) - Coords: epoch, sid, sv, system, band, code, freq_* - Data vars: At minimum SNR - Attrs: Must include "File Hash"

Parameters:

Name Type Description Default
keep_data_vars list of str

Data variables to include. If None, includes all available.

 None
**kwargs object

Implementation-specific parameters

 {}

Returns:

Type Description
Dataset

Dataset that passes :func:validate_dataset.

iter_epochs() abstractmethod

Iterate over epochs in the file.

Yields:

Type Description
Epoch

Parsed epoch with satellites and observations.

to_ds_and_auxiliary(keep_data_vars=None, **kwargs)

Produce the obs dataset and any auxiliary datasets in a single call.

Default: calls to_ds(**kwargs) and returns an empty auxiliary dict. Readers that produce metadata (e.g. SBF) override this to collect both in a single file scan.

Returns:

Type Description
tuple[Dataset, dict[str, Dataset]]

(obs_ds, {"name": aux_ds, ...}). Auxiliary dict is empty for readers with no extra data (RINEX v2/v3).

__repr__()

Return the string representation.

SignalID

Bases: BaseModel

Validated signal identifier (SV + band + code).

sid = SignalID(sv="G01", band="L1", code="C") str(sid) 'G01|L1|C' sid.system 'G'

system property

GNSS system letter (e.g. 'G' for GPS).

sid property

Full signal ID string ('SV|band|code').

from_string(sid_str) classmethod

Parse a signal ID string ('SV|band|code') into a SignalID.

Parameters:

Name Type Description Default
sid_str str

Signal ID in 'SV|band|code' format (e.g. 'G01|L1|C').

 required

Returns:

Type Description
SignalID

Validated signal identifier.

Raises:

Type Description
ValueError

If the string does not have exactly three pipe-separated parts.

DatasetBuilder

Guided builder for constructing valid GNSSDataReader output Datasets.

Handles coordinate arrays, dtype enforcement, frequency resolution, and contract validation automatically.

Parameters:

Name Type Description Default
reader GNSSDataReader

The reader instance (used for _build_attrs() and file hash).

 required
time_system str

Time scale of the epoch timestamps as given to :meth:add_epoch, recorded in the epoch coordinate's time_system attribute. Must be "GPS": canVODpy stores every epoch in GPS time, so convert other time scales (for example UTC, plus the leap seconds from :func:canvod.readers.gnss_specs.leap_seconds.utc_to_gps) before :meth:add_epoch. Required, since the builder cannot infer it from the timestamps; anything else raises ValueError.

 required
aggregate_glonass_fdma bool

Whether to aggregate GLONASS FDMA channels (default True).

 True

Examples:

>>> builder = DatasetBuilder(reader, time_system="GPS")
>>> for epoch in reader.iter_epochs():
...     ei = builder.add_epoch(epoch.timestamp)
...     for obs in epoch.observations:
...         sig = builder.add_signal(sv="G01", band="L1", code="C")
...         builder.set_value(ei, sig, "SNR", 42.0)
>>> ds = builder.build()   # validated Dataset

add_epoch(timestamp)

Register an epoch timestamp. Returns epoch index.

add_signal(sv, band, code)

Register a signal (idempotent). Returns validated SignalID.

set_value(epoch_idx, signal, var, value)

Set a data value for a given epoch, signal, and variable.

Parameters:

Name Type Description Default
epoch_idx int

Index returned by :meth:add_epoch.

 required
signal SignalID or str

Signal identifier (SignalID or 'SV|band|code' string).

 required
var str

Variable name (e.g. 'SNR', 'Pseudorange', 'Phase').

 required
value float

The observation value.

 required

build(keep_data_vars=None, extra_attrs=None)

Build, validate, and return the Dataset.

  1. Sorts signals alphabetically
  2. Resolves frequencies from band names via SignalIDMapper
  3. Constructs coordinate arrays with correct dtypes (float32 for freq)
  4. Attaches CF-compliant metadata from COORDS_METADATA
  5. Calls validate_dataset() before returning

Parameters:

Name Type Description Default
keep_data_vars list of str

If provided, only include these data variables. If None, includes all variables that had values set.

 None
extra_attrs dict

Additional global attributes to merge into the Dataset.

 None

Returns:

Type Description
Dataset

Validated Dataset with dimensions (epoch, sid).

DatasetStructureValidator

Bases: BaseModel

Validates that an xarray.Dataset meets the GNSSDataReader contract.

Wraps a Dataset and checks it against the contract constants above. Use this in tests and reader implementations to catch structural errors early with clear messages.

Examples:

>>> validator = DatasetStructureValidator(dataset=ds)
>>> validator.validate_all()          # raises ValueError on any violation
>>> validator.validate_dimensions()   # check just one aspect

validate_all(required_vars=None)

Run all validations, collecting all errors.

Delegates to :func:validate_dataset so the logic lives in one place.

validate_dimensions()

Check that required dimensions (epoch, sid) exist.

validate_coordinates()

Check that required coordinates exist with correct dtypes.

validate_data_variables(required_vars=None)

Check that required data variables exist with correct dims.

validate_attributes()

Check that required global attributes are present.

Rnxv3Obs

Bases: GNSSDataReader

RINEX v3.04 observation reader.

Attributes:

Name Type Description
fpath Path

Path to the RINEX observation file.
polarization str, default "RHCP"

Polarization label for observables.
completeness_mode {"strict", "warn", "off"}, default "strict"

Behavior when epoch completeness checks fail.
expected_dump_interval (str or Quantity, optional)

Expected file dump interval for completeness validation.
expected_sampling_interval (str or Quantity, optional)

Expected sampling interval for completeness validation.
apply_overlap_filter bool, default False

Whether to filter overlapping signal groups.
overlap_preferences (dict[str, str], optional)

Preferred signals for overlap resolution.
aggregate_glonass_fdma (bool, optional)

Whether to aggregate GLONASS FDMA channels.

Notes

Inherits fpath, its validator, and arbitrary_types_allowed from :class:GNSSDataReader.

header property

Expose validated header (read-only).

Returns:

Type Description
Rnxv3Header

Parsed and validated RINEX header.

file_hash property

Return cached SHA256 short hash of the file content.

Returns:

Type Description
str

16-character short hash for deduplication.

start_time property

Return start time of observations from header.

Returns:

Type Description
datetime

First observation timestamp.

end_time property

Return end time of observations from last epoch.

Returns:

Type Description
datetime

Last observation timestamp.

systems property

Return list of GNSS systems in file.

Returns:

Type Description
list of str

System identifiers (G, R, E, C, J, S, I).

num_epochs property

Return number of epochs in file.

Returns:

Type Description
int

Total epoch count.

num_satellites property

Return total number of unique satellites observed.

Returns:

Type Description
int

Count of unique satellite vehicles across all systems.

epochs property

Materialize all epochs (legacy compatibility).

Returns:

Type Description
list of Rnxv3ObsEpochRecord

All epochs in memory (use iter_epochs for efficiency)

__str__()

Return a human-readable summary.

__repr__()

Return a concise representation for debugging.

get_epoch_record_batches(epoch_record_indicator=EPOCH_RECORD_INDICATOR)

Get the start and end line numbers for each epoch in the file.

Parameters:

Name Type Description Default
epoch_record_indicator str

Character marking epoch record lines.

 '>'

Returns:

Type Description
list of tuple of int

List of (start_line, end_line) pairs for each epoch.

process_satellite_data(s)

Process satellite data line into a Satellite object with observations.

Reads the line with the rules of the dataset builder (see :meth:_read_satellite_line); a field that is neither blank nor in the standard form raises :class:InvalidEpochError.

iter_epochs()

Yield epochs one by one instead of materializing the whole list.

Returns:

Type Description
Generator

Generator yielding Rnxv3ObsEpochRecord objects

Yields:

Type Description
Rnxv3ObsEpochRecord

Each epoch with timestamp and satellite observations

iter_epochs_in_range(start, end)

Yield epochs lazily that fall into the given datetime range.

Parameters:

Name Type Description Default
start datetime

Start of time range (inclusive)

 required
end datetime

End of time range (inclusive)

 required

Returns:

Type Description
Generator

Generator yielding epochs in the specified range

Yields:

Type Description
Rnxv3ObsEpochRecord

Epochs within the time range

get_datetime_from_epoch_record_info(epoch_record_info)

Convert epoch record info to datetime object.

Parameters:

Name Type Description Default
epoch_record_info Rnxv3ObsEpochRecordLineModel

Parsed epoch record line

 required

Returns:

Type Description
datetime

Timestamp from epoch record

epochrecordinfo_dt_to_numpy_dt(epch) staticmethod

Return the epoch record's time as numpy datetime64[ns].

Parameters:

Name Type Description Default
epch Rnxv3ObsEpochRecord

Epoch record containing timestamp info

 required

Returns:

Type Description
datetime64

Numpy datetime64 with nanosecond precision

infer_sampling_interval()

Infer sampling interval from consecutive epoch deltas.

Returns:

Type Description
Quantity or None

Sampling interval in seconds, or None if cannot be inferred

infer_dump_interval(sampling_interval=None)

Infer the dump interval from the time the epochs span.

The span from the first to the last epoch plus one sampling interval. Epochs missing between the first and the last epoch therefore show up in the completeness check; epochs missing before the first or after the last epoch (e.g. a file cut short) do not, since only the file name or the caller knows the intended length.

Parameters:

Name Type Description Default
sampling_interval Quantity

Known sampling interval. Inferred from the epochs if not given.

 None

Returns:

Type Description
Quantity or None

Dump interval in seconds, or None if it cannot be inferred (fewer than two epochs).

validate_epoch_completeness(dump_interval=None, sampling_interval=None)

Validate that the number of epochs matches the expected dump interval.

The file must hold dump interval / sampling interval epochs.

Parameters:

Name Type Description Default
dump_interval str or Quantity

Expected file dump interval (e.g. "15 min"). If None, the span of the epochs is used (:meth:infer_dump_interval), which detects gaps between the first and the last epoch but not a file cut short.

 None
sampling_interval str or Quantity

Expected sampling interval. If None, the most common interval between consecutive epochs.

 None

Returns:

Type Description
None

Raises:

Type Description
MissingEpochError

If total sampling time doesn't match dump interval
ValueError

If intervals cannot be inferred

filter_by_overlapping_groups(ds, group_preference=None)

Filter overlapping bands using per-group preferences.

Parameters:

Name Type Description Default
ds Dataset

Dataset with sid dimension and signal properties.

 required
group_preference dict[str, str]

Mapping of overlap group to preferred band.

 None

Returns:

Type Description
Dataset

Dataset filtered to preferred overlapping bands.

create_rinex_netcdf_with_signal_id(start=None, end=None, parser='validated')

Create a NetCDF dataset with signal IDs.

Optionally restricts to epochs within a datetime range via post-filtering.

Parameters:

Name Type Description Default
start datetime

Start of time range (inclusive).

 None
end datetime

End of time range (inclusive).

 None
parser (validated, unvalidated_fast)

As for :meth:to_ds.

 "validated"

Returns:

Type Description
Dataset

Dataset with dimensions (epoch, sid).

to_ds(keep_data_vars=None, **kwargs)

Convert RINEX observations to xarray.Dataset with signal ID structure.

Parameters:

Name Type Description Default
outname Path or str

If provided, saves dataset to this file path

 required
keep_data_vars list of str or None

Data variables to include in dataset. None includes all available. A run passes the processing.params.keep_gnss_observables setting. The validated parser reads and checks only the observation fields these variables need, so a malformed field of a left-out variable does not drop its epoch.

 None
write_global_attrs bool

If True, adds comprehensive global attributes

 False
pad_global_sid bool

If True, pads to global signal ID space

 True
strip_fillval bool

If True, removes fill values

 True
add_future_datavars bool

If True, adds placeholder variables for future data

 True
keep_sids list of str or None

If provided, filters/pads dataset to these specific SIDs. If None and pad_global_sid=True, pads to all possible SIDs.

 None
parser (validated, unvalidated_fast)

"validated" checks every epoch (epoch line, satellite IDs, satellite count, the observation fields of the kept variables) and drops (and logs) epochs that fail; see :meth:_create_dataset_validated. "unvalidated_fast" is DANGEROUS: it slices fixed columns without any check, so corrupted records can enter the dataset as partial or wrong values; canVODpy takes no responsibility for its results, and every use emits an :class:UnvalidatedParserWarning. Both give the same dataset for a valid file. A run passes the processing.params.rinex_v3_parser setting.

 "validated"

Returns:

Type Description
Dataset

Dataset with dimensions (epoch, sid) and requested data variables

validate_rinex_304_compliance(ds=None, strict=False, print_report=True)

Run enhanced RINEX 3.04 specification validation.

Validates: 1. System-specific observation codes 2. GLONASS mandatory fields (slot/frequency, biases) 3. Phase shift records (RINEX 3.01+) 4. Observation value ranges

Parameters:

Name Type Description Default
ds Dataset

Dataset to validate. If None, creates one from current file.

 None
strict bool

If True, raise ValueError on validation failures

 False
print_report bool

If True, print validation report to console

 True

Returns:

Type Description
dict[str, list[str]]

Validation results by category

Examples:

>>> reader = Rnxv3Obs(fpath="station.24o")
>>> results = reader.validate_rinex_304_compliance()
>>> # Or validate a specific dataset
>>> ds = reader.to_ds()
>>> results = reader.validate_rinex_304_compliance(ds=ds)

SbfReader

Bases: GNSSDataReader

Read and decode a Septentrio Binary Format (SBF) observation file.

Parameters:

Name Type Description Default
fpath Path

Path to the *.sbf (or *.SBF, or receiver-named) binary file.

 required
aggregate_glonass_fdma bool

Must be True: GLONASS FDMA signals are always given as aggregated G1/G2 bands; False is refused.

 True

Examples:

>>> reader = SbfReader(fpath=Path("rref213a00.25_"))
>>> print(reader.header.rx_version)
4.14.4
>>> for epoch in reader.iter_epochs():
...     for obs in epoch.observations:
...         print(obs.system, obs.prn, obs.cn0)

Notes
  • All physical-unit conversions follow RefGuide-4.14.0.
  • :meth:iter_epochs, :meth:to_ds and :meth:to_ds_and_auxiliary share one decoder (:func:_decode_observations) and give the same values.
  • :meth:iter_epochs returns :class:pint.Quantity objects using the shared :data:~canvod.readers.gnss_specs.constants.UREG; the datasets hold plain floats.
  • The file is scanned once per :meth:iter_epochs call; use :attr:num_epochs for a pre-computed count (scans once on first access).
  • Inherits fpath, its validator, and arbitrary_types_allowed from :class:GNSSDataReader.

file_hash cached property

SHA-256 hex digest of the file (first 16 characters).

Returns:

Type Description
str

16-character hexadecimal prefix of the SHA-256 hash.

start_time cached property

Return the timestamp of the first decoded epoch.

Returns:

Type Description
datetime

First observation epoch, in GPS time.

Raises:

Type Description
LookupError

If the file contains no decodable epochs.

end_time cached property

Return the timestamp of the last decoded epoch.

Returns:

Type Description
datetime

Last observation epoch, in GPS time.

Raises:

Type Description
LookupError

If the file contains no decodable epochs.

systems cached property

Return sorted list of GNSS system codes present in the file.

Returns:

Type Description
list of str

Sorted list of RINEX system letters (e.g. ["E", "G", "R"]).

num_satellites cached property

Return the number of unique satellites observed in the file.

Returns:

Type Description
int

Count of unique system + PRN pairs across all epochs.

num_epochs cached property

Count the observation epochs (MeasEpoch blocks) in the file.

Returns:

Type Description
int

Number of epochs :meth:iter_epochs yields.

Notes

Scans the entire file once; result is cached.

header cached property

Parse the first ReceiverSetup block in the file.

Returns:

Type Description
SbfHeader

Receiver metadata.

Raises:

Type Description
LookupError

If no ReceiverSetup block is found.

iter_epochs()

Iterate over the observation epochs of the file.

Yields:

Type Description
SbfEpoch

One decoded observation epoch, with its signal observations in physical units as :class:pint.Quantity.

Notes
  • The file is scanned from start to finish on each call.
  • cn0 includes the MeasExtra CN0HighRes value of the same epoch where it is logged, as the SNR variable of :meth:to_ds.
  • Signals not in the signal table are skipped.

to_ds(keep_data_vars=None, pad_global_sid=True, strip_fillval=True, **kwargs)

Convert SBF observations to an (epoch, sid) xarray Dataset.

Produces the same structure as :class:~canvod.readers.rinex.v3_04.Rnxv3Obs and passes :func:~canvod.readers.base.validate_dataset. The values are those of the observation dataset of :meth:to_ds_and_auxiliary.

Parameters:

Name Type Description Default
keep_data_vars list of str

Data variables to retain. If None, all are kept: SNR, Pseudorange, Phase, Doppler, SSI, Smoothing, HalfCycle. LLI is not produced: SBF has no loss-of-lock indicator.

 None
pad_global_sid bool

If True, pads the dataset to the global SID space via :func:canvod.readers.preprocessing.pad_to_global_sid.

 True
strip_fillval bool

If True, removes fill values via :func:canvod.readers.preprocessing.strip_fillvalue.

 True
**kwargs object

keep_sids is forwarded to pad_to_global_sid; others are ignored.

 {}

Returns:

Type Description
Dataset

Dataset with dimensions (epoch, sid) that passes :func:~canvod.readers.base.validate_dataset.

to_ds_and_auxiliary(keep_data_vars=None, pad_global_sid=True, strip_fillval=True, store_raw_observables=True, **kwargs)

Decode the observations and the SBF metadata in one file scan.

Parameters:

Name Type Description Default
keep_data_vars list of str

Data variables to retain in the obs dataset.

 None
pad_global_sid bool

Pad obs dataset to the global SID space.

 True
strip_fillval bool

Strip fill values from the obs dataset.

 True
store_raw_observables bool

Add pre-correction "raw" observable variables to the obs dataset: SNR_raw, Pseudorange_unsmoothed, Pseudorange_raw, Phase_raw. Set to False to reduce dataset size when these are not needed.

 True
**kwargs object

keep_sids is forwarded to pad_to_global_sid; others are ignored.

 {}

Returns:

Type Description
tuple[Dataset, dict[str, Dataset]]

(obs_ds, {"sbf_obs": meta_ds}). meta_ds has the sids of obs_ds; it holds the broadcast satellite geometry (SatVisibility), the MeasExtra fields, and the receiver, PVT and quality data of each epoch.

__repr__()

Return a short string representation.

MatchedDirs dataclass

Matched directory paths for canopy and reference receivers.

Immutable container representing a pair of directories containing RINEX data for the same date.

Parameters:

Name Type Description Default
canopy_data_dir Path

Path to canopy receiver RINEX directory.

 required
reference_data_dir Path

Path to reference (open-sky) receiver RINEX directory.

 required
yyyydoy YYYYDOY

Date object for this matched pair.

 required

Examples:

>>> from pathlib import Path
>>> from canvod.utils.tools import YYYYDOY
>>>
>>> md = MatchedDirs(
...     canopy_data_dir=Path("/data/02_canopy/25001"),
...     reference_data_dir=Path("/data/01_reference/25001"),
...     yyyydoy=YYYYDOY.from_str("2025001")
... )
>>> md.yyyydoy.to_str()
'2025001'

PairMatchedDirs dataclass

Matched directories for a receiver pair on a specific date.

Supports multi-receiver configurations where multiple canopy/reference pairs may exist at the same site.

Parameters:

Name Type Description Default
yyyydoy YYYYDOY

Date for this matched pair.

 required
pair_name str

Identifier for this receiver pair (e.g., "pair_01").

 required
canopy_receiver str

Name of canopy receiver (e.g., "canopy_01").

 required
reference_receiver str

Name of reference receiver (e.g., "reference_01").

 required
canopy_data_dir Path

Path to canopy receiver RINEX directory.

 required
reference_data_dir Path

Path to reference receiver RINEX directory.

 required

Examples:

>>> pmd = PairMatchedDirs(
...     yyyydoy=YYYYDOY.from_str("2025001"),
...     pair_name="pair_01",
...     canopy_receiver="canopy_01",
...     reference_receiver="reference_01",
...     canopy_data_dir=Path("/data/canopy_01/25001"),
...     reference_data_dir=Path("/data/reference_01/25001")
... )
>>> pmd.pair_name
'pair_01'

sid_coords(signals, *, mapper)

Coordinates of the sid dimension for the given signals.

sid and its per-signal coordinates (sv, system, band, code, freq_center, freq_min, freq_max) as required by :data:~canvod.readers.base.REQUIRED_COORDS, with their metadata. Frequencies come from the band; a band without a known frequency gets NaN.

Parameters:

Name Type Description Default
signals list of SignalID

Signals in the order of the sid dimension.

 required
mapper SignalIDMapper

Band lookup for the frequencies.

 required

Returns:

Type Description
dict

Coordinates to pass to :class:xarray.Dataset.

validate_dataset(ds, required_vars=None)

Validate ds meets the GNSSDataReader output contract.

Collects all violations and raises a single ValueError listing every problem, rather than stopping at the first failure.

Parameters:

Name Type Description Default
ds Dataset

Dataset to validate.

 required
required_vars list of str

Data variables that must be present. Defaults to :data:DEFAULT_REQUIRED_VARS (["SNR"]).

 None

Raises:

Type Description
ValueError

If any contract violation is found.

validate_vod_dataset(ds)

Validate ds meets the VOD dataset contract.

The contract of VOD calculator output and of what the VOD store accepts: the dimensions and coordinates of :func:validate_dataset, the variables :data:VOD_REQUIRED_VARS (theta and phi in radians), every variable on (epoch, sid). Collects all violations and raises a single ValueError.

Parameters:

Name Type Description Default
ds Dataset

Dataset to validate.

 required

Raises:

Type Description
ValueError

If any contract violation is found.

RINEX v3.04

RINEX v3.04 observation file reader.

Classes: - Rnxv3Header: Parse RINEX v3 headers - Rnxv3Obs: Main reader class, converts RINEX to xarray Dataset

UnvalidatedParserWarning

Bases: UserWarning

Emitted on every use of the unvalidated fast RINEX v3 parser.

Rnxv3Header

Bases: BaseModel

Enhanced RINEX v3 header following the original implementation logic.

Key changes from previous version: - date field is now datetime (like original) - Uses the original parsing logic for __get_pgm_runby_date

Notes

This is a Pydantic BaseModel configured with ConfigDict (frozen, validate_assignment, arbitrary_types_allowed, str_strip_whitespace). Prefer :meth:from_file for construction.

time_system property

Time system of all time tags in the file (TIME OF FIRST OBS).

is_mixed_systems property

Check if the RINEX file contains mixed GNSS systems.

parse_marker_number(v) classmethod

Convert empty strings to None, parse valid integers.

from_file(fpath) classmethod

Create header from a RINEX file.

__repr__()

Return a concise representation for debugging.

__str__()

Return a human-readable header summary.

Rnxv3Obs

Bases: GNSSDataReader

RINEX v3.04 observation reader.

Attributes:

Name Type Description
fpath Path

Path to the RINEX observation file.
polarization str, default "RHCP"

Polarization label for observables.
completeness_mode {"strict", "warn", "off"}, default "strict"

Behavior when epoch completeness checks fail.
expected_dump_interval (str or Quantity, optional)

Expected file dump interval for completeness validation.
expected_sampling_interval (str or Quantity, optional)

Expected sampling interval for completeness validation.
apply_overlap_filter bool, default False

Whether to filter overlapping signal groups.
overlap_preferences (dict[str, str], optional)

Preferred signals for overlap resolution.
aggregate_glonass_fdma (bool, optional)

Whether to aggregate GLONASS FDMA channels.

Notes

Inherits fpath, its validator, and arbitrary_types_allowed from :class:GNSSDataReader.

header property

Expose validated header (read-only).

Returns:

Type Description
Rnxv3Header

Parsed and validated RINEX header.

file_hash property

Return cached SHA256 short hash of the file content.

Returns:

Type Description
str

16-character short hash for deduplication.

start_time property

Return start time of observations from header.

Returns:

Type Description
datetime

First observation timestamp.

end_time property

Return end time of observations from last epoch.

Returns:

Type Description
datetime

Last observation timestamp.

systems property

Return list of GNSS systems in file.

Returns:

Type Description
list of str

System identifiers (G, R, E, C, J, S, I).

num_epochs property

Return number of epochs in file.

Returns:

Type Description
int

Total epoch count.

num_satellites property

Return total number of unique satellites observed.

Returns:

Type Description
int

Count of unique satellite vehicles across all systems.

epochs property

Materialize all epochs (legacy compatibility).

Returns:

Type Description
list of Rnxv3ObsEpochRecord

All epochs in memory (use iter_epochs for efficiency)

__str__()

Return a human-readable summary.

__repr__()

Return a concise representation for debugging.

get_epoch_record_batches(epoch_record_indicator=EPOCH_RECORD_INDICATOR)

Get the start and end line numbers for each epoch in the file.

Parameters:

Name Type Description Default
epoch_record_indicator str

Character marking epoch record lines.

 '>'

Returns:

Type Description
list of tuple of int

List of (start_line, end_line) pairs for each epoch.

process_satellite_data(s)

Process satellite data line into a Satellite object with observations.

Reads the line with the rules of the dataset builder (see :meth:_read_satellite_line); a field that is neither blank nor in the standard form raises :class:InvalidEpochError.

iter_epochs()

Yield epochs one by one instead of materializing the whole list.

Returns:

Type Description
Generator

Generator yielding Rnxv3ObsEpochRecord objects

Yields:

Type Description
Rnxv3ObsEpochRecord

Each epoch with timestamp and satellite observations

iter_epochs_in_range(start, end)

Yield epochs lazily that fall into the given datetime range.

Parameters:

Name Type Description Default
start datetime

Start of time range (inclusive)

 required
end datetime

End of time range (inclusive)

 required

Returns:

Type Description
Generator

Generator yielding epochs in the specified range

Yields:

Type Description
Rnxv3ObsEpochRecord

Epochs within the time range

get_datetime_from_epoch_record_info(epoch_record_info)

Convert epoch record info to datetime object.

Parameters:

Name Type Description Default
epoch_record_info Rnxv3ObsEpochRecordLineModel

Parsed epoch record line

 required

Returns:

Type Description
datetime

Timestamp from epoch record

epochrecordinfo_dt_to_numpy_dt(epch) staticmethod

Return the epoch record's time as numpy datetime64[ns].

Parameters:

Name Type Description Default
epch Rnxv3ObsEpochRecord

Epoch record containing timestamp info

 required

Returns:

Type Description
datetime64

Numpy datetime64 with nanosecond precision

infer_sampling_interval()

Infer sampling interval from consecutive epoch deltas.

Returns:

Type Description
Quantity or None

Sampling interval in seconds, or None if cannot be inferred

infer_dump_interval(sampling_interval=None)

Infer the dump interval from the time the epochs span.

The span from the first to the last epoch plus one sampling interval. Epochs missing between the first and the last epoch therefore show up in the completeness check; epochs missing before the first or after the last epoch (e.g. a file cut short) do not, since only the file name or the caller knows the intended length.

Parameters:

Name Type Description Default
sampling_interval Quantity

Known sampling interval. Inferred from the epochs if not given.

 None

Returns:

Type Description
Quantity or None

Dump interval in seconds, or None if it cannot be inferred (fewer than two epochs).

validate_epoch_completeness(dump_interval=None, sampling_interval=None)

Validate that the number of epochs matches the expected dump interval.

The file must hold dump interval / sampling interval epochs.

Parameters:

Name Type Description Default
dump_interval str or Quantity

Expected file dump interval (e.g. "15 min"). If None, the span of the epochs is used (:meth:infer_dump_interval), which detects gaps between the first and the last epoch but not a file cut short.

 None
sampling_interval str or Quantity

Expected sampling interval. If None, the most common interval between consecutive epochs.

 None

Returns:

Type Description
None

Raises:

Type Description
MissingEpochError

If total sampling time doesn't match dump interval
ValueError

If intervals cannot be inferred

filter_by_overlapping_groups(ds, group_preference=None)

Filter overlapping bands using per-group preferences.

Parameters:

Name Type Description Default
ds Dataset

Dataset with sid dimension and signal properties.

 required
group_preference dict[str, str]

Mapping of overlap group to preferred band.

 None

Returns:

Type Description
Dataset

Dataset filtered to preferred overlapping bands.

create_rinex_netcdf_with_signal_id(start=None, end=None, parser='validated')

Create a NetCDF dataset with signal IDs.

Optionally restricts to epochs within a datetime range via post-filtering.

Parameters:

Name Type Description Default
start datetime

Start of time range (inclusive).

 None
end datetime

End of time range (inclusive).

 None
parser (validated, unvalidated_fast)

As for :meth:to_ds.

 "validated"

Returns:

Type Description
Dataset

Dataset with dimensions (epoch, sid).

to_ds(keep_data_vars=None, **kwargs)

Convert RINEX observations to xarray.Dataset with signal ID structure.

Parameters:

Name Type Description Default
outname Path or str

If provided, saves dataset to this file path

 required
keep_data_vars list of str or None

Data variables to include in dataset. None includes all available. A run passes the processing.params.keep_gnss_observables setting. The validated parser reads and checks only the observation fields these variables need, so a malformed field of a left-out variable does not drop its epoch.

 None
write_global_attrs bool

If True, adds comprehensive global attributes

 False
pad_global_sid bool

If True, pads to global signal ID space

 True
strip_fillval bool

If True, removes fill values

 True
add_future_datavars bool

If True, adds placeholder variables for future data

 True
keep_sids list of str or None

If provided, filters/pads dataset to these specific SIDs. If None and pad_global_sid=True, pads to all possible SIDs.

 None
parser (validated, unvalidated_fast)

"validated" checks every epoch (epoch line, satellite IDs, satellite count, the observation fields of the kept variables) and drops (and logs) epochs that fail; see :meth:_create_dataset_validated. "unvalidated_fast" is DANGEROUS: it slices fixed columns without any check, so corrupted records can enter the dataset as partial or wrong values; canVODpy takes no responsibility for its results, and every use emits an :class:UnvalidatedParserWarning. Both give the same dataset for a valid file. A run passes the processing.params.rinex_v3_parser setting.

 "validated"

Returns:

Type Description
Dataset

Dataset with dimensions (epoch, sid) and requested data variables

validate_rinex_304_compliance(ds=None, strict=False, print_report=True)

Run enhanced RINEX 3.04 specification validation.

Validates: 1. System-specific observation codes 2. GLONASS mandatory fields (slot/frequency, biases) 3. Phase shift records (RINEX 3.01+) 4. Observation value ranges

Parameters:

Name Type Description Default
ds Dataset

Dataset to validate. If None, creates one from current file.

 None
strict bool

If True, raise ValueError on validation failures

 False
print_report bool

If True, print validation report to console

 True

Returns:

Type Description
dict[str, list[str]]

Validation results by category

Examples:

>>> reader = Rnxv3Obs(fpath="station.24o")
>>> results = reader.validate_rinex_304_compliance()
>>> # Or validate a specific dataset
>>> ds = reader.to_ds()
>>> results = reader.validate_rinex_304_compliance(ds=ds)

adapt_existing_rnxv3obs_class(original_class_path=None)

Provide guidance to integrate the enhanced sid functionality.

This function provides guidance on how to modify the existing class to support the new sid structure alongside the current OFT structure.

Returns:

Type Description
str

Integration instructions

Base reader

Abstract base class for GNSS data readers.

Defines interface that all readers (RINEX v3, RINEX v2, SBF, future formats) must implement to ensure compatibility with downstream pipeline: - VOD calculation (canvod-vod) - Storage (canvod-store / MyIcechunkStore) - Grid operations (canvod-grids)

Contract constants (REQUIRED_DIMS, REQUIRED_COORDS, etc.) are the single source of truth for the output Dataset structure. Use :func:validate_dataset to check any Dataset against them.

DatasetStructureValidator

Bases: BaseModel

Validates that an xarray.Dataset meets the GNSSDataReader contract.

Wraps a Dataset and checks it against the contract constants above. Use this in tests and reader implementations to catch structural errors early with clear messages.

Examples:

>>> validator = DatasetStructureValidator(dataset=ds)
>>> validator.validate_all()          # raises ValueError on any violation
>>> validator.validate_dimensions()   # check just one aspect

validate_all(required_vars=None)

Run all validations, collecting all errors.

Delegates to :func:validate_dataset so the logic lives in one place.

validate_dimensions()

Check that required dimensions (epoch, sid) exist.

validate_coordinates()

Check that required coordinates exist with correct dtypes.

validate_data_variables(required_vars=None)

Check that required data variables exist with correct dims.

validate_attributes()

Check that required global attributes are present.

SignalID

Bases: BaseModel

Validated signal identifier (SV + band + code).

sid = SignalID(sv="G01", band="L1", code="C") str(sid) 'G01|L1|C' sid.system 'G'

system property

GNSS system letter (e.g. 'G' for GPS).

sid property

Full signal ID string ('SV|band|code').

from_string(sid_str) classmethod

Parse a signal ID string ('SV|band|code') into a SignalID.

Parameters:

Name Type Description Default
sid_str str

Signal ID in 'SV|band|code' format (e.g. 'G01|L1|C').

 required

Returns:

Type Description
SignalID

Validated signal identifier.

Raises:

Type Description
ValueError

If the string does not have exactly three pipe-separated parts.

GNSSDataReader

Bases: BaseModel, ABC

Abstract base class for all GNSS data format readers.

All readers must: 1. Inherit from this class 2. Implement all abstract methods 3. Return xarray.Dataset that passes :func:validate_dataset 4. Provide file hash for deduplication

This ensures compatibility with: - canvod-vod: VOD calculation - canvod-store: MyIcechunkStore storage - canvod-grids: Grid projection operations

Subclasses may override model_config to set frozen, extra, etc. The base class provides arbitrary_types_allowed=True which is needed by readers that use pint.Quantity or similar third-party types.

Examples:

>>> class Rnxv3Obs(GNSSDataReader):
...     def to_ds(self, **kwargs) -> xr.Dataset:
...         # Implementation
...         return dataset
...
>>> reader = Rnxv3Obs(fpath="station.24o")
>>> ds = reader.to_ds()
>>> validate_dataset(ds)

source_format abstractmethod property

Return the format identifier for this reader (e.g. "rinex3", "sbf").

A run puts it on each file's dataset (attribute "Source Format") and the GNSS store records it as its source_format: the root attribute and identity.source_format of the store's metadata. A run refuses files of different formats in one store.

file_hash abstractmethod property

Return SHA256 hash of file for deduplication.

Used by MyIcechunkStore to avoid duplicate ingestion. Must be deterministic and reproducible.

Returns:

Type Description
str

Short hash (16 chars) or full hash of file content

start_time abstractmethod property

Return start time of observations.

Returns:

Type Description
datetime

First observation timestamp in the file.

end_time abstractmethod property

Return end time of observations.

Returns:

Type Description
datetime

Last observation timestamp in the file.

systems abstractmethod property

Return list of GNSS systems in file.

Returns:

Type Description
list of str

System identifiers: 'G', 'R', 'E', 'C', 'J', 'S', 'I'

num_epochs property

Return number of epochs in file.

Default implementation iterates epochs. Subclasses may override with a faster approach.

Returns:

Type Description
int

Total number of observation epochs.

num_satellites abstractmethod property

Return total number of unique satellites observed.

Returns:

Type Description
int

Count of unique satellite vehicles across all systems.

to_ds(keep_data_vars=None, **kwargs) abstractmethod

Convert data to xarray.Dataset.

Must return Dataset with structure: - Dims: (epoch, sid) - Coords: epoch, sid, sv, system, band, code, freq_* - Data vars: At minimum SNR - Attrs: Must include "File Hash"

Parameters:

Name Type Description Default
keep_data_vars list of str

Data variables to include. If None, includes all available.

 None
**kwargs object

Implementation-specific parameters

 {}

Returns:

Type Description
Dataset

Dataset that passes :func:validate_dataset.

iter_epochs() abstractmethod

Iterate over epochs in the file.

Yields:

Type Description
Epoch

Parsed epoch with satellites and observations.

to_ds_and_auxiliary(keep_data_vars=None, **kwargs)

Produce the obs dataset and any auxiliary datasets in a single call.

Default: calls to_ds(**kwargs) and returns an empty auxiliary dict. Readers that produce metadata (e.g. SBF) override this to collect both in a single file scan.

Returns:

Type Description
tuple[Dataset, dict[str, Dataset]]

(obs_ds, {"name": aux_ds, ...}). Auxiliary dict is empty for readers with no extra data (RINEX v2/v3).

__repr__()

Return the string representation.

validate_dataset(ds, required_vars=None)

Validate ds meets the GNSSDataReader output contract.

Collects all violations and raises a single ValueError listing every problem, rather than stopping at the first failure.

Parameters:

Name Type Description Default
ds Dataset

Dataset to validate.

 required
required_vars list of str

Data variables that must be present. Defaults to :data:DEFAULT_REQUIRED_VARS (["SNR"]).

 None

Raises:

Type Description
ValueError

If any contract violation is found.

validate_vod_dataset(ds)

Validate ds meets the VOD dataset contract.

The contract of VOD calculator output and of what the VOD store accepts: the dimensions and coordinates of :func:validate_dataset, the variables :data:VOD_REQUIRED_VARS (theta and phi in radians), every variable on (epoch, sid). Collects all violations and raises a single ValueError.

Parameters:

Name Type Description Default
ds Dataset

Dataset to validate.

 required

Raises:

Type Description
ValueError

If any contract violation is found.

require_aggregated_glonass(reader, aggregate_glonass_fdma)

Refuse aggregate_glonass_fdma=False for a reader that cannot honor it.

Parameters:

Name Type Description Default
reader str

Reader name for the message (e.g. "SBF").

 required
aggregate_glonass_fdma bool

The requested setting.

 required

Returns:

Type Description
bool

True.

Raises:

Type Description
ValueError

If aggregate_glonass_fdma is False.

Dataset builder

Guided builder for constructing valid GNSSDataReader output Datasets.

Handles coordinate arrays, dtype enforcement, frequency resolution, and contract validation automatically.

Examples:

>>> builder = DatasetBuilder(reader, time_system="GPS")
>>> for epoch in reader.iter_epochs():
...     ei = builder.add_epoch(epoch.timestamp)
...     for obs in epoch.observations:
...         sig = builder.add_signal(sv="G01", band="L1", code="C")
...         builder.set_value(ei, sig, "SNR", 42.0)
>>> ds = builder.build()   # validated Dataset

DatasetBuilder

Guided builder for constructing valid GNSSDataReader output Datasets.

Handles coordinate arrays, dtype enforcement, frequency resolution, and contract validation automatically.

Parameters:

Name Type Description Default
reader GNSSDataReader

The reader instance (used for _build_attrs() and file hash).

 required
time_system str

Time scale of the epoch timestamps as given to :meth:add_epoch, recorded in the epoch coordinate's time_system attribute. Must be "GPS": canVODpy stores every epoch in GPS time, so convert other time scales (for example UTC, plus the leap seconds from :func:canvod.readers.gnss_specs.leap_seconds.utc_to_gps) before :meth:add_epoch. Required, since the builder cannot infer it from the timestamps; anything else raises ValueError.

 required
aggregate_glonass_fdma bool

Whether to aggregate GLONASS FDMA channels (default True).

 True

Examples:

>>> builder = DatasetBuilder(reader, time_system="GPS")
>>> for epoch in reader.iter_epochs():
...     ei = builder.add_epoch(epoch.timestamp)
...     for obs in epoch.observations:
...         sig = builder.add_signal(sv="G01", band="L1", code="C")
...         builder.set_value(ei, sig, "SNR", 42.0)
>>> ds = builder.build()   # validated Dataset

add_epoch(timestamp)

Register an epoch timestamp. Returns epoch index.

add_signal(sv, band, code)

Register a signal (idempotent). Returns validated SignalID.

set_value(epoch_idx, signal, var, value)

Set a data value for a given epoch, signal, and variable.

Parameters:

Name Type Description Default
epoch_idx int

Index returned by :meth:add_epoch.

 required
signal SignalID or str

Signal identifier (SignalID or 'SV|band|code' string).

 required
var str

Variable name (e.g. 'SNR', 'Pseudorange', 'Phase').

 required
value float

The observation value.

 required

build(keep_data_vars=None, extra_attrs=None)

Build, validate, and return the Dataset.

  1. Sorts signals alphabetically
  2. Resolves frequencies from band names via SignalIDMapper
  3. Constructs coordinate arrays with correct dtypes (float32 for freq)
  4. Attaches CF-compliant metadata from COORDS_METADATA
  5. Calls validate_dataset() before returning

Parameters:

Name Type Description Default
keep_data_vars list of str

If provided, only include these data variables. If None, includes all variables that had values set.

 None
extra_attrs dict

Additional global attributes to merge into the Dataset.

 None

Returns:

Type Description
Dataset

Validated Dataset with dimensions (epoch, sid).

sid_coords(signals, *, mapper)

Coordinates of the sid dimension for the given signals.

sid and its per-signal coordinates (sv, system, band, code, freq_center, freq_min, freq_max) as required by :data:~canvod.readers.base.REQUIRED_COORDS, with their metadata. Frequencies come from the band; a band without a known frequency gets NaN.

Parameters:

Name Type Description Default
signals list of SignalID

Signals in the order of the sid dimension.

 required
mapper SignalIDMapper

Band lookup for the frequencies.

 required

Returns:

Type Description
dict

Coordinates to pass to :class:xarray.Dataset.

GNSS specifications

GNSS specifications and core characteristics.

This module contains fundamental GNSS definitions including: - Constants: Unit registry, physical constants, RINEX parameters - Exceptions: GNSS-specific error types - Metadata: CF-compliant metadata for coordinates and observables - Models: Pydantic validation models for RINEX data structures - Signals: Signal ID mapping and band properties - Obs codes: RINEX observation code <-> signal ID mapping - Utils: File hashing, version extraction, data type checks

These components are used across all GNSS reader implementations.

Directory matching

Directory matching for RINEX data files.

Identifies and matches RINEX data directories across dates and receivers.

MatchedDirs dataclass

Matched directory paths for canopy and reference receivers.

Immutable container representing a pair of directories containing RINEX data for the same date.

Parameters:

Name Type Description Default
canopy_data_dir Path

Path to canopy receiver RINEX directory.

 required
reference_data_dir Path

Path to reference (open-sky) receiver RINEX directory.

 required
yyyydoy YYYYDOY

Date object for this matched pair.

 required

Examples:

>>> from pathlib import Path
>>> from canvod.utils.tools import YYYYDOY
>>>
>>> md = MatchedDirs(
...     canopy_data_dir=Path("/data/02_canopy/25001"),
...     reference_data_dir=Path("/data/01_reference/25001"),
...     yyyydoy=YYYYDOY.from_str("2025001")
... )
>>> md.yyyydoy.to_str()
'2025001'

PairMatchedDirs dataclass

Matched directories for a receiver pair on a specific date.

Supports multi-receiver configurations where multiple canopy/reference pairs may exist at the same site.

Parameters:

Name Type Description Default
yyyydoy YYYYDOY

Date for this matched pair.

 required
pair_name str

Identifier for this receiver pair (e.g., "pair_01").

 required
canopy_receiver str

Name of canopy receiver (e.g., "canopy_01").

 required
reference_receiver str

Name of reference receiver (e.g., "reference_01").

 required
canopy_data_dir Path

Path to canopy receiver RINEX directory.

 required
reference_data_dir Path

Path to reference receiver RINEX directory.

 required

Examples:

>>> pmd = PairMatchedDirs(
...     yyyydoy=YYYYDOY.from_str("2025001"),
...     pair_name="pair_01",
...     canopy_receiver="canopy_01",
...     reference_receiver="reference_01",
...     canopy_data_dir=Path("/data/canopy_01/25001"),
...     reference_data_dir=Path("/data/reference_01/25001")
... )
>>> pmd.pair_name
'pair_01'