luxonis_ml.data.parsers.UltralyticsNDJSONParser

class documentation

class UltralyticsNDJSONParser(BaseParser):

Constructor: UltralyticsNDJSONParser(dataset, dataset_type, task_name, full_warnings)

Parse Ultralytics NDJSON datasets into LDF.

NDJSON records may carry their own split names. When no split is present on an image record, the image is assigned to "train". "valid" and "validation" are normalized to "val".

Class Method	`validate`	Validate whether the dataset directory has the expected format.
Static Method	`validate_split`	Validate whether a split directory has the expected format.
Method	`from_dir`	Parse an Ultralytics NDJSON file into dataset records.
Method	`from_split`	Parse a single Ultralytics NDJSON file.
Method	`parse_dir`	Parse a full NDJSON dataset and preserve record-level splits.
Method	`parse_split`	Parse an NDJSON file that represents a single parser input.
Class Method	`_download_image`	Undocumented
Class Method	`_load_header`	Undocumented
Class Method	`_resolve_image_path`	Undocumented
Static Method	`_fit_boundingbox`	Undocumented
Static Method	`_get_class_names`	Undocumented
Static Method	`_normalize_split_name`	Undocumented
Static Method	`_resolve_ndjson_path`	Undocumented
Method	`_build_record_stream`	Undocumented
Class Variable	`_remote_file_downloader`	Undocumented

Inherited from BaseParser:

Class Method	`discover_dir_splits`	Return present and valid split directories keyed by their canonical split names.
Method	`__init__`	Create a parser for a target dataset.
Method	`get_parser_issue_messages`	Return parser issue messages collected during the last parse.
Method	`reset_parser_issue_messages`	Clear collected parser issue messages.
Static Method	`_apply_counts_to_pool`	Distribute images across splits based on requested counts.
Static Method	`_apply_counts_to_splits`	Apply count-based split requests to existing splits.
Static Method	`_canonicalize_split_name`	All current parsers use `train` and `test` split names whereas validation splits can vary in name between `val` `valid` and `validation`.
Static Method	`_compare_stem_files`	Compare sets of files by stem.
Static Method	`_get_added_images`	Return unique images yielded by a dataset generator.
Static Method	`_list_images`	List OpenCV-supported images in a directory.
Static Method	`_sample_from_splits`	Sample from each original split independently.
Method	`_get_parser_issue_messages`	Return parser issue messages collected during the last parse.
Method	`_log_skipped_annotation_summary`	Undocumented
Method	`_parse_available_splits`	Undocumented
Method	`_parse_split`	Parse data in one split subdirectory.
Method	`_remove_unsplit_records`	Remove records that are not assigned to any split.
Method	`_reset_parser_issue_messages`	Clear collected parser issue messages and warning counters.
Method	`_warn_skipped_annotation`	Undocumented
Method	`_wrap_generator`	Add configured task names to generated records.
Constant	`_CANONICAL_SPLIT_NAMES`	Undocumented
Constant	`_SKIPPED_WARNING_LIMIT`	Undocumented
Constant	`_SPLIT_NAMES`	Undocumented
Instance Variable	`_dataset`	Undocumented
Instance Variable	`_dataset_type`	Undocumented
Instance Variable	`_full_warnings`	Undocumented
Instance Variable	`_logged_skipped_annotation_warnings`	Undocumented
Instance Variable	`_parser_issue_messages`	Undocumented
Instance Variable	`_seen_parser_issue_messages`	Undocumented
Instance Variable	`_skipped_annotation_counts_by_reason`	Undocumented
Instance Variable	`_suppressed_skipped_annotation_warnings`	Undocumented
Instance Variable	`_task_name`	Undocumented

@classmethod

def validate(cls, dataset_dir: Path) -> bool: ¶

overrides luxonis_ml.data.parsers.BaseParser.validate

Validate whether the dataset directory has the expected format.

Parameters
dataset_dir:`Path`	Source dataset directory.

Returns
`bool`	Whether the dataset is in the expected format.

@staticmethod

def validate_split(split_path: Path) -> dict[str, Any] | None: ¶

overrides luxonis_ml.data.parsers.BaseParser.validate_split

Validate whether a split directory has the expected format.

Parameters
split_path:`Path`	Path to a split directory.

Returns
`dict[str, Any] \| None`	Keyword arguments for `from_split`, or `None` if the split is not in the expected format.

def from_dir(self, dataset_dir: Path, reuse_cached: bool = True, **kwargs) -> tuple[list[Path], list[Path], list[Path]]: ¶

overrides luxonis_ml.data.parsers.BaseParser.from_dir

Parse an Ultralytics NDJSON file into dataset records.

Parameters
dataset_dir:`Path`	Directory containing exactly one `.ndjson` file, or a direct path to an `.ndjson` file.
reuse_cached:`bool`	Whether to reuse cached remote images if they already exist.
**kwargs	Parser-specific arguments.

Returns
`tuple[list[Path], list[Path], list[Path]]`	Added images for the train, validation, and test splits.

Raises
`ValueError`	If no NDJSON dataset file can be resolved, the resolved file has an invalid header, a remote image download directory already exists, or pose annotations cannot infer keypoint dimensionality.

def from_split(self, ndjson_path: Path, reuse_cached: bool = True) -> ParserOutput: ¶

overrides luxonis_ml.data.parsers.BaseParser.from_split

Parse a single Ultralytics NDJSON file.

Parameters
ndjson_path:`Path`	Path to an Ultralytics NDJSON file.
reuse_cached:`bool`	Whether to reuse cached remote images if they already exist.

Returns
`ParserOutput`	Parser output containing annotation records, empty skeleton metadata, and added images.

Raises
`ValueError`	If the file has an invalid header, a remote image download directory already exists, or pose annotations cannot infer keypoint dimensionality.

def parse_dir(self, dataset_dir: Path, reuse_cached: bool = True, **kwargs) -> BaseDataset: ¶

overrides luxonis_ml.data.parsers.BaseParser.parse_dir

Parse a full NDJSON dataset and preserve record-level splits.

Parameters
dataset_dir:`Path`	Directory containing exactly one `.ndjson` file, or a direct path to an `.ndjson` file.
reuse_cached:`bool`	Whether to reuse cached remote images if they already exist.
**kwargs	Parser-specific arguments. `split_ratios` may be supplied to resample split assignments.

Returns
`BaseDataset`	Dataset with parsed images and annotations.

Raises
`ValueError`	If the NDJSON file cannot be resolved or parsed as a valid Ultralytics dataset.

@override

def parse_split(self, split: str | None = None, random_split: bool = True, split_ratios: dict[str, float | int] | None = None, reuse_cached: bool = True, **kwargs) -> BaseDataset: ¶

overrides luxonis_ml.data.parsers.BaseParser.parse_split

Parse an NDJSON file that represents a single parser input.

Parameters
split:`str \| None`	Optional split name to assign to all parsed images.
random_split:`bool`	Whether percentage `split_ratios` should resample all images.
split_ratios:`dict[str, float \| int] \| None`	Optional ratios or counts. Float values are treated as ratios; integer values are treated as counts.
reuse_cached:`bool`	Whether to reuse cached remote images if they already exist.
**kwargs	Parser-specific arguments. Must include `ndjson_path`.

Returns
`BaseDataset`	Dataset with parsed images and annotations.

Raises
`ValueError`	If `ndjson_path` is missing, the file has an invalid header, a remote image download directory already exists, or pose annotations cannot infer keypoint dimensionality.

@classmethod

def _download_image(cls, record: dict[str, Any], *, remote_image_dir: Path) -> Path: ¶

Undocumented

@classmethod

def _load_header(cls, path: Path) -> dict[str, Any] | None: ¶

Undocumented

@classmethod

def _resolve_image_path(cls, ndjson_path: Path, record: dict[str, Any], *, remote_image_dir: Path) -> Path: ¶

Undocumented

@staticmethod

def _fit_boundingbox(points: np.ndarray) -> dict[str, float]: ¶

Undocumented

@staticmethod

def _get_class_names(class_names: list[str] | dict[str, str]) -> dict[int, str]: ¶

Undocumented

@staticmethod

def _normalize_split_name(split_name: str | None) -> str: ¶

Undocumented

@staticmethod

def _resolve_ndjson_path(path: Path) -> Path | None: ¶

Undocumented

def _build_record_stream(self, ndjson_path: Path, reuse_cached: bool = True) -> tuple[DatasetIterator, dict[str, list[Path]], list[Path]]: ¶

Undocumented

_remote_file_downloader = ¶

Undocumented