luxonis_ml.data.parsers.FiftyOneClassificationParser

class documentation

class FiftyOneClassificationParser(BaseParser):

Constructor: FiftyOneClassificationParser(dataset, dataset_type, task_name, full_warnings)

View In Hierarchy

Parse FiftyOne image classification data into LDF.

Supports two directory structures:

Split structure with train/test/validation subdirectories:

dataset_dir/
├── train/
│   ├── data/
│   │   ├── img1.jpg
│   │   └── ...
│   └── labels.json
├── validation/
│   ├── data/
│   └── labels.json
└── test/
    ├── data/
    └── labels.json

Flat structure (single directory, random splits applied at parse time):

dataset_dir/
├── data/
│   ├── img1.jpg
│   └── ...
└── labels.json

The labels.json format is:

{
    "classes": ["class1", "class2", ...],
    "labels": {
        "image_stem": class_index,
        ...
    }
}

This parser supports the FiftyOne image classification export layout.

Static Method validate_split Validate whether a split directory has the expected format.
Method from_dir Parse all data in a source dataset directory.
Method from_split Parse data from one split subdirectory.
Constant _SPLIT_NAMES Undocumented

Inherited from BaseParser:

Class Method discover_dir_splits Return present and valid split directories keyed by their canonical split names.
Class Method validate Validate whether the dataset directory has the expected format.
Method __init__ Create a parser for a target dataset.
Method get_parser_issue_messages Return parser issue messages collected during the last parse.
Method parse_dir Parse an entire dataset directory into the target dataset.
Method parse_split Parse one split subdirectory into the target dataset.
Method reset_parser_issue_messages Clear collected parser issue messages.
Static Method _apply_counts_to_pool Distribute images across splits based on requested counts.
Static Method _apply_counts_to_splits Apply count-based split requests to existing splits.
Static Method _canonicalize_split_name All current parsers use train and test split names whereas validation splits can vary in name between val valid and validation.
Static Method _compare_stem_files Compare sets of files by stem.
Static Method _get_added_images Return unique images yielded by a dataset generator.
Static Method _list_images List OpenCV-supported images in a directory.
Static Method _sample_from_splits Sample from each original split independently.
Method _get_parser_issue_messages Return parser issue messages collected during the last parse.
Method _log_skipped_annotation_summary Undocumented
Method _parse_available_splits Undocumented
Method _parse_split Parse data in one split subdirectory.
Method _remove_unsplit_records Remove records that are not assigned to any split.
Method _reset_parser_issue_messages Clear collected parser issue messages and warning counters.
Method _warn_skipped_annotation Undocumented
Method _wrap_generator Add configured task names to generated records.
Constant _CANONICAL_SPLIT_NAMES Undocumented
Constant _SKIPPED_WARNING_LIMIT Undocumented
Instance Variable _dataset Undocumented
Instance Variable _dataset_type Undocumented
Instance Variable _full_warnings Undocumented
Instance Variable _logged_skipped_annotation_warnings Undocumented
Instance Variable _parser_issue_messages Undocumented
Instance Variable _seen_parser_issue_messages Undocumented
Instance Variable _skipped_annotation_counts_by_reason Undocumented
Instance Variable _suppressed_skipped_annotation_warnings Undocumented
Instance Variable _task_name Undocumented
@staticmethod
def validate_split(split_path: Path) -> dict[str, Any] | None:
overrides luxonis_ml.data.parsers.BaseParser.validate_split

Validate whether a split directory has the expected format.

Parameters
split_path:PathPath to a split directory.
Returns
dict[str, Any] | NoneKeyword arguments for from_split, or None if the split is not in the expected format.
def from_dir(self, dataset_dir: Path, **kwargs) -> tuple[list[Path], list[Path], list[Path]]:
overrides luxonis_ml.data.parsers.BaseParser.from_dir

Parse all data in a source dataset directory.

Parameters
dataset_dir:PathSource dataset directory.
**kwargsAdditional parser-specific arguments.
Returns
tuple[list[Path], list[Path], list[Path]]Added images for the train, validation, and test splits.
def from_split(self, split_path: Path) -> ParserOutput:
overrides luxonis_ml.data.parsers.BaseParser.from_split

Parse data from one split subdirectory.

Example

split_kwargs = parser.validate_split(split_path)
if split_kwargs is not None:
    parser.from_split(**split_kwargs)
Parameters
split_path:PathUndocumented
**kwargsParser-specific arguments, usually produced by validate_split.
Returns
ParserOutputLDF generator, skeleton metadata, and added images.
_SPLIT_NAMES: tuple[str, ...] =
overrides luxonis_ml.data.parsers.BaseParser._SPLIT_NAMES

Undocumented

Value
('train', 'validation', 'test')