extract.items

SupportsPipe Objects

class SupportsPipe(Protocol)

[view_source]

A protocol with the core Pipe properties and operations

name

Pipe name which is inherited by a resource

parent

A parent of the current pipe

gen

@property
def gen() -> TPipeStep

[view_source]

A data generating step

__getitem__

def __getitem__(i: int) -> TPipeStep

[view_source]

Get pipe step at index

__len__

def __len__() -> int

[view_source]

Length of a pipe

has_parent

@property
def has_parent() -> bool

[view_source]

Checks if pipe is connected to parent pipe from which it takes data items. Connected pipes are created from transformer resources

close

def close() -> None

[view_source]

Closes pipe generator

ItemTransform Objects

class ItemTransform(ABC, Generic[TAny])

[view_source]

placement_affinity

Tell how strongly an item sticks to start (-1) or end (+1) of pipe.

__call__

@abstractmethod
def __call__(item: TDataItems, meta: Any = None) -> Optional[TDataItems]

[view_source]

Transforms item (a list of TDataItem or a single TDataItem) and returns or yields TDataItems. Returns None to consume item (filter out)

ValidateItem Objects

class ValidateItem(ItemTransform[TDataItem])

[view_source]

Base class for validators of data items.

Subclass should implement the __call__ method to either return the data item(s) or raise extract.exceptions.ValidationError. See PydanticValidator for possible implementation.

placement_affinity

stick to end but less than incremental