ayx_python_sdk.providers.amp_provider.data_transport package¶
Submodules¶
ayx_python_sdk.providers.amp_provider.data_transport.amp_transport module¶
Arrows stream formatted transport interface.
- class ayx_python_sdk.providers.amp_provider.data_transport.amp_transport.AmpStreamTransport[source]¶
Bases:
ayx_python_sdk.providers.amp_provider.data_transport.transport_base.TransportBase
Transport interface to deliver data in arrows stream format.
- get_rec_stream(record_batch: pyarrow.lib.RecordBatch) pyarrow.lib.Buffer [source]¶
Get an arrows formatted stream buffer.
- iter_stream(stream: pyarrow.lib.Buffer) Generator[Tuple[bytes, bool], None, None] [source]¶
Break up a given stream by set chunk size.
- logger = <RootLogger root (WARNING)>¶
ayx_python_sdk.providers.amp_provider.data_transport.transport_base module¶
Transport implementation.
Transports provide the functionality for data management and IO. An implementation should provide an independant way to receive, parse, serialize, and send data to another service.
Module contents¶
Implementations of TransportBase.
- class ayx_python_sdk.providers.amp_provider.data_transport.AmpStreamTransport[source]¶
Bases:
ayx_python_sdk.providers.amp_provider.data_transport.transport_base.TransportBase
Transport interface to deliver data in arrows stream format.
- get_rec_stream(record_batch: pyarrow.lib.RecordBatch) pyarrow.lib.Buffer [source]¶
Get an arrows formatted stream buffer.
- iter_stream(stream: pyarrow.lib.Buffer) Generator[Tuple[bytes, bool], None, None] [source]¶
Break up a given stream by set chunk size.
- logger = <RootLogger root (WARNING)>¶
- receive_record(record_in_msg: RecordTransferIn) pa.Table [source]¶
Receive RecordTransferIn messages.
Returns None if the data received is only a chunk of a full record. Returns pa.Table if end_of_chunk is True.