Parsing

Parsing of the NYSE Arca Feed data.

NYSE supplies 4 different channel feeds (http://www.nyxdata.com/page/1084):

The NYSE Arca Integrated Feed now lists four channels each A-Z. What is significant about four channels?

The four channels will send data via matching engine ID’s, which are load balanced across all letter ranges A-Z. Four-channel delivery of this feed will benefit customers by reducing the overall delivery latency with this product. These channel configurations will help our publishers achieve the very lowest latency possible.

Parsers

Parsers for XDP streams.

class oxberrypis.parsing.parsers.FileXDPChannelUnpacker(channel_path, pkt_hdr_cls=None, msg_hdr_cls=None)

XDP channel parser (unpacker) with file stream.

classmethod get_channel_path(directory, channel, channel_file_name_fmt=None)

Get path to the channel found in the directory.

Channel file name format string defaults to string set in CHANNEL_FILE_NAME_FMT but it may be changed by calling the function with appropriate channel_file_name_fmt value.

classmethod get_channel_unpacker(directory, channel_id)

Factory for channel unpacker given directory and channel_id.

open_stream(channel_path)

Return opened stream for the given file found at channel_path.

class oxberrypis.parsing.parsers.XDPChannelUnpacker(stream, pkt_hdr_cls=None, msg_hdr_cls=None)

XDP channel parser (unpacker).

The parser reads the supplied stream, unpacks packet headers, reads all message headers and unpacks only known messages.

Note

The parser assumes that the stream position is not manipulated during the lifetime of this object. Packets and messages are assumed to appear in the stream consecutively.

The parsers uses headers.PacketHeader and headers.MsgHeader for parsing, but you can supply your own header classes by setting pkt_hdr_cls and msg_header_cls to appropriate values.

Parameters:

• stream – Readable stream which feeds the parser.
• pkt_hdr_cls – Packet header class. Defaults to headers.PacketHeader.
• msg_hdr_cls – Message header class. Defaults to headers.MessageHeader.

advance(offset)

Skip offset number of bytes in the stream.

parse()

Generator for all the known messages from the stream.

As long as there is a data remaining in the stream (assumed to be sufficiently long), it unpack the packet header. For each packet header parses the messages contained within.

parse_cls_from_stream(cls, size)

Read the stream and parse to create an instance of given class.

Assumes cls._make is a method accepting a tuple of unpacked data.

Returns None if the stream is empty and raises an error if unpacking failed.

parse_msg(msg_header)

Parse a message from the stream.

Message header is used to determined the size of the payload and the type of the message contained within it.

parse_packet(pkt_header)

Generator for messages found within packet.

Packet header is used to read off the number of messages contained within the packet. Then unpacks message headers one by one and if the message type is recognised, the message is parsed. Otherwise position of the stream is advanced.

Fields

Fields used in the XDP data stream packets.

class oxberrypis.parsing.fields.FieldsGenerator(fields, size_mapping)

Fields generator for NYSE Arca Integrated Feed stream.

Headers

XDP stream headers.

class oxberrypis.parsing.headers.MsgHeader

XDP Message header.

get_msg_cls(known_msgs=None)

Get payload message class.

get_msg_size()

Return the size of the payload (message) only.

Customers should not hard code msg sizes in feed handlers; instead the feed handler should use the Msg Size field to determine where the next message in the packet begins. This allows the XDP format to accommodate the different market needs for data content and allow the format to be more agile.

header_size = 4

Mapping between known message types and known message classes.

is_known(known_msgs=None)

Checked whether payload message is known.

classmethod register_known_msg(msg_cls)

Decorator for registering known messages.

class oxberrypis.parsing.headers.PacketHeader

XDP feed packet header.

get_datetime()

Get datetime when the packet was sent to the multicast channel for publication.

Messages

NYSE integrated feed messages.

class oxberrypis.parsing.messages.OBAddOrderMsg

Order Book Add Order Message.

class oxberrypis.parsing.messages.OBDeleteMsg

Order Book Delete Message.

class oxberrypis.parsing.messages.OBExecutionMsg

Order Book Execution Message.

class oxberrypis.parsing.messages.OBModifyMsg

Order Book Modify Message.

class oxberrypis.parsing.messages.TradeMsg

Trade Message.

Utilities

Various utilities used in the parsing module.

oxberrypis.parsing.utils.get_pkt_format_string(pkt_spec)

Create struct format string from spec tuple.

oxberrypis.parsing.utils.get_pkt_namedtuple(name, pkt_spec)

Create namedtuple from spec tuple.