Yes, changes to enable processing of many messages arriving on a stream are in the works.
https://issues.apache.org/jira/browse/DAFFODIL-1565 and https://issues.apache.org/jira/browse/DAFFODIL-1065 <https://issues.apache.org/jira/browse/DAFFODIL-1> are the primary tickets for this change. Steve Lawrence is the developer assigned on these.
Once the core API-level and internal changes in the I/O layer are complete, then there are other changes like making our command line interface tool use this API (https://issues.apache.org/jira/browse/DAFFODIL-1799) that would also be great to do, but nobody is assigned to currently. (Hint Hint)
This feature is an important one on the Daffodil roadmap for the 2.2.0 release. Lots of people want this message-by-message parsing capability for a variety of data formats.
The roadmap is here:
Your input/suggestions on this and the priority of various needs are of course welcome.
From: Perpignand, Weber <wperpignand@xxxxxxxxxxxxxxxxxxx>
Sent: Tuesday, April 17, 2018 3:03:55 PM
Cc: Clarke, Frederick
I am using incubator-daffodil-master version 2.1 and am interested to see it works with live streaming fixed format data like Link-16.
It seems that daffodil is a file-based system that can read and parse files like csv, pcap, and with some streaming using stdin.
1. Is there a daffodil version today that supports the fixed format data stream, i.e. Link-16?
2. Is re-directing the streaming data to stdin the expected path?
3. Or is there a plan to have daffodil read from a socket or pipe to support this?
Thanks in advance for your help.
Senior Software Engineer
+1 (203) 894-9342 x189
[Email Signature Logo]
See us at the following 2018 events:
RSA Conference, San Francisco, CA (April 16th – April 20th)<https://www.owlcyberdefense.com/events>
34th Annual Space Symposium, Colorado Springs, CO – (April 16th – April 19th)<https://www.owlcyberdefense.com/events>
Spring Intelligence Symposium, Springfield, VA (April 18th – April 19th)<https://www.owlcyberdefense.com/events>
PI World - OSIsoft User Conference, San Francisco, CA (April 23rd – April 27th)<https://www.owlcyberdefense.com/events>