Home Page › Forums › BizTalk 2004 – BizTalk 2010 › Biztalk 2006 – Flat File Splitting
- This topic has 5 replies, 1 voice, and was last updated 9 years, 10 months ago by
community-content.
-
AuthorPosts
-
-
March 31, 2008 at 9:20 AM #19203
I am relatively new to BizTalk and am having a problem solving what seems to be a simple problem. I have a flat file that is tab delimited. I am basically trying to split the file into 2 files.
There are no headers or footers on the file. It simply has detail records. There is a field that I named PlanType. If the PlanType is equal to ‘Retirement’, I want the record to be written to file 1. If the PlanType is equal to ‘Loan’, I want the record written to File 2.
I have an xsd for the flat file and tried promoting a field. I could not however as it was a ‘repeating’ field.
Any help would be appreciated.
Thanks, Gregg
-
March 31, 2008 at 9:59 AM #19204
Hi Gregg,
I have a similar situation with invoices, where I needed to break the file up by invoice number. The best way to do this from what I read and did was to write a custom flat file disassembler that “debatches” the flat file. Each group of records (for me all the Invoice Detail records with same invoice number) are batched together into a single message.
The custom flat file disassembler, written in C#, reads in each line (delimited by CRLF) from the file. I then examine the Invoice # in the record and compare it with the previous Invoice number (if there was one). If it’s the same invoice number I read the next line from the file, otherwise I take what has been read before the current line and send it to BizTalk as a message.
You’ll have to do some research on writing custom flat file disassemblers and pipeline components. “Pro BizTalk 2006” from APress is a great book and there are lots of articles on this Forum and MSDN about writing custom components. Here is a great MSDN article on the subject:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/ebiz_prog_pipe_eiji.asp
That should give you enough info to start your research. Don’t get discouraged, once you understand the process the code makes sense!
-
March 31, 2008 at 10:14 AM #19205
Hi Gregg
If you’re new to the world of BizTalk flat-files it can be quite daunting, so don’t worry if it is all a bit hard-going to start with!
You should probably start out with http://www.topxml.com/rbnews/BizTalk-2006/re-61406_Debatching-a-flat-file-in-BizTalk-2006.aspx to get a better understanding of flat-file splitting.
With regards to your particular problem, I’m afraid it isn’t quite that easy to acheive…..
AFAIK, the flat-file disassembler (the bit that translates the file into Xml and splits) won’t be able to split records between files based on a particular value – splitting works on nodes, splitting content and moving on to the new bit of data, rather than maintaining temporary messages that are built as the message is being parsed as you indicate in your post).
To achieve what you’re after, you will need to write a receive decode component that will sort your PlanType flat-file records so you have for example Loan records grouped together and then Retirement records; if you could put an identifier at the beginning of the record as well that will help the disassembler (as it won’t have to look deeply into your content to determine the record). The XSD that you define can then be configured to split on these grouped record ‘types’ generating two distinct Xml messages.
I appreciate that the details above may be a bit in-depth – if you have any questions, ask away!
Nick.
-
April 1, 2008 at 6:20 AM #19216
AFAIK, the flat-file disassembler (the bit that translates the file
into Xml and splits) won’t be able to split records between files based
on a particular value – splitting works on nodes, splitting content and
moving on to the new bit of data, rather than maintaining temporary
messages that are built as the message is being parsed as you indicate
in your post).The purpose of a Flat File Receive Pipeline is to take a Flat File and convert it into a BizTalk message according to a schema. The dissassembler has a number of purposes; debatching is one of those purposes. I thought about doing the debatching inside of an Orchestration or inside of a map and it proved to be far more difficult than writing a custom Flat File dissassembler. I was relatevely new to BizTalk when I wrote my dissassembler and it hasn’t given me any problems.
Inside the Flat File Dissassembler (FFD) there are essentially 2 function: Dissassemble and GetNext. Dissassemble is called first by BizTalk after yout Flat File is received by the File Adapter. It is passed a Base Message which is essentially your Flat File stuffed into an XML envelope. I get a copy of the Base Message as a Stream, validate my Flat File, then return.
GetNext() is now called repeatedly by BizTalk until you return NULL, which indicates that there are no more messages in the Flat File.
-
April 4, 2008 at 7:18 AM #19250
This is another good atricle from CodeProject about extending the FFDasmComp class to do your own debatching:
http://www.codeproject.com/KB/biztalk/DebatchingFlatfile.aspx
-
-
-
April 1, 2008 at 12:41 AM #19212
Gregg,
the flat-file component isn’t really intended to split files, it’s intended to convert a flat-file into XML. So, I would be tempted to just convert the incoming into XML and then use a different technique.
There is a technique which can split an incoming message into n messages if the incoming message is structured as envelope and contents – not your case I think.
Also, it would be possible to do this with a map. It’s possible to have a map with multiple output messages – so with a bit of messing about you could do this. But I don’t think I would recommend it.
I think the simplest will be to write a pipeline component to split your message, after it has been converted into XML. Could be done before the conversion I guess, but if it was me I’d prefer to manipulate XML.
John D.
-
-
AuthorPosts
- The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.