Creating Zip-Files in send pipeline

Home Page Forums BizTalk 2004 – BizTalk 2010 Creating Zip-Files in send pipeline

Viewing 1 reply thread
  • Author
    Posts
    • #14978

      Hi,

      we want to compress date with a customer pipeline conponent.
      Because i need the flatfile-assemply, so i create a codeing component.

      The compressing job workes well.
      But when i give the zip-file back to the pipeline, the pipline will manipulate the file, so that the output will not be a zip-file.

      My Programm runs in the ‘IComponent’ interface.
      And I give back the zip-file through ‘pInMsg.BodyPart.Data’.

      I tried MemoryStream as well as FileStream, but in both case it will not work correctly. The length of the output file will also be smaler than the lenght of the committed file.

      Wat can I do that i can give the zip-file back to the pipeline and get a correct file from the end of the pipeline?

      Thanks in advance

    • #14979

      My Component is the last step of the pipeline.

      here is code part:

      C#

      #region IComponent Member

      IBaseMessage Microsoft.BizTalk.Component.Interop.IComponent.Execute
      (IPipelineContext pContext, IBaseMessage pInMsg)
      {

      Stream strmInputFile, DataStream;
      string MessageID;
      string BizTalkTempFolder = \”c:\\\\BizTalk_Temp\\\\\”;
      IBaseMessagePart MsgPart, RecendFileName;
      byte[] ZipBuffer = new byte[32768];
      Int32 bytesRead;
      MemoryFile memFile;

      MsgPart = pInMsg.BodyPart;
      strmInputFile = MsgPart.GetOriginalDataStream();
      MessageID = MsgPart.PartID.ToString();

      FileStream OutPutFile = new FileStream(BizTalkTempFolder + MessageID + \”.exe\”, FileMode.Open , FileAccess.Read);
      BinaryReader OutStreamBinary = new BinaryReader(OutPutFile);

      MemoryStream OutputStream = new MemoryStream();

      byte[] OutBuffer = new byte[ 1024 ];
      int OutBufferRead;

      while( ( OutBufferRead = OutPutFile.Read( OutBuffer, 0, OutBuffer.Length ) ) > 0 )
      {
      OutputStream.Write( OutBuffer, 0, OutBufferRead );
      }

      OutputStream.Position = 0;
      MsgPart.Data = OutputStream;
      pContext.ResourceTracker.AddResource(OutputStream);

      // OutPutFile.Close();
      // delete OutPutFile.

      return pInMsg;

      }

      #endregion

      }

      The InputFile is a XML or Flate File and the Output File is a Zip or SFX File.

      So i Hope you can find the error in that code.

      • #14980

        The ‘OutPutFile’ i give back to the pipeline is the Zip file i create in my pipeline component.
        i create it on the filesystem because the file dosn’t come correctly out of the pipeline.
        so i create it as a file in the filesystem to test if the file i create is correct.
        at this, i see that my problem shut bee in the return part of that programm.
        when the component will be work, i will be create the file as a memfile.

        so i hat some problems to publish the full code sequence at the last time, i try it again.

        [code:1:f514ef8a27]
        IBaseMessage Microsoft.BizTalk.Component.Interop.IComponent.Execute
        (IPipelineContext pContext, IBaseMessage pInMsg)
        {

        Stream strmInputFile, DataStream;
        string MessageID;
        string BizTalkTempFolder = \"c:\\\\BizTalk_Temp\\\\\";
        IBaseMessagePart MsgPart, RecendFileName;
        byte[] ZipBuffer = new byte[32768];
        Int32 bytesRead;
        MemoryFile memFile;

        MsgPart = pInMsg.BodyPart;
        strmInputFile = MsgPart.GetOriginalDataStream();
        MessageID = MsgPart.PartID.ToString();

        memFile = new MemoryFile(\"RAM_File\", MessageID);
        // memFile = new MemoryFile(\"RAM_File\", \"TEST_ZIP.zip\");

        ZipArchive ZipFile = new ZipArchive(new DiskFile(BizTalkTempFolder + MessageID));
        // ZipArchive ZipFile = new ZipArchive(memFile);
        ZippedFile file = (ZippedFile)ZipFile.GetFile(ZippedFileNameString);

        if (!file.Exists)
        file.Create();

        // Wirte Information into ZipFile

        DataStream = file.OpenWrite(true);

        bytesRead = 0;

        // Loop to write in 32k parts.
        while ((bytesRead = strmInputFile.Read(ZipBuffer, 0, ZipBuffer.Length)) > 0)
        {

        DataStream.Write(ZipBuffer, 0, bytesRead);

        }

        DataStream.Close();

        if (SelfExtraction)
        {

        //Create Selfextraction File

        // Change its extension to .EXE.
        ZipFile.ZipFile.Name = MessageID + \".exe\";

        // Create and configure a XceedSfxPrefix object.

        XceedSfxPrefix sfx = new XceedSfxPrefix(new DiskFile
        (@\"C:\\Programme\\Xceed Components\\Bin\\Sfx\\xcdsfx32.bin\"));

        // Assign the prefix to the zip file.
        ZipFile.SfxPrefix = sfx;

        }

        FileStream OutPutFile = new FileStream(BizTalkTempFolder + MessageID + \".exe\", FileMode.Open , FileAccess.Read);
        BinaryReader OutStreamBinary = new BinaryReader(OutPutFile);

        MemoryStream OutputStream = new MemoryStream();

        byte[] OutBuffer = new byte[ 1024 ];
        int OutBufferRead;

        while( ( OutBufferRead = OutPutFile.Read( OutBuffer, 0, OutBuffer.Length ) ) > 0 )
        {
        OutputStream.Write( OutBuffer, 0, OutBufferRead );
        }

        OutputStream.Position = 0;
        MsgPart.Data = OutputStream;
        pContext.ResourceTracker.AddResource(OutputStream);

        // OutPutFile.Close();
        // delete OutPutFile.

        return pInMsg;

        }

        #endregion

        }
        [/code:1:f514ef8a27]

        • #14981

          I tried you suggestion.

          I delete the pContext.ResourceTracker.AddResource(OutputStream); entry but the result was the same.

          The file the pipeline leave is not equal than the file I give back to the pipeline.

          I think there must be a hidde coder/assembler in the end of the pipeline.

          I Think so because when i give my input-file, a UTF16 coded XML file, back to the pipeline, the output of the pipeline is a UTF8 coded XML file.
          And there is no other coder/assembler component in the pipeline as my zip component.

          So how can I tell the pipeline that the stream i give back to the pipeline is a binary file and not a text or XML file?

          • #14982

            I think so, because this is the begine of my input file in hex:

            FFFE 3C00 3F00 7800 6D00 6C00 2000 7600 6500 7200 7300 6900 6F00 6E00 3D00 2200 3100 2E00 30000 2200

            and so on.

            The Charset property of the BodyPart is set with \”null\”, so I think that is cleared.

            but have you ever tried this?

            I set a Return pInMsg
            to the end of the code and repace the Return OutMsg with this.
            So i told the component to give the stream, it gets from the pipeline, back to the pipeline without any processing.
            And than I start the pipeline.exe with a Zip file in place of the XML file as input.
            In this constallation I thought that the output file must be the same as the input file, because there is no processing on the file.
            But there was the same effect like I create my zip files from the xml file.

            • #14983

              You understand it right.

              And it happens also when i completly remove my component from the pipeline.

              but i don’t use a adapter to send out the message.

              i call the Pipeline out of the debugger from the VisualStudio2005 with this command:

              C:\\Programme\\Microsoft BizTalk Server 200606\\SDK\\Utilities\\PipelineTools\\Pipeline.exe

              and this parameter:

              \”C:\\VisualStudio\\BizTalk-Server\\Test\\PipelineTestZip\\PipelineTestZip\\ZipTest.btp\” -d \”C:\\VisualStudio\\BizTalk-Server\\Test\\In\\Preisdatei XML.xml\” -m C:\\VisualStudio\\BizTalk-Server\\Test\\out\\%MessageID%.zip

              In my pipeline the is only my Zip Component. And it is configured as an coder at the end of the pipeline.

              • #14984

                I’ll try your library, but I have some problems with it.

                First the reference ‘PipelineObjects’ from Winterdom.BizTalk.PipelineTesting will not be found. To which DLL/component does it refer?

                And second, how do i start the project from VisualSudio2005 with NUnit?

                You should know that this is my first project with VS and i had never written a dll before. 🙁

                My last programs are simple C console programs and that was long time ago.

                • #14985

                  Is your custom pipeline component running as the last step of the pipeline (i.e. in the Encode step)? You don’t want to run it before the assembler (it would’t run anyway).

                  It might also be that you’re forgetting to flush your stream correctly while writing to it, or that you’re not rewinding the stream correctly before handling it back to the messaging engine once you finish executing. What’s your code like?

                  • #14986

                    I’m not even sure I understand your code. Why are you creating a file from your pipeline component? What for?

                    • #14987

                      A couple of things I see:

                      1- You’re creating a BinaryReader you’re not using
                      2- Have you tried not doing this:
                      pContext.ResourceTracker.AddResource(OutputStream);

                      Other than that, I don’t see anything else that could be the problem (though the code is very inneficient and could certainly be made to work in a totally streaming fashion, I think).

                      • #14988

                        There isn’s such thing as a hidden assembler (besides, assembly happens before encoding in send pipelines).

                        However, it is possible that the adapter might interpret that somehow. At least, you might want to try clearing out the Charset property of the BodyPart, that might help somewhat.

                        That said, how do you know you’re giving the pipeline an UTF-16 encoded file originaly?

                      • #14989

                        What adapter are you using to send out the message? the FILE adapter?

                        Let me see if I understand what you’re saying: You say that if you remove all the ZIP’ing code and just trace the message as it gets to your pipeline component, you see something that looks like it’s UTF-16 encoded, but when the message finally comes out of the adapter, it looks like if it was UTF-8 encoded, right?

                        If so, does this happen too if you completely remove your custom pipeline component from the equation?

                        One final thing: How is your custom pipeline configured? (i.e. what components and at which stage?)

                      • #14990

                        Ahh, that explains it. Pipeline.exe does quite a few assumptions about how your pipeline runs, and it meddles with encodings quite a bit. It’s quite possible that’s the likely cause of your problem.

                        If you want something more controlled that still allows you to test your component outside of biztalk, try my PipelineTesting library [1] alongside NUnit (or a simple console runner if you want).

                        [1] [url]http://www.winterdom.com/weblog/2006/04/27/PipelineTestingLibraryPart1.aspx[/url]

                      • #14991

                        The PipelineObjects.dll library is included with BizTalk, it will be in the same directory as the Pipeline.exe tool you were using before.

                        For an example of using it with NUnit, check the included unit tests for the library itself; they are writen using NUnit. Just take a look at the Winterdom.BizTalk.PipelineTesting.Tests project.

                        To use NUnit, all you need to do is fire up the NUnit GUI tool, and load the test library you created and run it. For information on using NUnit itself, check out http://www.nunit.org/index.php?p=quickStart&r=2.2.8

                        Finally, if you don’t feel comfortable using NUnit, that’s OK. just write a simple console program that calls the library to do the test you want; it will work just as well!

Viewing 1 reply thread
  • The forum ‘BizTalk 2004 – BizTalk 2010’ is closed to new topics and replies.