delphi Drag and drop stream memory over 1GB

Question

I have serious problem and i'm stuck on this :( I need to transfer from my application big files (sometimes more than 1GB) with process it to another application i.e. TotalCommander or Explorer. To do it i need to read part of file from source file to stream, process it and save to output stream. I can't load entire file i need to read about 1-30MB, process it and save. I'm using Drag Drop component suite for Delphi 7

Sample code (in desperate i try many variations):

Var
 F13Stream : TmemoryStream;

Procedure Process_Data;
Var
 FileIn : TFileStream;
begin
 FileIn := TFileStream.Create('c:\SampleData.dat', fmOpenRead);
 Repeat
 ..read from FileIn, process data
 ..save to F13Stream
 ..write F13Stream to output? and free memory to next part data//Sorry i have no idea what to doit :(
 Until eof(source_file);
 FileIn.Free;
end;

procedure TForm13.OnGetStream(Sender: TFileContentsStreamOnDemandClipboardFormat;
  Index: integer; out AStream: IStream);
begin
 Form13.F13Stream := TMemorystream.Create;
 Process_Data;
 try
  AStream := nil;
  AStream := TFixedStreamAdapter.Create(F13Stream, soOwned);
 except
  F13Stream.Free;
  raise;
 end;
end;

procedure TForm13.LMDShellList1MouseDown(Sender: TObject;
  Button: TMouseButton; Shift: TShiftState; X, Y: Integer);
begin
 TVirtualFileStreamDataFormat(DataFormatAdapterSource.DataFormat).FileNames.Clear;
 TVirtualFileStreamDataFormat(DataFormatAdapterSource.DataFormat).FileNames.Add('TestFile');
 DropEmptySource1.Execute;
end;

This is example what i try to do. Maybe another component/code will do this, without problems. I can copy files without size limitations, but i need partially process big files and partially send to another applications as stream/array only by memory not to using temporary disk drive memory. Thanks for any help.

UPDATE:

procedure TForm13.OnGetStream(Sender: TFileContentsStreamOnDemandClipboardFormat;
  Index: integer; out AStream: IStream);
var
 plk1 : TFileStream;
 buff : array[0..65535] of byte;
 read : Longint;
begin
 AStream := nil;
 Form13.F13Stream := TMemoryStream.Create;
 AStream := TFixedStreamAdapter.Create(Form13.F13Stream, soOwned);
 plk1 := TFileStream.Create('h:\Sample2GB_file.dat', fmOpenRead);
 repeat
  read := plk1.Read(buff, 65536);
  AStream.Write(@buff, read, PLongint(65536));
  AStream.SetSize(0);
 until read <> 65536;
 plk1.Free;
end;

It also not work. As result i have 0 bytes file output.

UPDATE2:

If i remove line: "AStream.SetSize(0);" then i have "Out of memory" error.

@David Heffernan I need to drag files from my application and drop it into another application. Files with size more than 1GB and i need to process it (decrypt). — wcale, Sep 07 '11 at 15:47
You need to spend some time cleaning this up. Explain exactly where you are stuck. I can't see any decryption here. I can't see what doesn't work. — David Heffernan, Sep 07 '11 at 15:50
I think he's trying to (OLE) drag-and-drop a virtual file from his app to the explorer (total commander). Like picking up something in your app, dragging it out and having explorer know that a file is available but you only IStream the data when it is actually dropped. He seems to need a way to process the input data in chunks when making it available to OLE... — Marjan Venema, Sep 07 '11 at 15:53
@David Heffernan - You not see any decryption code, because it is very complicated. It is in Process_data procedure. I can't list hundred thousand lines of source code. Problem is in dragging files from my application. I need to transfer files like i.e. WinRar: You drag file from WinRar and drop into desktop. I do something like this: read entire file, decrypt to F13Stream and write it to output as drop. Problem is when files have more than 1GB i can't allocate that size. — wcale, Sep 07 '11 at 18:42
@David Heffernan - I want to see a sample code how to do this (in delphi). How to write something into another application as drop into it (user can drag and drop files). I really don't know how to do this. — wcale, Sep 07 '11 at 19:04
I'm sorry I still can't work out what you want. Maybe someone else has more of a clue. — David Heffernan, Sep 07 '11 at 19:24
@David Heffernan - i want to do something like WinRar do with drag and drop. — wcale, Sep 07 '11 at 19:31
"As result I have 0 bytes file output". That's because you call `AStream.SetSize(0)`. — David Heffernan, Sep 07 '11 at 21:24
@David Heffernan - Yes, but if i don't set AStream.SetSize(0) then i have "Out of memory" error. And this is main problem. — wcale, Sep 07 '11 at 21:38
What's the TMemoryStream for? Where is the external IStream to which you write when you drop? — David Heffernan, Sep 07 '11 at 21:41
@David Heffernan - I don't know where is IStream and how to use it :( Can i write to external IStream more than 1GB without "Out of memory" error ? — wcale, Sep 07 '11 at 21:46
@David Heffernan - I don't know. I used that from example demo from component "Drag and Drop Component Suite Version 5.2". I try to catch when data is transferred by this component, but i fail :( — wcale, Sep 07 '11 at 21:51
It's premature to ask this question. You need to be understanding the very basic mechanisms of drag and drop. Forget all about 1GB files and so on. You have to know how it's all plumber before you can build further. — David Heffernan, Sep 07 '11 at 21:52
But you didn't ask that. The question is clouded in all sorts of confusing detail. You need to step right back and understand the most basic drag and drop example you have. Then work up from there step by step, adding one piece of functionality at a time. That's the best advice I have. — David Heffernan, Sep 07 '11 at 22:03
I would think "AsyncSource with Filestreams" example is just what you need. — Sertac Akyuz, Sep 08 '11 at 08:40
@Sertac Akyuz - No i can't. I can write data partially to stream (buf) but transfer to another application can be on one with: FWriteStream.Write(buf, count); second and next write will not transfer more data. — wcale, Sep 08 '11 at 20:09

score 0 · Answer 1 · answered Apr 24 '17 at 18:13

You cannot actually write to AStream in OnGetStream. It is a source IStream that you need to set and the receiver (where the drop occurs) will read your data from it. If you need to process your source file, do it before the drag start and save the result in another file. Then just configure your new file as drag source. You can simply do it in your Mouse Down event, or if you prefer to use the Virtual Method, do it in OnGetStream event:

procedure TForm13.OnGetStream(Sender: TFileContentsStreamOnDemandClipboardFormat;
  Index: integer; out AStream: IStream);
var
 plk1 : TFileStream;
begin
 AStream := nil;
 // TODO: Create the file to stream (read your source and save it to new_source.dat after your processing)
 plk1 := TFileStream.Create('h:\new_source.dat', fmOpenRead);
 AStream := TFixedStreamAdapter.Create(plk1, soOwned);
end;

You can also create your own TStream class to read, process and return your data on the fly in Read method.

delphi Drag and drop stream memory over 1GB

1 Answers1