0

This one is going to kill me. I'm so close to getting this done except for this one stupid problem. And I am not sure I will be able to adequately describe the problem, but I'll try.

My enterprise app uses the iPhone camera to take pictures of receipts of purchases made by our field personnel. I used a real cool API for turning the jpeg data to base 64 (https://github.com/nicklockwood/Base64) to send via TCP connection to the VB 2010 server, which reads it as a text string and converts it back to a binary.

When the base64 file is created, it is first saved to disk on the phone, because there may be more images. Then when ready to send, the process will read each base64 file and send it one at a time.

The text string created by the base64 function is quite large, and at first it was only sending about 131,000 bytes, which would convert back to binary easily enough but would render about 1/4 to 1/3 of the image. I just figured that the data was being truncated because the app was trying to get ahead of itself.

So then I found a nice snippet that showed me how to use the NSStreamEventHasSpaceAvailable event to split the base64 string into several chunks and send them sequentially. (http://www.ios-developer.net/iphone-ipad-programmer/development/tcpip/tcp-client) That works great insofar as it sends the full file -- that is, the resulting file received by the server is the correct size, the same as the base64 file before it's sent.

The problem here is that at some point the file received by the server is corrupted because it seems to start all over at the beginning of the file... in other words, the data starts to repeat itself.

The odd part is that the repeating part starts at exactly the same spot in the received file every time: at position 131016. It doesn't start the repetition at the end of the file, it just interrupts the file at that point and jumps back to the beginning. And it happens that that was the size of the file that was sent before I started using the HasSpaceAvailable event. I can't figure out what the significance of the value 131,016 is. Maximum buffer size somewhere?

Using all kinds of NSLogs and breakpoints, I have pretty much determined that the data is leaving the phone that way, and not being scrambled by the server. I also wrote in an NSMailComposeViewer method that would email me the base64 file as an attachment, and it comes through perfectly.

Here is the code for when the file is read from disk and sent to the server:

    int i;
    for (i = 0; i < [imageList count];i++){
    NSArray *documentsPath = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *documentsDirectory= [documentsPath objectAtIndex:0]; //Get the docs directory
    NSString *imagePath = [documentsDirectory stringByAppendingPathComponent:imageFileName];
    imageFileName = [imageList objectAtIndex:i] ;
    NSLog(@"Image index: %d - image file name: %@",i,imageFileName);

    BOOL fileExists = [[NSFileManager defaultManager] fileExistsAtPath:imagePath];
    if(fileExists == YES){
        NSString *imageReceipt = [NSString stringWithContentsOfFile:imagePath encoding:NSASCIIStringEncoding error:nil];
        int32_t imageStringLen = [imageReceipt length];
        NSString *imageSize = [NSString stringWithFormat: @"%d",imageStringLen];
        NSString *currentImage = [NSString stringWithFormat:@"image,%@,%@,%@",imageFileName,imageSize,imageReceipt]; //creates a CSV string with a header string with the filename and file size, and then appends the image data as the final comma-separated string.
        data = [[NSMutableData alloc] initWithData:[currentImage dataUsingEncoding:NSASCIIStringEncoding]];

        [outputStream write:[data bytes] maxLength:[data length]];

And then here is the code that uses the HasSpaceAvailable event:

        case NSStreamEventHasSpaceAvailable:
        if (data != nil)
        {
            //Send rest of the packet
            int ActualOutputBytes = [outputStream write:[data bytes] maxLength:[data length]];
            int totalLength = [data length];

            if (ActualOutputBytes >= totalLength)
            {
                //It was all sent
                data = nil;

            }
            else
            {
                //Only partially sent
                [data replaceBytesInRange:NSMakeRange(0, ActualOutputBytes) withBytes:NULL length:0];       //Remove sent bytes from the start
            }
        }

        break;

(I especially like this code because it would allow placing a ProgressView control on the screen.)

The network stream event handler code is in the root view controller, but the image data in base64 is being sent from another view controller. My instinct tells me that this is not a problem because it has worked fine until now, but with much shorter strings.

Now, there's one other issue that may be related -- and probably is. I can't seem to complete the transfer of the data unless I close the app. The server doesn't see it until the connection is closed, I guess. I have tried placing [outputStream close] in various places in the code to no avail. I've also tried terminating the base64 string with a linefeed or carriage return or both.

The server is programmed to save the file when it has seen the correct number of bytes, but that never happens until the app is closed. I know by using WireShark on the server that some of the data is being received, but the remaining data, as I have said, doesn't arrive until the app is closed.

I suspect that this last part (completing the transfer) is the problem, but for the life of me, I can't find anything online that addresses this, unless I am just too ignorant to know what search terms to use.... which is highly likely.

I hope I have given enough information. Can anyone help me?

Bill Norman
  • 883
  • 11
  • 29
  • I neglected to factor in the header bytes when calculating the size of the boundary (131016). The header adds 56 bytes to the file, making the entire transmission (before the truncation) 131072, which is 2^17. I still don't know the significance of it. It's my understanding that NSData can hold as much as 2gb. – Bill Norman Feb 18 '13 at 19:51

1 Answers1

1

EDIT

The solution to the problem appeared to be different than what was suspected in the original answer. Through the discussion in the comments the QP has been led to a solution. Here is a summary:

The transmission of data through a raw TCP socket requires thorough handling of the enqueing logic, which was not properly taken into account by the QP. I recommended to use the socket library CocoaAsyncSocket which handles this part of the task and which in turn led the QP to a working solution.

Original Answer:

I'd guess NSString is not up to the task. It's made to hold, well, strings.

Try to read the file into an NSData directly instead and sending it as binary. (Its ascii in the end, isn't it?) Besides, this will be much more resource friendly than your current code.

ilmiacs
  • 2,566
  • 15
  • 20
  • Thanks for your response. I've been playing with the idea, but I have to wonder: At what point would NSString be inadequate? The base64 conversion program returns the data as an NSString, and that string works when it's saved as a file, and then when it's retrieved from the file and attached to an email; just not when it's being sent via TCP. One (at my minimal level of expertise) would think that if an NSString worked in those places, it ought to work up to the point where it's being converted to NSData and sent over the network. – Bill Norman Feb 19 '13 at 17:05
  • Don't get the idea that I am doubting you. I only wonder if it's advisable to avoid the NSString through the entire process, though I don't know how it can be done. At any rate, I have tried moving the data from the file to the outputStream entirely as NSData, and although there is a slight difference in behavior, it's still only sending part of the data. – Bill Norman Feb 19 '13 at 17:07
  • 1
    Yes, you are right, my guess was too wild. After a second read I recognize that you are using raw tcp sockets. Enqueuing them is something notoriously hard to get right. So I'd consider using a tcp socket library that would take that burden from your shoulders. Take a look at [CocoaAsyncSocket](https://github.com/robbiehanson/CocoaAsyncSocket/) which does just that. (It's in the public domain, so using it with your project should be no problem.) – ilmiacs Feb 19 '13 at 20:52
  • I looked into CocoaAsynSocket, and tried to get it to work in my app. My problem is that I am not smart enough to know what the developer expects me to know without his having to tell me. I managed to get it to where it was throwing only 12 Apple Mach-O Linker errors, but I couldn't get it past that. I can't help but think there's gotta be a much simpler solution to this issue. It keeps getting more and more complicated, and for a guy like me who's really struggling to adapt to the increasingly convoluted world of iOS programming, complexity is my enemy. I'll keep trying. That's how we learn. – Bill Norman Feb 20 '13 at 19:46
  • My friend, what you are trying to achieve here is by no means a pure iOS programming task. You are into networking. This is a huge, complex body of technology and knowledge. Be sure to read at least [the common pitfalls section](https://github.com/robbiehanson/CocoaAsyncSocket/wiki/CommonPitfalls) from the maker of CocoaAsyncSocket. (The linker errors are yet a different problem. You could post details and hope someone would help.) – ilmiacs Feb 21 '13 at 09:58
  • If you desire a simpler solution, you should use some existing proven protocol over TCP and not try to implement your own one. What about HTTP POST? This would allow you to send the image as raw jpeg, no need for the base64 overhead you are creating. – ilmiacs Feb 21 '13 at 10:05
  • My mention of the linker errors was based solely on my frustration that there isn't more fundamental information available for the novice to understand how to use the API. It seems to be the norm on github, that supporting documentation is targeted for those developers who are already as advanced as the github authors. But that's a different problem, I won't go further into that. Thanks, ilmiacs, for your help. I'll try again to get the CocoaAsyncSocket working. Would HTTP POST require having an IIS running on the server? – Bill Norman Feb 21 '13 at 15:54
  • "Would HTTP POST require having an IIS running on the server?" No. Any HTTP server would do. What tcp server do you use right now? "It seems to be the norm on github..." I strongly disagree. Github is not a closed community. Developers of any level participate. ...I think your attitude should change to develop your own skills instead of blaming others not to come down to your level. What you are trying to achieve is easy **if** you know the tools that enable you to do it right. The path to acquire that knowledge may be hard, though. – ilmiacs Feb 21 '13 at 16:28
  • As to the usage of `CocoaAsyncSocket`, there are a lot of examples out there. [Here](http://stackoverflow.com/a/6434761/1651543) is of them. – ilmiacs Feb 21 '13 at 16:32
  • My apologies if my frustrations are seen as blaming others. It was not intended with that sentiment. And I apologize for making a blanket statement about the github authors. In my defense, I said "it seems..." It was a personal observation, not a declaration. I wish not to cause any antagonism on this board because it has been a great help for me over the past year or so, as I have been struggling to learn Objective C for iOS. – Bill Norman Feb 21 '13 at 16:50
  • Okay, I fell back, took a deep breath and had my Thursday burger and fries, then started back into it fresh with CocoaAsyncSocket. This time I managed to get it working insofar as it is communicating for the basic functions. I am glad you encouraged me to continue. At this point it remains to be seen if it works out for my original problem, but I am at least feeling optimistic. Thanks again! – Bill Norman Feb 21 '13 at 21:40
  • Well, I wrestled with CocoaAsynSocket and finally go it working the way I need it to. I also learned a lot in the process. I accepted the answer, not because it was the right one, but because it led to the right one. Can I do that? Thank you! – Bill Norman Feb 24 '13 at 18:29
  • Congrats! I'm glad the discussion helped. I'll update my answer so it reflects more what solved your problem. – ilmiacs Feb 25 '13 at 07:23