Java reading files via FIO

Question

I am making a TFTP application. As per protocols RFC all data must be send in chunks of 512 bytes in max size. So each packet can be <= 512 bytes.

I read each file in to a byte[] outgoingData = new byte[512]; array and i send it to Client however it seems that something goes wrong when this gets a file that is smaller than 512 bytes in total size like a ascii file or a .ini, .css, .html, etc..

Oddly for UDP protocol each transfer up to 3mb has came past without a big loss. The only loss that seems to happen is when the last chunk of a file is read that is less than 512 bytes.

private void sendData() throws Exception
{
    DatagramPacket data = new DatagramPacket(outgoingData, outgoingData.length, clientAddress, clientPort);
    InputStream fis = new FileInputStream(responseData);

    int a;
    while((a = fis.read(outgoingData,0,512)) != -1)
    {
        serverSocket.send(data);
        Thread.sleep(5);
    }
}

Since this is a problem regarding reading the file how can i fix the loss at the end of the file and the problem where it does not read a file smaller than 512

Client:

private void receiveData() throws Exception
{
    DatagramPacket receiveData = new DatagramPacket(incomingData, incomingData.length);
    OutputStream fos = new FileOutputStream(new File("1"+data));
    while(true)
    {
        clientSocket.receive(receiveData);
        if(receiveData.getLength() == 512)
        {
            fos.write(incomingData);xx
        } else {
            fos.write(incomingData);
            fos.close();
            break;
        }
    }
    clientSocket.close();
}

possible duplicate of [Java - UDP sending data over socket.. not rec. all data](http://stackoverflow.com/questions/9589905/java-udp-sending-data-over-socket-not-rec-all-data) — user207421, Mar 07 '12 at 01:06

Daniel Lubarov · Answer 1 · 2012-03-07T00:49:10.647

1

Your packet (data) has a fixed length of outgoingData.length (512?) right now. So when you call serverSocket.send(data), it sends all the data in the outgoingData array.

The problem is that fis.read(outgoingData, 0, 512) sometimes reads less than 512 bytes of data. This is likely to happen at the end of the file input stream, when there simply isn't that much data left to read. But it could also happen earlier (unlikely in practice, but you should still check to be safe).

You've already stored the number of bytes that is actually read in a. Just pass this number to setLength so that the DatagramSocket will know to only send this much data.

This should do the trick:

private void sendData() throws Exception
{
    DatagramPacket data = new DatagramPacket(outgoingData, outgoingData.length, clientAddress, clientPort);
    InputStream fis = new FileInputStream(responseData);

    int a;
    while((a = fis.read(outgoingData,0,512)) != -1)
    {
        data.setLength(a);
        // or this
        //data = new DatagramPacket(outgoingData, a, clientAddress, clientPort);

        serverSocket.send(data);
        Thread.sleep(5);
    }
}

For receiving:

private void receiveData() throws Exception {
    DatagramPacket receiveData = new DatagramPacket(new byte[512], 512);
    OutputStream fos = new FileOutputStream(new File("1"+data));
    while (true) {
        clientSocket.receive(receiveData);
        if (receiveData.getLength() == 512) {
            fos.write(receiveData.getData());
        } else {
            fos.write(receiveData.getData(), receiveData.getOffset(), receiveData.getLength());
            break;
        }
    }
    fos.close();
    clientSocket.close();
}

edited Mar 07 '12 at 00:49

answered Mar 07 '12 at 00:10

Daniel Lubarov

7,796
1
37
56

Your saying i should put DatagramPacket bellow the while loop and instead of calling outgoingData.length i should call a ? – Sterling Duchess Mar 07 '12 at 00:14
@kellax yeah that should work also, if I understand you. (You mean construct a new DatagramPacket inside the while loop, right?) I would keep `data` as it is and just call `setLength` in the while loop. But either way should be fine. – Daniel Lubarov Mar 07 '12 at 00:21
Yes but how in this method i cannot put it in data = new..(..,data.setLenth(a)) since a has not been initialized yet. Can you show on my example – Sterling Duchess Mar 07 '12 at 00:23
Hm I must have misunderstood you. I added some code similar to your example, is it clear? – Daniel Lubarov Mar 07 '12 at 00:29
I have it written like so. iv put data.setLength(a) in the while loop and on the other side .. the client receives this as shown bellow in second answer however file smaller than 512 still isnt being passed. – Sterling Duchess Mar 07 '12 at 00:34
On the receiving end, are you sure you're reading the length of the received DatagramPacket (`packet.getLength()`), rather than the length of the underlying buffer (`packet.getData().length`)? – Daniel Lubarov Mar 07 '12 at 00:40
I added the code that receives the data on the Client side to my first question. – Sterling Duchess Mar 07 '12 at 00:42
You need to give the `FileOutputStream` the length of the packet, like `fos.write(incomingData, receiveData.getOffset(), receiveData.getLength())`. – Daniel Lubarov Mar 07 '12 at 00:44
I tried adding receiveData.getLength() but cannot call it in the arguments. – Sterling Duchess Mar 07 '12 at 00:44
Could there be a conflict here: receiveData = new DatagramPacket(incomingData, incomingData.length); as datagram is expecting a 512 byte length but it gets less ? – Sterling Duchess Mar 07 '12 at 00:49
Here is my server and client class: http://www.copypastecode.com/180187/ <- Server http://www.copypastecode.com/180191/ <- Client – Sterling Duchess Mar 07 '12 at 00:51
I have it written like that however it does not seem to ever get past 512. For some reason its always expecting 512 and that while(true) never stops since Client keeps running until user interrupted. Seems it keeps getting 512 block size even tho less was sent. – Sterling Duchess Mar 07 '12 at 01:01
@kellax it isn't clear to me how the receiver is ever going to know when the file has completely arrived. – user207421 Mar 07 '12 at 06:09
@EJP yeah the code assumed that the last chunk of the file would have less than 512 bytes while the other chunks would fill the buffer. Likely to work in most cases but not a safe assumption. – Daniel Lubarov Mar 07 '12 at 07:44
@kellax (to make sure he reads this) *Not at all* a safe assumption. It fails if (a) the file is a multiple of 512 bytes, (b) a short read happens in the middle of reading the file, (c) the datagrams arrive out of order, (d) the last datagram doesn't arrive, ... – user207421 Mar 07 '12 at 08:51

WalterM · Answer 2 · 2012-03-07T01:36:44.543

1

    private void sendData() throws Exception
{
    DatagramPacket data = new DatagramPacket(outgoingData, outgoingData.length, clientAddress, clientPort);
    InputStream fis = new FileInputStream(responseData);

    int a;
    while((a = fis.read(outgoingData,0,512)) != -1)
    {
        data.setLength(a); //'a' is the number of bytes read setLength(int) lets you set the length of bytes you want to sent
        serverSocket.send(data);
        Thread.sleep(5);
    }
}

setLength(int) Method DatagramPacket on the receiving end now, you do something like this:

byte[]buffer = new byte[512];
DatagramPacket packet = new DatagramPacket(buffer,buffer.length);
while(socket.isBound() && !socket.isClosed()){
    socket.receive(packet);
    system.out.println("Packet Length: "+packet.getLength());
    //code here
}

FIX: IN Server.java

private void sendResponse(String res) throws Exception
        {
            if(res.equals("Y"))
            {
                // Send ACK -> Send File
                DatagramPacket x = new DatagramPacket(outgoingData, outgoingData.length, clientAddress, clientPort);
                serverSocket.send(x); //<<SEND ACK
                sendData();
            } else {
                String error = "ERROR: The file you requested does not exist.";
                outgoingData = error.getBytes();
                DatagramPacket err = new DatagramPacket(outgoingData, outgoingData.length, clientAddress, clientPort);

                serverSocket.send(err);
            }
        }

I noticed your not sending an ack and the client is expecting it, that seems to be your only problem. the first packet currently does nothing for you. now you just need to set the ack packet to what you want. I just sent it while it doesn't contain the string "ERROR" in it.

edited Mar 07 '12 at 01:36

answered Mar 07 '12 at 00:23

WalterM

2,686
1
19
26

It is exactly how i have it written apart from sys.out being a output stream. However any block smaller than 512 from what i guess is not being passed. – Sterling Duchess Mar 07 '12 at 00:37
Try printing out `packet.getLength()`, I want to see what it says. – WalterM Mar 07 '12 at 00:52
Says 512 even tho what it had passed was only about 20 bytes. My code http://www.copypastecode.com/180187/ <- Server http://www.copypastecode.com/180191/ <- Clientode: – Sterling Duchess Mar 07 '12 at 00:55
think I found your problem. in this Method: `receiveResponse` in Client.java you are creating a new String but using the whole buffer. instead of that try this: `String rsp = new String(response.getData(),0,response.getLength());` – WalterM Mar 07 '12 at 01:07
nvm doesn't look like your doing anything with that string really, but you should fix that anyways – WalterM Mar 07 '12 at 01:10
Problem still persists. I am receiving less than 512 bytes however i am even printing getLen and i keep getting 512 length. – Sterling Duchess Mar 07 '12 at 01:12
That string lets me check is the response is an Error if server cannot find the requested file to send it will send out a Error message. – Sterling Duchess Mar 07 '12 at 01:13
alright fixed it and even tested compiling it and it works. file was copied. look up^^^ – WalterM Mar 07 '12 at 01:37

Java reading files via FIO

2 Answers2