How to create HTTP GET request Scapy?

Question

I need to create HTTP GET request and save the data response. I tried to use this:

    syn = IP(dst=URL) / TCP(dport=80, flags='S')
    syn_ack = sr1(syn)
    getStr = 'GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n'
    request = IP(dst='www.google.com') / TCP(dport=80, sport=syn_ack[TCP].dport,
            seq=syn_ack[TCP].ack, ack=syn_ack[TCP].seq + 1, flags='A') / getStr
    reply = sr1(request)
    print reply.show()

But when I print reply I don't see any data response. In addition, when I checked in 'Wireshark' I got SYN, SYN/ACK but I didn't get an ACK.

Image:

Edit:

I try to do that now:

# Import scapy
from scapy.all import *

# Print info header
print "[*] ACK-GET example -- Thijs 'Thice' Bosschert, 06-06-2011"

# Prepare GET statement
get='GET / HTTP/1.0\n\n'

# Set up target IP
ip=IP(dst="www.google.com")

# Generate random source port number
port=RandNum(1024,65535)

# Create SYN packet
SYN=ip/TCP(sport=port, dport=80, flags="S", seq=42)

# Send SYN and receive SYN,ACK
print "\n[*] Sending SYN packet"
SYNACK=sr1(SYN)

# Create ACK with GET request
ACK=ip/TCP(sport=SYNACK.dport, dport=80, flags="A", seq=SYNACK.ack, ack=SYNACK.seq + 1) / get

# SEND our ACK-GET request
print "\n[*] Sending ACK-GET packet"
reply,error=sr(ACK)

# print reply from server
print "\n[*] Reply from server:"
print reply.show()

print '\n[*] Done!'

but its print me in reply from server;

0000 IP / TCP 192.168.44.130:23181 > 216.58.208.164:http A / Raw ==> IP / TCP 216.58.208.164:http > 192.168.44.130:23181 A / Padding None

And I need Line-based text data: text/html.

reply.show() already prints itself, there is no need for the print statement — Yotam Salmon, Jun 07 '16 at 15:12
@YotamSalmon but i need only the data for save it in html file. — Yair B., Jun 07 '16 at 15:15
B I know you do, but the reply variable is an Object, and not a String. You have to get the response load from the object. I don't remember how right now, but let me check and I'll post an answer soon if I find the solution. Is this question for Magshimim project in Networks? — Yotam Salmon, Jun 07 '16 at 15:17
@YotamSalmon Yes, I tried for a long time and I i did not succeed — Yair B., Jun 07 '16 at 15:24
Well, I'm sorry. I tried so hard, and yet couldn't find a solution. By the way, I would recommend taking a look at this website: ell.stackexchange.com — Yotam Salmon, Jun 07 '16 at 16:03
Well, sorry for sounding offensive. I recommended because this website helped me a lot with my English (Which used to be terrible)... Hope you do understand that comment was not meant to offend or laugh at you. — Yotam Salmon, Jun 07 '16 at 19:57
@YotamSalmon Yes, thank you for comment, but how this site can help me ? — Yair B., Jun 08 '16 at 06:27
Err, just reading and trying to answer questions surprisingly improves your English skills. — Yotam Salmon, Jun 08 '16 at 14:10
Try posting your question here: http://magshimim.online/ There are people here that have already finished the project. — Yotam Salmon, Jun 09 '16 at 12:49

score 3 · Answer 1 · edited May 23 '17 at 10:30

You are sending a RST segment in response to the SYN-ACK because your kernel has no knowledge of the SYN you sent via Scapy (see here). This could be solved with an iptable rule:

iptables -A OUTPUT -p tcp --tcp-flags RST RST -s <your ip> -j DROP

Because you are ending the connection with that RST segment, when you send your HTTP request, the endpoint answers with a RST too because connection is not established and so you are using show() on a RST segment with no data, that is why you do not see anything.

score 2 · Accepted Answer · answered Jun 09 '16 at 02:45

2

You are sending a SYN and correctly receiving a SYN_ACK. At this point, you should generate and send an ACK based on the SYN_ACK that you've received, and THEN finally transmit the HTTP GET request. It seems that you are somewhat confused about the TCP 3-way handshake mechanism. In short, you are not supposed to 'get' an ACK, you are supposed to generate and send this yourself.

answered Jun 09 '16 at 02:45

wookie919

3,054
24
32

So you can fix my code, I try and it also don't working for me. – Yair B. Jun 11 '16 at 15:50
@YairB. Firstly, have you implemented the workaround that was mentioned earlier to get around the RST issue? First ensure that RST does not get automatically sent by your OS. Then try generating an ACK in response to the SYN_ACK. If this doesn't work, please upload a new wireshark capture at this point then we can take it from there. – wookie919 Jun 12 '16 at 21:34
@wookie919 He is already acknowledging the SYN-ACK in its `request` segment, however, you are right, it is good practice to send an ACK without data before the actual request as this is a faster way to acknowledge segments (and establish the connection in this example). – Jeff Bencteux Jun 13 '16 at 07:20
Now I update question.. Look what I try to do and what its print me. – Yair B. Jun 13 '16 at 10:26

score 1 · Answer 3 · answered Nov 11 '16 at 23:50

After setting the rule in your iptables as has been suggested above, you could do the following :

from scapy.all import *

seq = 12345
sport = 1040
dport = 80

ip_packet = IP(dst='192.168.56.107')
syn_packet = TCP(sport=sport, dport=dport, flags='S', seq=seq)

packet = ip_packet/syn_packet
synack_response = sr1(packet)

next_seq = seq + 1
my_ack = synack_response.seq + 1

ack_packet = TCP(sport=sport, dport=dport, flags='A', seq=next_seq, ack=my_ack)

send(ip_packet/ack_packet)

payload_packet = TCP(sport=sport, dport=dport, flags='A', seq=next_seq, ack=my_ack)
payload = "GET / HTTP/1.0\r\nHOST: 192.168.56.107\r\n\r\n"

reply, error = sr(ip_packet/payload_packet/payload, multi=1, timeout=1)
for r in reply:
    r[0].show2()
    r[1].show2()

Hope this helps. Basically, the first response you get back does not really hold the HTTP response data. I tested the script against an INETSIM simulated HTTP server and in that case (at least) the first packet (after the 3-way TCP handshake) that the server responded with was a series of NULL (0x00) bytes. Hence using multi somehow did the stuff in my case.

How to create HTTP GET request Scapy?

3 Answers3

Linked