1

I just finished python course on a Codeacademy and read a book about networks. Now I want to send a GET message using HTTP protocol to a google server(or somewhere else) and read a request using python script only. But I don't want to use anything but socket module

The reason is that it seems to be very easy. Just create a TCP connections and send a message, then receive the answer. I feel being a looser using a special library for this thing, I just need to make it by myself!

import socket

servername = 'google.com'
serverport = 80

clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sentence = 'GET / HTTP/1.1\nHost: google.com\nConnection: close'.format(servername)
print(sentence)

clientsocket.connect((servername, serverport))
clientsocket.send(sentence.encode())

new_sentence = clientsocket.recv(1024)
print('from server: {}'.format(new_sentence.decode()))

clientsocket.close()

I've tried different approaches of code, but all at all it should looks like this as far as I understand, what is the problem? Why it doesn't work?

Olvin Roght
  • 7,677
  • 2
  • 16
  • 35
Danil Kutny
  • 69
  • 1
  • 3
  • HTTP is an easy enough protocol, you could achieve your goal by writing and reading plain text to and from sockets. HTTPS on the other hand, which is the only way most modern websites, Google included, will talk to you - good luck with implementing that from scratch without any libraries. – Tomalak Aug 25 '19 at 11:58
  • My guess is that you don’t have a couple of new lines at the end of your request. That is part of the HTTP protocol to show the end of the empty body. – quamrana Aug 25 '19 at 11:58
  • @Tomalak, it's not as hard as it seems to be. Ofc, you can't create such as monster as `requests` by yourself, but write simple lib with basic features is more than real. Anyway, there's no reason to do this other then educational purposes. – Olvin Roght Aug 25 '19 at 12:19
  • *"The reason is that it seems to be very easy."* - it's not. Things like HTTP keep-alive, chunked transfer encoding, compressed transfer (content-encoding) .. can make it sufficiently complex. You can make it a bit simpler by using HTTP/1.0 instead of HTTP/1.1 so that you don't need to deal with chunked transfer encoding and have implicitly HTTP keep alive off. Apart from that, read the standard. It is long and there is a reason for it. – Steffen Ullrich Aug 25 '19 at 13:27

2 Answers2

3

You don't recieve response from server, cause it still wait data from you. In HTTP protocol you should send \r\n\r\n(CRLF CRLF) to inform server about end of request headers.

CRLF = '\r\n'
sentence = CRLF.join(['GET / HTTP/1.1', 'Host: ' + servername, 'Connection: close', '', ''])
Olvin Roght
  • 7,677
  • 2
  • 16
  • 35
  • 2
    Actually, it should be `\r\n` instead of `\n`. And some servers will complain if you don't do it properly. Also, this will implicitly enable HTTP keep-alive (since HTTP/1.1 is used) which means that the server might not close the connection after the response was sent - which again makes the code hang. – Steffen Ullrich Aug 25 '19 at 13:16
  • @SteffenUllrich, yes, you're right. Once before I've had some issues on Windows with `\r\n`. Python somehow wrote `0d 0d 0a` instead `0d 0a`. That's why I've mistakenly wrote `\n`. Edited. – Olvin Roght Aug 25 '19 at 13:20
0
import socket

servername = 'google.com'
serverport = 80

clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

sentence = '''GET / HTTP/1.1
Host: {}
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US;q=0.5,en;q=0.3
Connection: close


'''.format(servername)
print(sentence)

clientsocket.connect((servername, serverport))
clientsocket.send(sentence.encode())
new_sentence = clientsocket.recv(1024)
print('from server: {}'.format(new_sentence.decode()))
clientsocket.close()
Роман Александров
  • 23
  • 6