1

Simply put, I have a dictionary (dictData[name]=namedTuple) of namedTuples (records) on a server.

Objective: I want to send the entire thing (dictData) or a single instance (dictData[key]) to the client via a SOCKET connection so it can be printed (shown on screen).

To send a single record I have tried to do the following:

response = dictData["John"]
print (response) #ensure it is the correct record
s.send(response)

However this generates the following error:

"TypeError: 'record' does not support the buffer interface"

I have tried to encode it and convert it but nothing I do seems to work. I am even open to converting it to a STRING and sending the string but I can't seem to find out how to convert a namedTuple to a string either.

And then, no clue where to start to send the entire dictionary to the client so they can print the entire set?

Any help would be much appreciated.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
JSchwartz
  • 2,536
  • 6
  • 32
  • 45

1 Answers1

3

Sockets can only send and receive bytes, so you need to serialise your named tuples and dictionary to something else.

You could use JSON to produce a string representation of your tuples and dictionary, for example. The json library produces a (unicode) string when encoding, you'll need to encode that to UTF-8 (or similar) to produce bytes:

import json

# sending one tuple
response = json.dumps(dictData["John"])
s.send(response.encode('utf8'))

# sending all of the dictionary
response = json.dumps(dictData)
s.send(response.encode('utf8'))

This will not preserve the named tuple attribute names; the values are sent over as a JSON array instead (so an ordered list).

Another option is to use the pickle module; this would require the listener on the other side to also be coded in Python and to have the same record named tuple type importable from the exact same location, however.

When Pickle loads the data, the name of the full qualifying name of the namedtuple type is included, and you must be able to import that type on both ends of the socket. If you have a line in a module at the global level:

record = namedtuple('record', 'field1 field2 field3')

then from yourmodule import record is possible, but the exact same import should work on the other side too.

pickle.dumps() produces a bytes object which can be written to the socket without encoding.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Actually, a better encoding in this case would be [pickle](https://docs.python.org/2/library/pickle.html). – whereswalden Jul 12 '14 at 17:48
  • 2
    @whereswalden: that depends on what is listening on the other side. – Martijn Pieters Jul 12 '14 at 17:48
  • @Martijn you mentioned ordered, can this be sorted by name as well? – JSchwartz Jul 12 '14 at 17:49
  • Pickle or JSON - not sure I know which is best ... the other side is just a dumb client (socket) that will print the data to a user. – JSchwartz Jul 12 '14 at 17:49
  • @JSchwartz: for your named tuples? – Martijn Pieters Jul 12 '14 at 17:49
  • @JSchwartz: If the other side is also coded in Python and you can have the `record` object definition importable (pickle will use the same full name with module to import the same class again) then you could use pickle here and get the same object types. – Martijn Pieters Jul 12 '14 at 17:51
  • @MartijnPieters well the requirement is for "sorted users", specifically the client is prompted "select operation" and if they choose 7 (print report) or 6 (print record for "John") I send the request to the server who needs to respond with the requested data. Client is only used to SEE data on the server – JSchwartz Jul 12 '14 at 17:51
  • The other side is PYTHON 3.x also (I am writing both together) – JSchwartz Jul 12 '14 at 17:51
  • In that case, pickle would work fine. Just unpickle it before handling it on the other side. – whereswalden Jul 12 '14 at 17:52
  • @JSchwartz: Then you mean to sort the *keys* of the dictionary. Do so on the receiving end when printing; dictionaries themselves are unordered. – Martijn Pieters Jul 12 '14 at 17:52
  • @MartijnPieters but the receiving end (using pickle) will have a dictionary anyways no? how do I sort it? – JSchwartz Jul 12 '14 at 17:53
  • @JSchwartz: Yes, you'd have a dictionary if you use JSON or pickle. You'd use `for key in sorted(yourdict): value = yourdict[key]` to iterate over the sorted keys. – Martijn Pieters Jul 12 '14 at 17:56
  • Odd ... I don't think Pickle plays nicely with a DICT[key]=namedTuples because of the nesting – JSchwartz Jul 12 '14 at 18:25
  • @JSchwartz: Pickle can handle nesting just fine. If you are getting an error then something else is up. – Martijn Pieters Jul 12 '14 at 18:28
  • Hate putting code in comments but ... this is what I have. 'UserData = namedtuple("name", "age address phone")' defined globally then within my SocketServer Handler I do this 'response = self.server.dictData' and then 'pickledData = pickle.dumps(response, -1)' and this generates _pickle.PicklingError: Can't pickle : attribute lookup name on __main__ failed – JSchwartz Jul 12 '14 at 18:32
  • @MartijnPieters when I creating my dict I do 'user = UserData(age, address, phone)' to create an instance of my namedTuple and I add it to my dictionary 'self.dictData[name] = user' – JSchwartz Jul 12 '14 at 18:33
  • 1
    @JSchwartz: right, that's because your `namedtuple()` is called *`name`*, not `UserData`. Use `namedtuple('UserData', 'age address phone')`. – Martijn Pieters Jul 12 '14 at 18:34
  • @MartijnPieters LOL - dead on :) Ok ... last question on this ... on the CLIENT side, how do I rebuild my dictionary or namedTuple (depending on if I sent a single record or the enter dictionary)? – JSchwartz Jul 12 '14 at 18:39
  • @JSchwartz: `object = pickle.loads(data_from_socket)`. – Martijn Pieters Jul 12 '14 at 18:39
  • @MartijnPieters and it will somehow know that object is either a dict[name,namedTuple) or a namedTuple with the proper fields? (you can see I am a C++ developer lol) – JSchwartz Jul 12 '14 at 18:47
  • @JSchwartz: yes, the pickle protocol includes type information. Which is *why* the `namedtuple` object must be importable on the other side too. – Martijn Pieters Jul 12 '14 at 18:48
  • @MartijnPieters can you take a look at my new posting? I am getting EOFError's sometimes and I am worried it is because I am not getting the full pickle data in one shot ... if so ... not sure what to do from here ... http://stackoverflow.com/questions/24726495/pickle-eoferror-ran-out-of-input-when-recv-from-a-socket – JSchwartz Jul 13 '14 at 20:32