46

I'm refactoring legacy C++ system to SOA using gSoap. We have some performance issues (very big XMLs) so my lead asked me to take a look at protocol buffers. I did, and it looks very cool (We need C++ and Java support). However protocol buffers are solution just for serialization and now I need to send it to Java front-end. What should I use from C++ and Java perspective to send those serialized stuff over HTTP (just internal network)?

PS. Another guy tries to speed-up our gSoap solution, I'm interested in protocol buffers only.

Nazgob
  • 8,502
  • 4
  • 40
  • 42
  • FYI. the performance was discussed with the gsoap team (I participate in this) and we recommended to use the `SOAP_XML_TREE` flag or compile with `-DWITH_NOIDREF`. Without this flag the performance can be much lower because of the SOAP encoding with multi-ref elements (id-ref) to serialize graphs (i.e. detect co-referenced objects, analyze cyclic data structures, etc). The suggested flag turns this feature off to serialize XML as trees. Messaging speed is greatly improved. The only bottleneck to performance is network latency and bandwidth. – Dr. Alex RE Feb 27 '16 at 22:38
  • @Alex SOAP is still getting updated? That's actually impressive! That question is 7 years old so not gonna act on that :) – Nazgob Mar 02 '16 at 21:41
  • From what I learned everyone else has read the FAQ (since 2005) with this recommendation. What makes you believe this is a recent development? – Dr. Alex RE Mar 05 '16 at 14:43
  • @alex I did not read that FAQ so your argument can't be true, clearly not everybody have read it. Its also irrelevant as I stated that I was only interested in protocol buffers. – Nazgob Mar 06 '16 at 15:20

5 Answers5

72

You can certainly send even a binary payload with an HTTP request, or in an HTTP response. Just write the bytes of the protocol buffer directly into the request/response, and make sure to set the content type to "application/octet-stream". The client, and server, should be able to take care of the rest easily. I don't think you need anything more special than that on either end.

Sean Owen
  • 66,182
  • 23
  • 141
  • 173
  • 42
    We're using `application/x-protobuf` as described on slide 21 here: http://www.slideshare.net/mokeefe/javaone-2009-ts5276-restful-protocol-buffers – Tim Sylvester Sep 15 '09 at 21:17
  • 1
    Ah even better. Didn't know anyone had suggested a MIME type for it. – Sean Owen Sep 16 '09 at 07:52
  • 7
    MIME types having a name starting with "x-" are nowhere near standard and usually treated as application/octet-stream. – n0rd Sep 16 '09 at 10:52
  • 14
    Looks like there was [a commit in Apr 2012](https://code.google.com/p/google-protorpc/source/detail?r=b742c5cbace121879c5c3544a466c5ff261516af&path=/python/protorpc/protobuf.py) in the ProtoRPC project that states that `application/octet-stream` is the preferred content-type nowadays. – Tommy Stanton Sep 08 '13 at 00:56
  • @znlyj If you have your own client and server, yes, otherwise it is not guaranteed. – Alex Bitek Jun 03 '16 at 11:31
25

ProtoBuf is a binary protocol. It doesn't mix well with SOAP. I suggest you either stick with gSOAP or convert to ProtoBuf entirely.

With protoBuf, you define your protocol in a special format like this,

message Product {
  required string id = 1;
  required string description = 2;
  required int32 quantity = 3;
  optional bool discontinued = 4;
}

The protoc tool can generate code in C++/Java/Python so you can serialize it on one end and deserialize on another.

As you can see, ProtoBuf is designed to serialize individual object. It doesn't provide all the facilities provided by SOAP, like headers. To get around this issue, we use ProtoBuf inside ProtoBuf. We define an Envelope like this,

message Envelope {
  enum Type { 
    SEARCH = 1;
    SEARCH_RESPONSE = 2;
    RETRIEVE = 3;
    RETRIEVE_RESPONSE = 4; 
  }
  required Type type = 1;

  required bytes encodedMessage = 2;

  message Header {
    required string key = 1;
    required bytes value = 2;
  }    
  repeated Header headers = 3;
}

The encodedMessage is another serialized ProtoBuf message. All the stuff in SOAP header now goes to headers.

ZZ Coder
  • 74,484
  • 29
  • 137
  • 169
10

Google frontends prefer application/protobuf.

The ProtocolBufferModel of the Google API client uses application/x-protobuf.

rds
  • 26,253
  • 19
  • 107
  • 134
7

You can serialize/de-serialize protobuf encoded data to/from strings. Send the serialized string as the body of an HTTP POST to Java and de-serialize it. That is one approach. Another way is to make use of the protobuf Service interface. Protobuf allows you to define a service interface in a .proto file and the protocol buffer compiler will generate service interface code and stubs in your chosen language. You only need to implement the protobuf::RpcChannel and protobuf::RpcController classes to get a complete RPC framework. Probably you can write an HTTP wrapper for these classes. See the following links for more information:

http://code.google.com/apis/protocolbuffers/docs/proto.html#services http://code.google.com/apis/protocolbuffers/docs/reference/cpp-generated.html#service http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.service.html

Vijay Mathew
  • 26,737
  • 4
  • 62
  • 93
2

To my knowledge protocol buffers support is available in both C++ and Java, you should be able to exchange protocol buffer serialized data between both systems.

That said, it seems your real question is "How do I send stuff over HTTP between a C++ backend and Java client"

It sound like you need to learn how to use gSOAP, read the docs.

Alternatively you could host a RESTful web server from your C++ app: Look at this: https://stackoverflow.com/questions/298113/how-can-i-implement-a-restful-webservice-using-c++

Next you would need to access the data hosted on your new C++ RESTful server: Look at this: Rest clients for Java?

Community
  • 1
  • 1
Matthew
  • 913
  • 1
  • 8
  • 19
  • 2
    gSOAP is massively overcomplicating the issue. Protobufs work fine over HTTP in their native binary format. – Will Mar 29 '12 at 19:16
  • Overcomplicated? Sure, if you want to transfer binary content then just use Protobufs and REST. But for XML it made our work a lot easier. The complicated parts of WSDL, SOAP, and XML are all taken care of. Also, we got over 5,000 service invocations per second with gsoap. The main overhead is with the TCP/IP, not the gsoap stack. – Dr. Alex RE Feb 27 '16 at 18:40