42

Trying to get my mind around google protobuf. I found some implementation of protobuf in C# but they seems to lack one feature: the ability to generate .proto files automatically from an existing C# class decorated with attributes.

The reason I want to do it this way instead of going from auto-generated C# classes from .proto file is because I already have the C# classes defined in my project and I don't want to duplicate them just to satisfy ProtoBuf.

Does anyone have encountered such a scenario?

Update

Is this possible to just decorate a C# class and not use a .proto file to use protobuf?

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
Stécy
  • 11,951
  • 16
  • 64
  • 89

2 Answers2

49

Good news; what you have described (having existing C# classes) is the expected use-case of protobuf-net. All the .proto stuff ("protogen", the VS add-in, etc) were all added as afterthoughts. The core of protobuf-net doesn't know about them or care about them.

protocol buffers defines a DSL (.proto, as you mention) that is shared between implementations, and is (sometimes) used for code generation. When I first wrote protobuf-net, the code-generation aspect wasn't my biggest concern - simply that .NET developers are generally guilty (myself included) of "implementation first" rather than "contract first".

As a consequence, protobuf-net doesn't need .proto files to work; an attributed class is sufficient to unambiguously serialize/deserialize. Just use Serializer.Serialize , .Merge and .Deserialize (etc).

That said; it does include some very under-developed and experimental support for this:

string proto = Serializer.GetProto<YourType>();

This is far from complete, but may work for simple types. If you have some specific cases where it fails, then let me know (add a comment or log an issue). However; most of the time, people interested in .proto would write the .proto first and work from there.

Examples of working decorated types are shown on the project home page; it is entirely up to you whether you use WCF attributes, xml attributes or protobuf-net attributes (although the latter provide more control over some specific serialization points, such as inheritance and numeric layouts).

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • 2
    Great, this would fit my requirements then. However, one more thing, does Protobuf-Net implements the whole specification of Protobuf? – Stécy Aug 26 '09 at 14:52
  • Pretty-much; the wire-format spec isn't vast, to be honest. Are you looking to interop with a separate .proto implementation? It should work fine, but if you *have* a .proto (for the other end), I'd recommend generating a DTO (from the .proto) and shim to that from your types. It also gets creative, allowing protobuf-net to support inheritance (which is **not** part of the core .proto spec), but done in a way that still allows full interop with other clients. – Marc Gravell Aug 26 '09 at 14:58
  • I am not looking to interop with another implementation. I will use protobuf-net as an efficient mechanism for serializing/deserializing a whole hierarchy of objects between two applications. – Stécy Aug 26 '09 at 15:04
  • One other thing, is it possible to have a serialization of a property defined as an IList but the actual object is a List? – Stécy Aug 26 '09 at 19:46
  • That should be fine as long as your class creates the concrete list. If it doesn't work, let me know (I'll try to remember to test it on the train tomorrow). – Marc Gravell Aug 26 '09 at 21:48
  • I've just checked, and a pre-initialized `IList` works fine; I've also tweaked it to use `List` if the list *isn't* pre-initialized. – Marc Gravell Aug 27 '09 at 06:08
  • You're right! I was not initializing the concrete list in my private constructor. I like ProtoBuf-Net more and more... :) – Stécy Aug 27 '09 at 11:47
  • 2
    @Marc Gravell - How does this fair with generating .proto definitions for protobuf "services"? We're adding protobuf-net support to an existing Web Service so I'd like to keep the service as "the definition" rather than create a .proto from scratch. The client is not .NET so we can't go .proto-less – Richard Szalay Dec 16 '10 at 11:51
  • 1
    @Richard - nothing written there as-yet, but you could probably hack something together via reflection. – Marc Gravell Dec 16 '10 at 12:20
  • Is this still possible? I have tried this and looked at the current version (r480) and trunk and they seem to throw NotImplementedException. – Mike Schall Apr 23 '12 at 15:22
  • 1
    @Mike it will be, but not today - it is perhaps the largest thing not yet reimplemented – Marc Gravell Apr 23 '12 at 17:12
  • 2
    Marc thanks for your great support on .NET and Protobuf but as you said "protobuf-net doesn't need .proto files to work; an attributed class is sufficient to unambiguously serialize/deserialize" it kind of breaks the rules. There is no another api in other platforms which does not need .proto files. And Google's definition also requires it, so protobuf-net is cool and easy to use with attributes, it kind of breaks interoperability and point to use protobuf. Its name should be corrected as "protobuf based serializer for .net" since its not the protobuf as creators intented. – Alper Dec 15 '16 at 13:03
  • 5
    @Alper my intent is not to tick boxes with Google; my intent is to provide a tool that is useful, convenient, and compatible; if you want a .proto file, there is functionality built in to export one, as shown in the answer. – Marc Gravell Dec 15 '16 at 14:49
  • @MarcGravell What's the current state now (5 years later)? Does .proto generation still work, and does it also work for gRPC? Our use case is that we have existing C# interfaces which we want to gRPCify, but also open them for clients in other languages. – MarkusSchaber May 05 '21 at 15:11
  • 1
    @MarkusSchaber yes, protobuf-net.Grpc.Reflection - something like `SchemaGenerator` IIRC - it isn't *as complete* as the schema generation from protobuf-net, though – Marc Gravell May 05 '21 at 16:06
  • I am so confused with `protobuf-net`. When I add a `.proto` file to the project, VS generates a file that I can use to generate my service endpoints and my client. I don't have to do another thing to consume that client. How does `protobuf-net` work here? Is it ONLY for generating the models? Do I still need to manually create a client project that consumes these models? How do I implement the service? Do I have to manually serialize/deserialize the models? Where is the documentation for this thing?!? – Serj Sagan Nov 22 '21 at 08:00
  • 1
    @SerjSagan it sounds like you're talking about services, so: more the gRPC side of things than just serialization, in which case: https://protobuf-net.github.io/protobuf-net.Grpc/ (the "getting started" page is a good place to ... start) - and since you seem to be working "schema first" (i.e. from a .proto file), the build tools are documented here: https://protobuf-net.github.io/protobuf-net/contract_first - however! I can't see your csproj, but I wonder if you're actually using the Google build tools, not the protobuf-net ones – Marc Gravell Nov 22 '21 at 09:20
13

Before Skeet Marc runs in here and gets massive ups, let me point out protobuf.net.

  • 14
    Actually, that would be me ;-p – Marc Gravell Aug 26 '09 at 13:44
  • 1
    Yeah, I looked at the project but can't seem to find documentation about how to generate .proto files from a decorated C# class. Or is it needed to have a .proto file to use protobuf at all? – Stécy Aug 26 '09 at 13:45
  • Oh, Jon has a pb implementation; but a different one; dotnet-protobufs: http://github.com/jskeet/dotnet-protobufs/tree/master – Marc Gravell Aug 26 '09 at 21:48
  • 1
    And for the record, it wouldn't be a "fit" for this question (else I would have happily recommended it) - as it uses the codegen approach exclusively (exactly what the OP *doesn't* want). – Marc Gravell Aug 26 '09 at 21:49