12

I want to build a decentralized, reddit-like system using P2P. Basically, I want to retain the basic capabilities of reddit, but make it decentralized, to make it more robust and immune to censorship. This will also allow people to develop different clients to match the way they want to browse it.

Could you recommend good p2p libraries to base my work on? They should be open-source, cross-platform, robust and easy to use. I don't care much about the language, I can adapt.

Mike Pennington
  • 41,899
  • 19
  • 136
  • 174
static_rtti
  • 53,760
  • 47
  • 136
  • 192
  • 1
    You should check out Distributed Hash Tables, there exist several implementations of production quality. Possibly you won't need much more...Also: I like your idea very much ;) – zerm Apr 10 '11 at 14:54
  • Thanks! I like the idea too, but it seems like a very hard task... If you're interested in helping me out, you can subscribe to reddit/r/decentralized. I'll try to recruit more people when the idea gets clearer. – static_rtti Apr 10 '11 at 17:30
  • 2
    I warn you that historically distributed solutions have gained very little traction, and also the market is saturated with reddit clones. So, you will have to try damn hard to promote it and sign up enough initial users to maintain momentum. Good luck! – Marcin Apr 13 '11 at 12:51
  • I should add that your users will probably not be using any kind of roman script, so whatever you do, make sure unicode works properly throughout; maybe also provide a feature for people to filter what they see and search by the characterset in use? – Marcin Apr 13 '11 at 13:00
  • Why do you think decentralized solution have had no traction? Is there something intrisically wrong about them, or are they simply harder to build than centralized solutions? – static_rtti Apr 13 '11 at 15:27
  • 1
    I found this question researching the same idea. The biggest problem you'll face is people trying to game the system. Being centralized, Reddit can easily ensure one user gets one vote per comment and post, and that moderators have control over subreddits. A de-centralized version, by definition, would have no one with the authority. That's not saying it's impossible, but it's an extra problem you'll have to consider. – Cerin Jan 01 '12 at 23:49
  • @Cerin: I agree! That's why I want to base my link selection system on personalized recommendations rather than democracy. The system will select links based on learning what the user likes and which users he trusts, not on vote counts. – static_rtti Jan 02 '12 at 21:02
  • I'm precisely working on this part of the system at the moment, developing it as a layer over reddit, with the idea of replacing reddit by a p2p system when the top layer is ready. – static_rtti Jan 02 '12 at 21:03
  • @static_rtti, Interesting idea. Focus on online learning algorithms. Vowpal Wabbit might be a good place to start. – Cerin Jan 03 '12 at 01:33
  • One possible solution to limit vote spam is to make it computationally expensive to vote. If it takes 1 second to make a vote instead of 1 millisecond that's 1000x less spam votes to worry about ;) – Martin Oct 25 '12 at 03:44
  • @Martin: that's not a bad idea. How woud you implement it? – static_rtti Oct 25 '12 at 13:04
  • HashCash http://en.wikipedia.org/wiki/Hashcash is exactly such a system for email, I'd implement it exactly like that. – Martin Oct 25 '12 at 13:46
  • this is a good read on decentralized social networks https://8bitmen.com/the-ultimate-guide-to-federated-architecture-decentralized-social-networks/ – underdog May 14 '19 at 07:29

3 Answers3

6

Disclaimer: warning, self-promotion here !!!

Have you considered JXTA's latest release? It is probably sufficient for what you want to do. Else, we are working on a new P2P framework called Chaupal, but it is not operational yet.

EDIT

There is also what I call the quick-and-dirty UDP solution (which is not so dirty after all, I should call it minimal).

  1. Just implement one server with a public address and start listening for UPD.
  2. Peers located behind NATs contact the server which can read how their private IP address has been translated into a public IP address from the received datagrams.
  3. You send that information back to the peer who can forward it to other peers. The server can also help exchanging this information between peers.
  4. Then peers can communicate directly (one-to-one) by sending datagrams to these translated addresses.

Simple, easy to implement, but does not cover for lost datagrams, replays, out-of-order etc... (i.e., the typical stuff that TCP solves for you at the IP stack level).

Jérôme Verstrynge
  • 57,710
  • 92
  • 283
  • 453
  • Thanks for your answer! I won't mark it as accepted immediately, because I want to offer a bounty for the question to get it as much attention as possible. On a side note, do you have any comments / suggestions as to how one might implement a decentralized, p2p-based, reddit-like system? – static_rtti Apr 12 '11 at 08:47
  • What you want to achieve is similar to FreeNet, so I would take a look at it for inspiration. – Jérôme Verstrynge Apr 12 '11 at 09:48
  • No, what I want to do is very different from freenet. Freenet is basically an anonymous version of the web. I don't care much about anonymity. My goal is to allow people to exchange links and comments on these links, in a decentralized fashion. – static_rtti Apr 12 '11 at 11:05
  • What I meant is that when it comes to the underlying architecture (not what the user sees or the functionalities offered by your application), you should separate the communication level from the application level, which is what Freenet achieves. You can skip the anonymous part. – Jérôme Verstrynge Apr 12 '11 at 12:48
  • 1
    Used JXTA myself and can recommend it :) – Jan Zyka Apr 19 '11 at 09:11
4

I haven't had a chance to use it, but Telehash seems to have been made for this kind of application. Peer2Peer apps have a particular challenge dealing with the restrictions of firewalls... since Telehash is based on UDP, it's well suited for hole-punching through firewalls.

EDIT for static_rtti's comment:

If code velocity is a requirement libjingle has a lot of effort going into it, but is primarily geared towards XMPP. You can port off parts of the ICE code and at least get hole-punching. See the libjingle architecture overview for details about their implementation.

Mike Pennington
  • 41,899
  • 19
  • 136
  • 174
1

Check out CouchDB. It's a decentralized web app platform that uses an HTTP API. People have used it to create "CouchApps" which are decentralized CouchDB-based applications that can spread in a viral nature to other CouchDB servers. All you need to know to write CouchApps is Javascript and learn the CouchDB API. You can read this free online book to learn more: http://guide.couchdb.org

The secret sauce to CouchDB is a Master-to-Master replication protocol that lets information spread like a virus. When I attended the first CouchConf, they demonstrated how efficient this is by throwing a "Couch Party" (which is where you have a room full of people replicating to the person next to them simulating an ad hoc network).

Also, all the code that makes a CouchApp work is public by default in special entities known as Design Documents.

P.S. I've been thinking of doing a similar project, but I don't have a lot of time to devote to it at the moment. GOD SPEED MY BOY!

pokstad
  • 3,411
  • 3
  • 30
  • 39
  • Thanks, I will definitely look into it. – static_rtti Oct 25 '12 at 13:05
  • 1
    No problem! If you have any questions, feel free to ask. I really think this is the ideal solution for a decentralized website. Also, there are mobile versions of CouchDB that can be embedded into Android & iOS apps so that you can actually take websites with you where ever you go. One last thing, I highly recommend this Google Tech Talk on "Ground Computing" with CouchDB: http://googlecode.blogspot.com/2009/09/chris-anderson-couchdb-relaxing-offline.html – pokstad Oct 25 '12 at 16:29