Have developed for both Social Engine 4.x and phpFox 2.x and 3.x. Use phpFox. The updates are more regular and more significant, it's resource usage is remarkably better, and I foresee it being around for longer than SocialEngine, thus, will be easier to obtain support in the future.
While Elgg is free, the $99 you will spend on a basic phpFox package will pay for itself 300 times over the first time you have to pay someone to create an addon that likely already exists for phpFox as it's third party developer community is much larger.
I promise, I don't work for phpFox. I have, however, tried to develop for and maintain a SE4 install and found it to be infuriating. It seems like it was written by three or four people that had little to no contact with each other and if you want sub 2 second page loads, you'll need to run it on a supercomputer at Los Alamos.