Depends a bit on what you mean with performance and where you see potential issues with it. There is the perfomance of the website in general where a video hosting site is not much different then any other site. There is the frontend performance, etc.
Where a video hosting site is different to most other sites is the ammount of data to be transfered and the performance for video rendering. Means probably the first wall to hit is the bandwith of ones webserver (if you would be starting with one standard machine) or the CPU going mad.
-> First step would be to seperate website from video delivery and encoding to be able to scale.
Now I don't know about ASP.NET but with Drupal there are many people who have done similar sites to youtube [1] and there are many contrib modules [2] to support also above mentioned tasks like Media Mover [3]. Might give you a starting point.
[1] http://www.google.com/search?&q=drupal+youtube+clone
[2] http://drupal.org/project/modules?filters=tid%3A67
[3] http://drupal.org/project/media_mover