Having a strong background in Java/C++ i wonder if it is possible to develop a somewhat larger JavaScript application without having to cut back on quality.
Any hints are appreciated regarding:
I'm also curious how projects like JavaScript PC Emulator and JavaScript Game Engine handled those issues in case some of you know.
Development Environment Well, you need a web server (depends on server-side architecture) like Apache or IIS to simulate the AJAX communication. Sometimes an editor for javascript is included in the editor of the server-side development language.
There's a interesting question about javascript IDEs: https://stackoverflow.com/questions/209126/good-javascript-ide-with-jquery-support
Debugging Techniques & Profiling Use built-in browser debugging and profiling tools like Firebug.
You can also look at this profiling tool.
Unit Testing If jQuery is used I'd recommend http://docs.jquery.com/Qunit. In the development version of the javascrit application the javascript test files are loaded. When the application is published, the test files aren't loaded.
Security
Design
Application--------------------------------
Framework----------------------------------
The framework provides the base functions. For example a base framework is jQuery and knockoutjs. And on top of this framework the application is built. Of course you can create your own framework for your application. But when choosing jQuery for example, you mostly don't need to deal with cross-browser bugs, because jQuery makes that for you.
Communication with Server: It's a good idea to provide a RESTful Service for communicating. You also have to choose between JSON and XML. JSON is more lightweight than XML, so often JSON is elected.
Design Patterns: If the javascript application is really large, it's a good idea to implement design patterns like MVC or MVVM.
There are some MVC/MVVM frameworks outside for javascript (knockoutjs for example).
This is a really useful article about design patterns in javascript: http://www.addyosmani.com/resources/essentialjsdesignpatterns/book/
But at the end you have to decide yourself how your application should be structured etc. The Design Patterns can show you a good way - but every application is different and not every solution works for all problems.
And don't forget that performance is a big deal when using javascript. So compressing and combining javascript files is a good idea: http://code.google.com/intl/de/speed/articles/. At this point Lazy Loading might help, too.
I've participated in writing large JavaScript application with both SproutCore and Cappuccino and without any "Macroframework" at all. This is what I think:
First of all, all the same "good design" principles that you've acquired in your work with Java still apply: don't write Spaghetti code, encapsulate, separate concerns and use MVC.
Lots of people start writing a "web 2.0" or "web 3.0" (whatever that means) app, just add jQuery to it and go down a path of pain and misery, as their code becomes bigger and bigger and completely unmaintainable.
"Big" frameworks like Cappuccino or SproutCore help you to avoid doing that. Cappuccino is great for desktop-style apps while SproutCore has shifted focus in 2.0 to "web style" apps like New Twitter, while still giving you great ways to structure your apps and your code.
Now to your specific areas or interest:
I personally use MacVim. I've heard good things about Cloud9IDE, an in-browser IDE for JS development. For Cappuccino, you can use Apple's Xcode IDE, even to design the UIs (which is very cool).
Both Cappuccino and SproutCore apps can be tested right in the browser, without the need for a web server. If you do need to talk to a web server (which you probably will), Ruby on Rails or node.js are commonly used to easily get a backend up and running.
Debugging is still sort of a sore spot when it comes to JavaScript development. The Google Chrome developer tools are the best out there at the moment. You can set breakpoints right in the browser and all sorts of neat things. It's the tool you'll want to use.
SproutCore uses QUnit, Cappuccino comes with OJUnit/OJSpec. Then there are projects like JSDOM and env.js that let you simulate a browser on the server and give you an environment to run automated tests without a browser.
Also projects like Selenium or Jasmine are worth checking out.
You can do profiling with the Chrome Dev Tools. YSlow is great for general web application profiling (including asset loading and the like).
Use MVC from the get-go. Lots of people start with a small app and add some JavaScript here to read a value from a field and some more JavaScript there to update a label. They do that again. And again. And dinner is served: Spaghetti code. Use a framework like SproutCore or backbone.js to prevent that and to structure your code.
This is a great tutorial for SproutCore, this is one for backbone.js.
Same goes for Cappuccino, here a tutorial I wrote about a year ago. It's a little dated, but gives you the general idea. Here's an up-to-date demo app I did for an article I wrote for MacTech magazine in May.
So structure your code just as you would in any other development environment. Maybe read this book or this book, too. These videos are also a great resource.
For Cappuccino you can use Apple's Xcode Interface Builder to graphically lay out your UI. For most other systems you'll design your UI with standard HTML and CSS.
If you want to develop for mobile, you must read this article.
Refer to the books and videos I've mentioned above. And consult this general coding style. Naturally people disagree on some aspects of the style laid out on that page, but it's a good starting point.
JavaScript is an exciting environment to develop for and it has a very vibrant community! You should join the fun! I hope my opinions were helpful to you.
About the tools:
As far as IDEs are concerned, there is no single development environment that is proved more effective. The best people in the field only use the text editor they fancy most (@phoboslab, the man behind ImpactJS, uses KomodoEdit, for instance. Paul Irish uses TextMate, which is good, but Mac-only. A lot of people use Vim. Fabrice Bellard, the guy behind JSLinux, uses his own Emacs version, I think. This does not matter a single bit).
Unit testing is important, but that is never an issue. Javascript is powerful enough that you can build yourself a better-suited unit testing software in a couple of lines than anything out there. What does matter is that, thanks to node.js (the JS prompt that I recommend above), you can automate those tests by putting them in a *.js script file and launch all the testing with the single line: $ node test.js.
What really counts to be effective is to have the mdn javascript documentation under your pillow, and the html specification always open. Mind you, the version I point you to is not widely known, but it is by far the best out there! It even uses the cache manifest so that you can re-read the pages you have already downloaded, when you're offline! Let alone an outstanding search feature!
And now, since I really want that bounty, I'll give you one nifty page that lists all the documentation that you will ever need to build a web app. It really is a jewel. It contains a link to all information you need. That is the index of all bibles out there.
In the end, the question that really targets what you are wondering is, can you do a huge app in javascript?
The answer is yes. Javascript does have what Crockford calls "bad parts", but using JSLint warns you against them. On the other hand, Javascript has powerful weapons:
JSON.parse(aString) and a JSON.stringify(anObject) built-in object.This makes working efficient and easy. There are some specific patterns that you can use in Javascript. I'll let Paul Irish enlighten you.
One last advice, when using javascript in the browser: never use <script>/* some javascript here */</script>. Always use <script src="javascript-file.js"></script>.
And a couple more links.
You can check out Google's Closure library and compiler. They write some pretty large Javascript apps too.
You should look into alternatives to JavaScript that still allow you to deploy in a JavaScript like environment.
A lot of writing large apps is having a tool chain that make it easy for individuals and teams to get code working together without having to have everyone communicate with everyone else, an O(n**2) problem.
Things like GWT allow you to write in Java so if you have a team that is capable of coordinating development in Java or another statically typed application development language, they might find the transition to GWT much easier than to JavaScript. GWT also provides solid core libraries, templating, in-java-IDE debugging, minimized code per-browser, and a whole lot of other goodies.
GWT also works with java unit testing facilities, namely junit and many java teams are already experienced with integrating junit into continuous build systems and test dashboards.
Note that this post is more web focused, as that is where I primarily work
Development Environment
If you prefer IDEs, JetBrains has a really good web editor WebStorm that makes working in javascript a lot easier. Aptana is another good option. Otherwise text editors are always a great option (TextMate is my preference).
Debugging Techniques
Mentioned above, browser DOM tinkering tools like firebug are a must. You need to be able to declare and evaluate your code in the environment where it is being used without having to save and reload constantly.
Another point to mention here is jslint, which is strict validation of your javascript. This is equivalent to compiler time errors and can be invaluable when fixing issues in javascript.
Unit Testing
The best unit testing framework for javascript is jasmine (based off of rspec style tests). Some people do not like BDD style declarations, though most people that use TDD accept BDD is just TDD done right. Personally, I find the BDD style helps focus people towards what they are trying to test. It has extensive assertion support and a lot of work has been done to handle the asynchronous nature of javascript, without getting too dirty.
Profiling/Instrumentation
Most browsers have integrated development support. Both firefox and chrome are great here. Though the tooling doesn't provide great detail, they can easily be used to recognize that a slow point is in your server or in your client side code. With that information, deeper analysis would be needed, but it should be just work to track down and fix the issue.
This is definitely a space that is going to see a lot of growth outside of browsers with the growing popularity of nodejs.
Design (System/Code/Interface)
Any major javascript application is going to run into all of the issues that any dynamically typed language will. Furthermore, javascript also has a very long history in which a lot of bad code was and is written everyday. Javascript does nothing to push you into the pit of success, hence books like, "Javascript, the good parts".
On the other hand, javascript has great support for OOP, as well as functional programming (functions are first class citizens in the language). Object literals are a beautiful combination of objects in the class sense and dictionaries in the structure sense. On top of this, there are some truly unique features too the language and they can allow you to accomplish many powerful things.
With these considerations, you can apply any patterns you would use in other languages. There are numerous frameworks that take advantage of this. There is a great article that goes through many of the common frameworks for dependency management, code organization, templating, module communication, build process (including minification) and finally testing.
It's still possible to use Java like tools when developing with JS. The company that I work for has a 500K+ loc application, and we employ tools like Maven to deal with library dependencies. We also use Eclipse for JS development using the WTP plugin.
We use JS Test Driver for UnitTests and selenium for ATs. For degugging and profiling, we generally use Firebug and the IE debugger built into VS Web Express.
For code compilation, we use Google closure and a Java Servlet for serving JS during the development process, which is able to load the each JS file and it's dependencies in an order that ensures that class dependencies are available.
Majority of your questions has already been answered, however, I'm very surprised that nobody mentioned DynaTrace Ajax as profiling tool, since it's by far the best tool available on the market right now for in-browser JavaScript application profiling.
For debugging, you will have to rely on build-in browser debuggers (developer tools in Chrome/Safari, Developer toolbar in IE, Firebug in Firefox), but the most powerful debugging tool is still IE + Visual Studio, though not very comfortable at times.
