0

We build a workflow-like application.

use case 1: The user visually builds some workflow and then runs it to see results. Then the user fix the WF and runs again, and again, until he is happy.

use case 2: Once the WF is done the user schedules it to run a few times a day (possibly many times)

My friend says: When the user saves his WF let's first save its model to DB (so we can open later) and then generate c# code for the run time from this model.

  • "This is the only way to get good performance at run time, especially when use case #2 implies many runs a day"

I say: Let's just save to DB. Then we will build a generic runtime that runs over the DB rows and executes the model commands (interperter-like).

  • This gives much better perceived performance for #1 since waiting for compilation after every fix is frustrating
  • this will not have such a big affect on the runtime itself if done correctly

What is your take?

Yaron Naveh
  • 23,560
  • 32
  • 103
  • 158
  • What do you mean by "we will build a generic runtime"? – Adrian K Aug 12 '11 at 00:18
  • The rt will run over the model from DB no matter what order are the elements in the current wf. For each model element we probably have some compiled code that handled it. So the generic rt needs to implement flow control logic like loops, conditions etc. once it finds these on db – Yaron Naveh Aug 12 '11 at 00:53

3 Answers3

3

You say the db contains flow control elements like loops and conditionals. That says to me what you are storing in the db is at least a simple procedural scripting language.

When that happens, there will be pressure for enhancements making it more like a "real" language. Something like subroutines will be wanted, with parameters, and variables and expressions.

You can jump ahead on this process by having an actual language, rather than a set of rows in a database, and save it as a text file.

Then a good way to "interpret" it is to actually generate C# code on the fly, and compile/run it on the fly. That can actually happen very quickly.

The reason for doing this is a) no need to write the interpreter, and b) leap-frog the future enhancements.

Mike Dunlavey
  • 40,059
  • 14
  • 91
  • 135
  • but then reading the model (in design time, for the purpose of editing) becomes parsing code, which is harder than reading from db, especially if there will be extensions. – Yaron Naveh Aug 12 '11 at 01:28
  • @Yaron - I'm not sure I can see an impact due to where the code comes from; it's taking the instructions and making them runable which is the most difficult. – Adrian K Aug 12 '11 at 03:09
  • @Yaron: Make it a really simple language, and learn how to write a simple recursive-descent parser. It's easy. You can find out how on SO, and it's a very useful skill. I've seen this a number of times, where people try to use a db when what they need is a language, because they think parsing is hard. – Mike Dunlavey Aug 12 '11 at 13:31
  • @Adrian: What I've done in other languages is simply print the program I want out to a file, then run a batch file to compile and link it into a dll, then load the dll and run it. This all takes about a second. You must be able to do this in C# as well. – Mike Dunlavey Aug 12 '11 at 13:36
  • @Mike: So you suggest to have the language as my model instead of db? As for compilation I have did a simple poc and the generation of the c# code + compilation takes a few seconds. This is a long code. I would hate my user to wait this time after every change. This is why I was thinking to interpert a db, so no need for generate+compile. I could also interpert the intermediate language you suggest me to build so the question is not db vs. new language, but interpert vs. generate+compile. – Yaron Naveh Aug 12 '11 at 22:35
  • @Yaron: Well, I'm telling what's worked for me on several projects over the years. If some particular approach seems to have a performance problem, that might be easy to fix, [as this example shows](http://stackoverflow.com/questions/926266/performance-optimization-strategies-of-last-resort/927773#927773). I would be pretty bummed out if I made a system design decision based on a perceived performance problem that in hindsight was actually easy to fix. Good luck. – Mike Dunlavey Aug 13 '11 at 01:09
  • @Yaron: An alternative is to have an interpreted byte-code language handy. (In fact, that's what Java and C# are.) The parsing and byte-code generation should be lightning fast, especially since you don't need optimization, and the interpretation will be no slower than any interpreter you could write. Just a thought. (But be especially wary not to depend on a 3rd-party tool that could get ripped out from under you in a few years' time. Boy, is that a headache.) – Mike Dunlavey Aug 13 '11 at 01:16
  • @Mike: Yes you can compile on the fly; my question is why would you want to (in this case). – Adrian K Aug 13 '11 at 10:25
  • @Adrian: because you don't have to write an interpreter. You get all the power you want, almost for free. – Mike Dunlavey Aug 14 '11 at 15:19
  • @Mike: True - but what I mean is why not compile at design time rather than runtime? I guess your point is that the benefits in this case potentially outweigh the disadvantages. I just would have thought that a supervised compile at design time would be 'better' as you'd catch potential issues sooner. – Adrian K Aug 15 '11 at 00:06
  • @Adrian: My viewpoint is that data structures range on a spectrum according to the memory needs of the programs that interpret them, from simple data, to stack, to arbitrary heap, corresponding the the hierarchy of automata - finite state, pushdown, turing. So when the OP says his data structure contains loops, etc. that tells me his data structure is tending toward universal, so if he interprets it he is tending toward writing a universal interpreter. There's more than one way to do that, inclusing using an existing language... – Mike Dunlavey Aug 15 '11 at 00:39
  • @Adrain: (Sorry for the length.) ... I don't view it so much as catching issues sooner as avoiding writing an interpreter, because every interpreter is pulled along by late-discovered requirements that make it more universal - loops, conditionals, subroutines, variables, etc. One way to interpret data is to compile it into an existing language that already has an interpreter/compiler. If users can edit the data at run time, that implies the interpretation/compilation be done at run time, so there we go. (You might notice this is one of my favorite techniques.) – Mike Dunlavey Aug 15 '11 at 00:45
0

I think a combination would be ideal:

case #1:
As long the user doesn't schedule the WF use your approach (#1)...

case #2:
when the user schedules a WF then compile (perhaps even asynchronisely, just needs to be ready for the first scheduled run).

Yahia
  • 69,653
  • 9
  • 115
  • 144
  • both alternatives are huge development effort in our case. I need to choose one... – Yaron Naveh Aug 12 '11 at 00:24
  • what sort of WF are we talking about ? how often does run when scheduled ? – Yahia Aug 12 '11 at 00:25
  • think of anything you can do with MS workflow. I wish I knew how many times it will run, this is a question only time will tell. We want our architecture to be ready. – Yaron Naveh Aug 12 '11 at 00:33
0

My take is that whilst waiting to recompile the workflow every time you modify it will get tedious, I'd rather have the performance benefits when it runs "for real"; so I guess I agree with your friend.

This fits with the current .Net development approach where source-code is complied into byte-code. Once you "think" you've finished with the source code then you might as well compile it, right? Plus you'll get all the benefits of a compile time checks before you decide to deploy.

Sure the JIT does the final compile into machine code - but that steps usually not as long as the first part (source code to byte-code).

Adrian K
  • 9,880
  • 3
  • 33
  • 59