On Fri, Aug 12, 2011 at 5:49 AM, Nick Knutov <mail@knutov.com> wrote:
I agree, forking is another way to do it, but as far as I know, it can lead to very high load. Also, dying process is a heavy operation, as far as I remember.

I don't fully understand the role of server load here.

On the one hand you want to perform those heavy operations inside your Dancer process after the page has been delivered. I would say that this is reasonable if the load on your server is very low, because you're asking your front-end infrastructure to take care of back-end duties as well.

On the other hand you're very concerned with load. If this is the case, are you sure that blocking your front-end process to carry out the back-end duties is the right way to go? You might shave off the fork penalties (but you have to do your benchmarking homework here, and probably with a reasonable copy-on-write handling you don't suffer that much), but this would be nothing compared to how stuck your front-ends would become.

If the load is your concern, you should decouple the frontend from the backend as much as possible. I'd suggest to serialize the relevant data in some way and push to a queue that is dealt with by some other process, which might even run on another machine if the load on a single server gets too high. If you need more, with proper sharding you could also reach full horizontal scalability.

Cheers,

    Flavio.