[Dancer-users] deploying many, many apps

[Puneet Kishor]

On Sep 14, 2011, at 9:20 AM, Brian E. Lozier wrote:

> On Wed, Sep 14, 2011 at 6:11 AM, Puneet Kishor <punk.kish at gmail.com> wrote:
> 
>> I have asked, or hinted at, other variants of the same question, but here
>> again --
>> 
>> I am now developing and deploying many Dancer apps. Have been experimenting
>> with, and am quite happy with Starman serving them hiding behind Apache2
>> proxies. My home grown, amateurish web_ctl [
>> https://github.com/punkish/web_ctl] helps me manage them as well. However,
>> with every app, I am starting an additional 10 Starman processes, so now
>> already I have about 60 Starman processes running on my computer.
>> 
>> 1. Why 10? I once tried 2 Starman processes for a very, very low traffic
>> app, and found it coughing and sputtering, but surely, 10 is too many, isn't
>> it?
>> 
> 
> Well, is starman also serving all your static content?  One page view can
> have dozens or hundreds of server requests due to images, css, etc.  You
> could use a tool like abench or something to test for requests per second
> and concurrent requests per second.


Brian, I have read the above sentiment on many of these Dancer emails, and I just don't get how I can separate my static content from my dynamic content. As you said, a web site is a complex beast, made up of both dynamic and static parts. Just keeping the damn thing straight in my head is job in itself. If a web site were completely static, I could have Apache serve it. But, with a mixed set, there are only a few things that I know are static. Here is my set up. Please review and comment on it. I would love the feedback --


On my development machine:

~/Sites/lib
	- jquery
	- html5
	- OpenLayers
	- other static JavaScript and CSS libraries

The above are served by Apache2 at http://myserver/<folder_name>, so, http://myserver/jquery, http://myserver/html5, etc.

~/Documents/www
	- app1
	- app2
	- app3

The above are served by Starman at http://127.0.0.1:<port>/, where the different port points to different apps

All the content for each app is within its directory, all its static content appropriately under /public except for the static content being served by Apache2 above.

So, as you see from above, static content that can be served whole via Apache2 are mainly the JavaScript libraries common to all applications. Application-specific static content comes via Starman.

I now have another little wrinkle thrown in -- I've just got a new machine, and that is becoming both my development machine as well as my test server. So, I have to figure out a way to isolate the two sides, but that is a different problem.


> 
>> 
>> 2. Why not Apache2 with mod_perl or fastcgi or mod_psgi? Apache seems to
>> launch 5-10 "children," but no matter how many apps are being served by it,
>> that number of child processes seems to hover around that number. It doesn't
>> get to n * 10 for Starman?
>> 
> 
> The numbers are completely configurable on apache.  You have to look at
> StartServers and Min/Max SpareServers or MinSpareThreads/MaxSpareThreads
> depending on which type of apache you're running.  Apache will try to deal
> with having a reasonable amount running depending on load but it's always
> bound by the parameters.  If your load gets to be more than the 10 children
> can handle then requests will start timing out and things will get sluggish.


Perhaps so, but I have never had to mess with StartServers/SpareServers/SpareThreads settings. It just works, and quite admirably.


> 
> 
>> 
>> The Starman approach does have one clear advantage -- each app can be
>> turned off or on separately without having to restart Apache entirely.
>> 
>> I want to get the crowd-wisdom on serving, say 50 different Dancer apps on
>> the same computer. The apps will range from very, very low traffic to
>> moderate traffic. However, all the apps will be data-intensive, in that,
>> even if the number of user-hits on the server are few per hour, each hit
>> does a fair bit of database work and sends back largish amounts of data.
>> Data may be on the same or a different server -- that part is out of the
>> scope for this thread.
>> 
>> 
> I wouldn't have a problem having 500 mostly idle starman processes hanging
> out, but I haven't run in this situation under high load so I don't know
> what that would look like.  The advantage, being able to install and
> maintain the apps separately, is a big one.  Where I work now we have 20
> different apps all sharing the same apache and it's a nightmare.


Yeah, I agree with the second part of the above statement. Being able to start and stop apps separately is cool. But the former part is a bit bothersome, if nothing than from an aesthetic point of view. Ever been to a restaurant or a department store with way too many servers/sales people than customers? It looks odd.

Perhaps, one area I should explore is "why 10"? As I said, I tried "2" once, and I started getting errors. The app would not return any content or an error (I forget the exact error message), and then I would hit refresh and the content would be delivered. I bumped the number of processes to 10 and things were ok again.


> 
> 
>> Many thanks in advance.
>> 
>> Puneet.
>> _______________________________________________
>> Dancer-users mailing list
>> Dancer-users at perldancer.org
>> http://www.backup-manager.org/cgi-bin/listinfo/dancer-users
>>