November 2015 - dancer-users - lists.preshweb.co.uk

Having multiple INCLUDE_PATH for the Template Toolkit engine
by Gabor Szabo 13 Nov '15

13 Nov '15

HI, The Perl Maven and the Code Maven sites run using the same code-base and thus the same set of templates in the views directory. https://github.com/szabgab/Perl-Maven I'd like to have some additional templates that differ in the two sites, but I don't want to have two separate sets of templates. So I wanted to add multiple views directories which worked in the config file like this engines: template: template_toolkit: encoding: 'utf8' INCLUDE_PATH: - 'views' - 'config/templates' start_tag: '<%' end_tag: '%>' but I'd like to set this during the run-time, for example in the 'before' hook. So far I could not do it. I tried set views => [dir1, dir1] but does not work as views is required to be a string. I could not figure out how else to change the INCLUDE_PATH during run-time. Is there a solution? If currently not,would it be possible to allow views to be either a string or an array ref to strings? regards Gabor

2 3

Dancer history lesson:)
by Andrew Solomon 11 Nov '15

11 Nov '15

Enjoy! http://blog.geekuni.com/2015/11/interview-alexis-sukrieh-choreographer.html -- Andrew Solomon Mentor@Geekuni http://geekuni.com/ http://www.linkedin.com/in/asolomon

2 1

Advent 2015, medium-scale Dancer, part 6: Hot Code Reloading
by Warren Young 10 Nov '15

10 Nov '15

I intended to stop the series with 5 parts, but I just thought of another important topic that fits into the series: hot code reloading. Hopefully, I’m done now. I’ll be accepting comments for another few days, though, if someone wants to see some changes in these articles before they’re published. Sawyer, to publish these, do I need to modify a fork of the Advent web site code, or will someone from the developer core publish them for me? If the latter, I’ve made significant refinements to all of the articles since posting each to the list for comment. ------------------- Many other web frameworks have a hot code reloading feature: simply by changing a source file on disk, the new code in that file runs for each subsequent web hit that calls that code. This makes development quicker by reducing the reload time in the edit/reload/test/debug cycle. Dancer does not include this feature out of the box, but the power of Perl makes it fairly easy to add to your app. There are, in fact, numerous CPAN modules to do this already, such as `Module::Reload`, but it turns out that rolling your own can provide some serious benefits. There are details you want to take care of manually during the reloading process which rule out the canned solutions. Consider this Dancer-aware reloader based on Hack 30 in [Perl Hacks](http://shop.oreilly.com/product/9780596526740.do): package App::Reloader; use Dancer2 appname => 'App'; my $g_last_reload_check_time = 0; my %g_module_times; INIT { while (my ($module, $path) = each %INC) { $g_module_times{$module} = -M $path; } } sub reload { my ($min_delta) = @_; my $now = time; return if ($now - $g_last_reload_check_time) < $min_delta; $g_last_reload_check_time = $now; while (my ($module, $time) = each %g_module_times) { my $mt = -M $INC{$module}; next if $mt == $time; next if $module eq 'App/Const.pm'; # can't redefine constants next if $module eq 'App/Reloader.pm'; # self-reload breaks things delete $INC{$module}; eval { # Suppress complaints about sub redefinition no warnings 'redefine'; local $SIG{__WARN__} = sub {}; # Reload the module, unloaded above require $module; debug "Reloaded changed module $module."; # Only mark it updated if that didn't throw an exception. $g_module_times{$module} = $mt; }; error "Reloading changed module $module: $@!" if $@; } } 1; You also need to merge the following into the module containing your Dancer hooks, `lib/App.pm` by default: use App::Reloader; use Readonly; Readonly::Scalar my $min_reload_check_time => (config->{environment} eq 'development' ? 1 : 15); hook before => sub { # do checks that cause an immediate client rejection first App::Reloader::reload($min_reload_check_time); # now we can do other stuff that requires up-to-date modules }; This mechanism gives us a number of features you don't get from the canned solutions: **Logging** We can use Dancer's logging mechanism to report what the reloader is doing. Third-party loaders are typically either silent, which makes debugging more difficult, or they use the `STDOUT` or `STDERR` streams. Using `STDOUT` with Dancer is a serious problem, since that ends up injecting the message into the rendered route output, which typically breaks its client-side handling. Using `STDERR` isn't much better, since your Dancer app typically runs in the background; unless you redirect the app's `STDERR` to a second log file, you won't see these complaints. **Performance** Our custom reloader needn't check for reloadable modules on every web hit. Chances are, the code we loaded from the modules on disk for the last hit are still good. The example above checks at most once a second in development mode, and once every 15 seconds in production. You can adjust the timing to suit your purposes. `Module::Reload` exposes a `check()` function similar to `reload()` above, so you could wrap it in your own timing logic, but I've moved that internal to the reloader, so you only need to pass in the time constant. Other reloaders add a constant burden to the system, so they're recommended only for development. While reloaders are exceedingly useful during development, it's also nice to be able to do an app upgrade in production without restarting the app. We therefore want a reloader that we aren't reluctant to run full-time. **Easy Exclusions** In the first article in this series, I proposed that you might want an `App::Const` module for holding app-wide constants. If you use the CPAN module `Readonly`, demonstrated above, you will find that you get an error from Perl about redefining constants if you let `App::Reloader` reload that file. We need to make the reloader aware of this. Generic reloaders typically won't cope with cases like this, or they expose a somewhat clumsy API for customizing the set of modules it monitors. It's simpler to just add an exclusion rule to this file when you come across a module that can't be reloaded for one reason or another. This is a good reason for having an app-wide "constants" module, by the way: by aggregating all of the constants into a single location, we cause only that one module to be un-reloadble. If you scatter `Readonly` calls throughout your other modules instead, they'll be un-reloadable, too.

1 0

Dancer2::Plugin::Database side effects?
by Mike Cu 07 Nov '15

07 Nov '15

I've replaced all my $dbh = DBI->connect, $sth = $dbh->prepare calls with the Dancer2::Plugin::Database equivalents Before that, when there was interaction with the UI, I would explicitly connect, run all my sql and then disconnect, on every Ajax call . After I switched to Dancer2::Plugin::Database,I have to do the same interaction, for example, clicking a button which invokes an Ajax call which in turn calls the database retrieving the new data and updates the UI, does not have an immediate effect ,the UI is not being updated. So I have to do the same move (click a second time) for getting the desirable outcome Has that something to do with caching and reusing the connection?

4 6

Advent 2015, medium-scale Dancer, part 5: REST APIs and JSON
by Warren Young 07 Nov '15

07 Nov '15

In previous parts of this article series, I defined each route handler as a one-liner, like so: prefix '/mf' => sub { get '/' => sub { retrieve(context); }; prefix '/subfeature' => sub { get '/' => sub { _get(context); }; post '/' => sub { _post(context); }; put '/' => sub { _put(context); }; del '/:id' => sub { _del(context); }; }; }; This is very much on purpose, despite the fact that Dancer lets you define all of the code for each route handler within the `sub { ... }` block. Why add the extra function call layer? Simple: I want to make the URL structure of my web app explicit in the code. To some extent, you can see this by looking at your `views` directory, since a prior article in this series encouraged you to map view files 1:1 to Dancer routes, as much as possible. The problem with relying on the `views` directory contents as a document for the web app's URL structure is that the OS's file system normally shows you only one layer of the directory hierarchy at a time. You have to go out of your way to see the whole tree in a compact form, on a single screen. That means you may never actually find yourself studying the hierarchy as a whole, and thus may never consider whether it is a cohesive design. This practice of defining Dancer route handlers as one-liners solves that. It ensures that every time I adjust the route handlers for my site, I am doing so while looking at the neighboring routes; I define new routes in context, not independently. This counteracts the entropic forces that result in API design drift, because you are implicitly encouraged to define new routes that fit logically into the scheme you defined previously. Without this extra Perl function call layer, the route definitions are visually broken up, so that you cannot see the whole design on a single screen. The resulting URL scheme can turn into a mess, as developers hack on it over the course of years. This is also why I have carefully lined up the blocks and [fat commas](https://en.wikipedia.org/wiki/Fat_comma) in the route definitions: I want patterns in the URL scheme to jump out at me. Left-justifying everything obscures these patterns. Now you may be asking, why does it matter that the URL scheme be "clean?" Simple: laying out your route handlers like this makes patterns graphically apparent. Just by looking at the code, you may see patterns that you never would have noticed when the code is arranged as in the first part of this article series. Some of those patterns may point out some previously-hidden practical value, in the same way that graphing a numeric data set sometimes does. It turns out that the example route scheme we've been defining throughout this article series includes a web API; it's just been sitting there, waiting to be discovered. With a bit of polishing, it might even turn into a proper [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) API. Let's try. Look back up at that route definition block above. What does it remind you of? If you said [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete), go to the head of the class. Now, HTTP verbs do not correspond directly to CRUD, because HTTP was not designed to be a CRUD API. There are competing design philosophies when it comes to mapping HTTP verbs to the CRUD model of persistent data storage. I prefer a simple 1:1 mapping: | DB term | HTTP verb |------------|---------- | C = create | `POST` | R = read | `GET` | U = udpate | `PUT` | D = delete | `DELETE` The R = `GET` and D = `DELETE` rules are obvious, but it may not be clear to you why the other two rules are correct. [The HTTP spec](http://www.w3.org/Protocols/rfc2616/rfc2616.html) defines `POST` and `PUT` so that they're nearly interchangeable, which is how we ended up with competing REST/CRUD API design philosophies. Some API designers like to use `POST` for both create and update operations, since you can distinguish the cases based on whether the HTTP request contains a database ID: if so, it's an update for an existing record, else it's a create operation. I don't like doing this in Dancer code because it means you need to define two different operations in a single route handler. This design choice makes more sense in a web framework that defines the URL scheme in the file system, such as PHP, ASP, or JSP, since it is normal to handle all four verbs in a single file in that sort of web framework. Although the HTTP spec defines `PUT` and `POST` as partially interchangeable, you must not swap the `PUT` and `POST` verbs from the way I've defined them above. The HTTP spec says that [`PUT` is idempotent](http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1), meaning that the browser is allowed to cache `PUT` requests, suppressing subsequent identical calls if the request parameters do not change. In CRUD terms, this means you cannot create two records with identical contents, but different IDs, which means you cannot define create as `PUT`. Browsers are not allowed to treat identical `POST` calls as idempotent, so we use that verb for CRUD's create operation, leaving `PUT` = update. That's enough design philosophy. How does all that affect our Dancer code? We might decide that since we've almost got a REST API here, that we might want to push it the rest of the way. We might move some of the code from `lib/App/MajorFeature.pm` to `lib/App/API/SubFeature.pm`: prefix '/api' => sub { prefix '/subfeature' => sub { post '/' => sub { _post(context); }; get '/' => sub { _get(context); }; put '/:id' => sub { _put(context); }; del '/:id' => sub { _del(context); }; }; }; My choice to use leading underscores in these module-internal route handler function names lets us parallel the Dancer DSL keyword naming scheme. You might instead prefer to make the HTTP-to-CRUD mapping more explicit: prefix '/api' => sub { prefix '/subfeature' => sub { post '/' => sub { _create(context); }; get '/' => sub { _read(context); }; put '/:id' => sub { _update(context); }; del '/:id' => sub { _delete(context); }; }; }; As in the previous article in this series, the leading underscore convention saves us some aggravation here, since `delete` is a Perl keyword, and `read` is a core Perl function. The only other thing to change here is that REST APIs normally return JSON or XML. I prefer JSON, since that's directly usable by the JavaScript code that made the Ajax call to one of these APIs. One of the best features of Dancer is that automatic JSON encoding is built right in. We might define the `_create()` function referenced above as: sub _create { my ($ctx) = @_; # see Part 2 for the definition of context() my $params = $ctx->{request}->params; my $db = $ctx->{dbconn}; if ($db->create_something($params->{arg1}, $params->{arg2})) { return { success => JSON::true, id => $db->last_insert_id(), }; } else { return { success => JSON::false, error => 'failed to create record: ' . $db->last_error(), }; } } In Dancer, a hash reference returned from a route handler is automatically JSON-encoded, and the HTTP response's content type is set to `application/json`. Parsing and using that reply on the client side in JavaScript is therefore trivial. If your new REST API route handlers previously returned HTML, and you have existing code that still needs this data in HTML form, I prefer to separate those Ajax call handlers into a different section of the URL hierarchy: `/parts`. The rule is simple: `/api` routes return JSON, `/parts` routes return HTML fragments. Those fragments are intended to be inserted into an existing DOM via JavaScript. The `/parts` section of the URL hierarchy separates out such route handlers from those that return whole HTML pages. This scheme gives you a utility function that returns a hashref to other parts of your Dancer app so they can build HTML from it, but returns JSON when called from a Dancer route handler. **Conclusion** Well, that's it, the end of this article series. We have gone from a monolithic app structure as generated by `dancer2 -a App` to a nicely-factored application that's ready to grow into the thousands of lines of code. Along the way, we discovered that we had a REST API sitting there in our app's URL structure all along, and matched up the rest of the URLs to resources in a way that will make debugging simpler by following some sensible naming conventions. I hope this reduces some of the growing pains in your own Dancer applications!

1 0

Advent 2015, medium-scale Dancer, part 4: Client-Side Code
by Warren Young 06 Nov '15

06 Nov '15

This will be a short article, because the way you write your client-side JavaScript code has almost nothing to do with Dancer. Nevertheless, there are a few things we can say here that tie into the prior articles in this series. **Consolidate JavaScript Files** Because each HTTP round-trip over the Internet takes human-scale time — tens of milliseconds — an important part of making a fast web site is to consolidate your JavaScript code into as few files as possible. Ideally, you'll end up with just one JavaScript file, perhaps called `public/js/app.js`. This seems to go against the whole thrust of this article series so far, which has been about factoring monolithic structures into smaller, purpose-focused ones. But, there's a trick that lets you have your cake and eat it, too: tools like [RequireJS](http://requirejs.org/) allow you to "compile" multiple JavaScript files down to a single file. Coupled with [minification](https://en.wikipedia.org/wiki/Minification_(programming), [HTTP compression](https://en.wikipedia.org/wiki/HTTP_compression), and aggressive [file caching](https://en.wikipedia.org/wiki/Web_cache) rules in your front-end web server, your app will feel a lot snappier to your users, even though the size of the web app on disk is no smaller than before. You might not choose to compile all of your JavaScript down to a single file. If several files makes more sense for your purposes, I suggest that you name those files after the top-level URL that uses it. For example, the `/mf` URLs we've been using in examples so far should be served by a `public/js/mf.js` file. You might still wish to develop `mf.js` in several different purpose-driven JS files, compiled down to `mf.js` by RequireJS or a similar tool. **Serve Static Assets Via a Front-End Reverse Proxy Server** The [Dancer2 Deployment Guide](https://metacpan.org/pod/distribution/Dancer2/lib/Dancer2/Manual/Dep… talks about running your app behind a front-end reverse proxy server, such as nginx or Apache's `mod_proxy`. I strongly encourage you to do this, even in development. The server setup given in the deployment guide will cause the front-end proxy server to handle all URL for static content files itself, passing URLs down to Dancer only when it cannot find a static asset with the requested name. This takes a huge load off of Dancer, which, you must remember, is dynamic Perl code, so it will naturally be slower than the highly optimized C code in the proxy server. You might want to temporarily add something like this to your Dancer app to make sure the front-end proxy is doing what it ought to: hook before => sub { my $path = request->path; my $static = $path =~ m{\.html$|^/(bitmaps|css|fonts|js|svg)/}; warning "Static URL got past proxy server: $path\n" if $static; }; Adjust the `$static` regex to match your site's URL scheme, then reload the web app, and work through all of the pages. In normal operation, you should never get this warning. Take this code out for production, though, because it can be triggered by visiting bogus URLs such as `/bitmaps/does-not-exist.png`. There are bots on the web that pound on web sites with random URLs, hoping to find a vulnerability in the code. You don't want one of them to fill your disk up with log noise. Running a typical web app this way might reduce the number of hits that get down to the Dancer level by a factor of 10 to 100, since so much of a modern web app is composed of static asset files: bitmaps, fonts, CSS files, JavaScript files, etc. In the next part of this article series, we will discover the REST API that was hidden inside this monolithic web app the whole time.

1 0

Advent 2015, medium-scale Dancer, part 3: Views
by Warren Young 06 Nov '15

06 Nov '15

The prior parts of this series are here: http://lists.preshweb.co.uk/pipermail/dancer-users/2015-October/005237.html http://lists.preshweb.co.uk/pipermail/dancer-users/2015-October/005238.html The final two parts will be coming shortly, after I finish my editing pass. ====================== The first two parts of this article series broke the monolithic `lib/App.pm` file up into a series of Perl modules, each focused on some functional centroid of your application. If we describe the default `lib/App.pm` file as monolithic, then we might get away with describing the view files as polylithic, because we do not need to break the view files up; they naturally map to web pages, so the app's URL scheme effectively keeps them separated by purpose. Although we do not start this part of the series needing to break up an overly large file, I can pass along some tips along that might cause you to rethink the current structure of your app's `views/*` tree. **The View Naming Scheme Should Match the URL Scheme** Older web frameworks like PHP, ASP, and JSP define the web app's URL structure in the file system. The route scheme defined in the previous part of this article series would look like this in PHP: index.php mf/index.php mf/subfeature/index.php The view code — HTML and inline JavaScript — is intermixed with the server-side code in the same file. Worse, single files like `mf/subfeature.index.asp` have to handle multiple HTTP verbs; in our example, `GET`, `POST`, `PUT`, and `DELETE`. If each verb returns a different web page, you end up with a huge top-level `if/elseif/else` blocks dividing the file up into a bunch of logically separate chunks. There are other bad consequences to this web framework design choice, such as that if you want your text editor to do syntax coloring correctly, it needs to be able to switch among PHP, JavaScript, and HTML syntax, all within the same file. The programmer ends up needing to do the same sort of context switching, too. Simply put, this is a mess. [Template::Toolkit](http://www.template-toolkit.org/) — Dancer's default template system — solves part of this problem by separating the view files from the server-side Perl code. Modern web app development frameworks like [Angular](https://angularjs.org/) solve the rest of it by encouraging you to move all of your JavaScript code into separate `*.js` files, rather than put it inline in the HTML file. There is nothing about the way Dancer works that enforces or even encourages any particular `views/*` file naming scheme. Thus this tip: for your own sanity, I recommend that you name your view files after the corresponding route. The Dancer scheme corresponding to the above PHP scheme might look something like this: views/top.tt views/mf/top.tt views/mf/subfeature/get.tt views/mf/subfeature/post.tt views/mf/subfeature/put.tt views/mf/subfeature/delete.tt This maps Dancer routes 1:1 to view files, using a naming scheme that is easy to remember, since it's nearly identical to the URL scheme. It's so simple a mapping that you could write a Perl one-liner to do the transform. By restricting yourself to a simple naming convention, you have removed a whole class of details that you have to keep in mind while working on your web app; you don't have to guess where the view file for a given route handler lives, you already *know* where it lives. You will quickly rediscover this simple mapping years from now when you have to work on the web app again. This naming scheme is very much a suggestion, rather than a prescription. For one thing, you may find that the `DELETE /mf/subfeature` route needs no view, because it just deletes one database record and then redirects you to `GET /mf/subfeature` to show the change in the normal view. In that case, you would not need `views/mf/subfeature/delete.tt`. The important thing is to *have* a well-thought-out naming scheme, not necessarily to follow the one I've laid out above. **Factor Common UI Elements Out** If you reuse a particular UI element across different parts of your web app, put its view file into a special view directory. I use `views/parts` for this. If you have a web page that is incorporated into your web app dynamically on the client side via Ajax, you should have a corresponding Dancer route for it, and probably a Perl module for it, too: views/parts/component.tt lib/App/Parts/Component.pm Then in `lib/App/Parts/Component.pm`: sub _get { # ...work out how to build the component return Dancer::template '/parts/component' => { stuff => 'goes here' }; } prefix '/parts' => sub { get 'component' => sub { _get(); }; }; I chose the naming scheme for the function carefully. I want to be able to use short names here that correspond to the HTTP verb that the function handles without colliding with the short names exported by Dancer's DSL. The [standard Perl style rules](http://perldoc.perl.org/perlstyle.html) say that a leading underscore marks a function that isn't meant to be called from outside the module, which is exactly correct here; only Dancer should be calling that function. You may prefer a different scheme, such as `Get()`, even though this is unconventional Perl style. As I've said in previous parts of this series, the important thing is to *have* a consistent style, not that you use the style I've chosen here. One argument in favor of `Get()` over `_get()` is that you might need to be able to reuse this route handler in another of your Perl modules, bodily incorporating the HTML fragment within another web page on the server side, thus saving an Ajax round trip the first time that that web page is served. For example, `lib/App/MajorFeature.pm` might have something like this in it: sub Get { my ($ctx) = @_; # see Part 2 for the definition of context() # Work out how to build the main part of the top-level /mf page ...Perl code here... # Assemble the pieces return Dancer::template '/mf/top' => { component => App::Parts::Component::Get(), other_stuff => 'goes here', }; } prefix '/mf' => sub { get '/' => sub { Get(context); }; # other route handlers here }; Then within `views/mf/top.tt`, you can reference `$component` to bodily include that component's HTML fragment into the larger page. Now that we have sorted out the purely server-side HTML and Perl code files, we will look at the Dancer app design decisions that affect its presentation to the browser.

1 0

Re: [dancer-users] Template tags in skel and app
by Rick Leir 05 Nov '15

05 Nov '15

On Thu, Nov 5, 2015 at 7:00 AM, <dancer-users-request(a)dancer.pm> wrote: > > skel/ contains *our* templates for generating a new application. We use [% > %] in our templates when we generate a new application. If we used all [% > %] or <% %>, the tags for the sample app would get processed out when we > generated a new application. So by having two sets of tags, we can use one > set generating the app, and leaving the second set intact in the skeleton > application. > > Good question :) Does my answer make sense? > > Jason Yes, and your answer might be 'authoritative' when some new user is trying to figure this out and Google leads to here! In addition, here is the config for layouts: the WRAPPER path. template: "template_toolkit" charset: UTF-8 engines: template: template_toolkit: ENCODING: utf8 INCLUDE_PATH: 'views' start_tag: '<%' end_tag: '%>' WRAPPER: layouts/emails.tt Then in app code, you can refer to a template using a layout like this: my $email_body = template( 'emails/activate.tt', { real_name => $fields{ name }, recipient => $fields{ name }, }); File locations: views/emails/activate.tt views/layouts/emails.tt Cheers -- Rick

1 0

Re: [dancer-users] kernel (off-topic)
by Rick Leir 05 Nov '15

05 Nov '15

On Thu, Nov 5, 2015 at 7:00 AM, <dancer-users-request(a)dancer.pm> wrote: > > So I would assume that it is still valid even for 4.3. > Are you aware of any significant changes in this field ? > Anyway the topic starter mentioned RedHat, the kernel version must be quite > ancient there :) RedHat is funny. They used kernel 2.6.32 for a long while, back-porting security fixes and namespace (vitualization) improvements. Goal: be Stable and Current both at the same time. Recently (2014) they jumped to 3.10. Cheers -- Rick ref: https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux

1 0

TT default layout
by Rick Leir 04 Nov '15

04 Nov '15

Why would there be both [% and <% in the default layout? Am I missing something? Dancer2$ more share/skel/views/layouts/main.tt <!DOCTYPE html> <html lang="en"> <head> <meta charset="<% settings.charset %>"> <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes"> <title>[% appname %]</title> <link rel="stylesheet" href="<% request.uri_base %>/css/style.css">  <script src="//code.jquery.com/jquery-2.1.4.min.js"></script> <script type="text/javascript">/* <![CDATA[ */ !window.jQuery && document.write('<script type="text/javascript" src="<% request.uri_base %>/javascripts/jquery.js"><\/script>') /* ]]> */</script> </head> <body> <% content %> <div id="footer"> Powered by <a href="http://perldancer.org/">Dancer2</a> <% dancer_version %> </div> </body> </html> -- Cheers -- Rick

3 2