July 2021 - dancer-users - lists.preshweb.co.uk

Debugging sessions
by Nikola Mitev 29 Jun '23

29 Jun '23

Hi all, I have an annoyingly intermittent problem where on successful authentication the session info disappears and the user is endlessly redirected to the login page. It happens about once a month on a site with 10-20 logins daily. Once it happens to a user, it keeps happening for that user only. A restart of the app usually fixes the issue, and I am yet to see someone who had this problem to report it again at a later stage. Had about 10 cases so far in total. The cookie is generated on accessing the login page, stored in the browser and sent with the login POST request so I can verify cookies are enabled. The user is then authenticated, data added to the session including an 'authenticated' flag and they are redirected to their dashboard. Right before the redirection the session data is present. They make a request for the dashboard following the redirect, this triggers a session authenticated check which fails as no data is available for the session. They are then redirected for authentication and so on in a loop. If I restart the app the issue vanishes. hook before => sub { # <snip> if (!session('authenticated')) { forward '/login'; } } post '/login' => sub { # user authentication and session storage here, then the debug code and the redirecton: my $authenticated = session('authenticated'); $log->info("User authenticated ($authenticated), redirecting to $target_route"); return redirect $target_route; }; I don't have an explicit check on cookie size - not sure how to check it doesn't exceed 4K - but find it unlikely this is the issue. If that was the problem it should be happening persistently for a user regardless of restarts etc? Hope all this makes sense, I am looking for general advice on how to troubleshoot this but any ideas at all will be very much appreciated. Nik

4 11

Re: [dancer-users] dancer-users Digest, Vol 124, Issue 4
by Mikko Johannes Koivunalho 27 Jul '21

27 Jul '21

The alternatives suggested all have shortcomings in my view. A script which creates config_local.yaml on the fly and then starts Dancer2 app would otherwise work but it means that container won't be immutable. Immutable is one of the architectural best practices listed by Google. As it happens, this container is planned to run on Google Container Engine. You mentioned there are other ways to do this in Kubernetes. How? Using a tmpfs is also not an option because it would mean that container is dependent on the way it is being run (the runtime environment). Substituting variables in config.yaml is not overly complicated in my mind. Many platforms use environment variables to pass information to containers, docker or others, e.g. Java jar. I have used Openshift platform and it gave me a very good example of how env vars are governed in order to provide secrets to the container. By stating the env vars in the configuration, we make it clear to the programmer and to operations what is expected from the runtime environment. Besides this, it also makes local development easier if you "practice dotenv" - what The Twelve-Factor App recommends. I keep my environment in a .env file and I also use the same file when testing the container with local docker. See: * https://cloud.google.com/architecture/best-practices-for-operating-containe… * https://docs.docker.com/storage/tmpfs/ * https://12factor.net/config On 27/07/2021 13:00, dancer-users-request(a)lists.preshweb.co.uk wrote: > Send dancer-users mailing list submissions to > dancer-users(a)lists.preshweb.co.uk > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.preshweb.co.uk/mailman/listinfo/dancer-users > or, via email, send a message with subject or body 'help' to > dancer-users-request(a)lists.preshweb.co.uk > > You can reach the person managing the list at > dancer-users-owner(a)lists.preshweb.co.uk > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of dancer-users digest..." > > > Today's Topics: > > 1. Re: Passing environment to dockerized Dancer2 (Adam Clarke) > 2. Re: Passing environment to dockerized Dancer2 (Adam Clarke) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 19 Jul 2021 10:51:08 +1000 > From: Adam Clarke <adamc(a)strategicdata.com.au> > To: Perl Dancer users mailing list <dancer-users(a)lists.preshweb.co.uk> > Subject: Re: [dancer-users] Passing environment to dockerized Dancer2 > Message-ID: > <FF6A5F5E-448C-4F0D-B829-1DA2BDAE3BAC(a)strategicdata.com.au> > Content-Type: text/plain; charset=utf-8 > > >> On 19 Jul 2021, at 2:54 am, Mikko Johannes Koivunalho <mikko.koivunalho(a)iki.fi> wrote: >>> Problem: >>> How do I pass the environment dependent parts of the configuration to >>> the docker? I cannot use file `config_local.yaml` because then I would >>> need to copy that file somehow into the container when it starts. >>> >>> Possible solution: >>> I would pass parts of the configuration as environment variables, e.g. >>> /DB_HOST/ and/SITE_URL/, and then apply these over the values in >>> config.yaml when I am creating the endpoints. >>> >>> Suggestion: >>> Change the command `/config/` (or the underlying file reader function) >>> so that it can transpose environment variables into config strings, e.g. >>> plugins->database->connections->main->host = '${DB_HOST}'. >> >> I have built a solution to this. >> I have not yet submitted a Pull Request. >> I am using this solution in my build and web application. >> Please comment. Would this be a good pull request? > > I?m not clear that this complexity is required give that the solution can be > achieved using a startup script as per Racke?s previous reply. Also I?m > assuming you?re not running your containers under Docker swarm or > Kubernetes but if you are there are more advanced ways to achieve what I > describe below. > > In case Racke?s approach wasn?t clear, what you do is add an entry point > script to your Docker image build using the ENTRYPOINT instruction. > e.g. startup.sh. This script will always run at container start up. > > The startup script would use the environment you had set to build an > appropriate config_local.yaml inside your container before Dancer is > invoked. There are lots of ways to do this, one simple and way from a > shell script is to use envsubst. The last thing the startup script would do > is start your Dancer app. > > If you want or need your container to run in read only mode then you > can combine this technique with a tempfs mount for config_local.yaml. > > See: > * https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#e… > * https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.h… > > > > ------------------------------ > > Message: 2 > Date: Tue, 20 Jul 2021 17:52:48 +1000 > From: Adam Clarke <adamc(a)strategicdata.com.au> > To: Perl Dancer users mailing list <dancer-users(a)lists.preshweb.co.uk> > Subject: Re: [dancer-users] Passing environment to dockerized Dancer2 > Message-ID: > <EF3743C7-C62A-477D-BD13-D82F0E7714DB(a)strategicdata.com.au> > Content-Type: text/plain; charset=utf-8 > > >> On 19 Jul 2021, at 2:54 am, Mikko Johannes Koivunalho <mikko.koivunalho(a)iki.fi> wrote: >>> Problem: >>> How do I pass the environment dependent parts of the configuration to >>> the docker? I cannot use file `config_local.yaml` because then I would >>> need to copy that file somehow into the container when it starts. >>> >>> Possible solution: >>> I would pass parts of the configuration as environment variables, e.g. >>> /DB_HOST/ and/SITE_URL/, and then apply these over the values in >>> config.yaml when I am creating the endpoints. >>> >>> Suggestion: >>> Change the command `/config/` (or the underlying file reader function) >>> so that it can transpose environment variables into config strings, e.g. >>> plugins->database->connections->main->host = '${DB_HOST}'. >> >> I have built a solution to this. >> I have not yet submitted a Pull Request. >> I am using this solution in my build and web application. >> Please comment. Would this be a good pull request? > > I?m not clear that this complexity is required give that the solution can be > achieved using a startup script as per Racke?s previous reply. Also I?m > assuming you?re not running your containers under Docker swarm or > Kubernetes but if you are there are more advanced ways to achieve what I > describe below. > > In case Racke?s approach wasn?t clear, what you do is add an entry point > script to your Docker image build using the ENTRYPOINT instruction. > e.g. startup.sh. This script will always run at container start up. > > The startup script would use the environment you had set to build an > appropriate config_local.yaml inside your container before Dancer is > invoked. There are lots of ways to do this, one simple and way from a > shell script is to use envsubst. The last thing the startup script would do > is start your Dancer app. > > If you want or need your container to run in read only mode then you > can combine this technique with a tempfs mount for config_local.yaml. > > See: > * https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#e… > * https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.h… > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > dancer-users mailing list > dancer-users(a)lists.preshweb.co.uk > https://lists.preshweb.co.uk/mailman/listinfo/dancer-users > > > ------------------------------ > > End of dancer-users Digest, Vol 124, Issue 4 > ******************************************** >

1 0

dancer1 and DBIx: can I modify the returned result in a view?
by John Stoffel 21 Jul '21

21 Jul '21

Hi all, I'm tasked with hacking an old Dancer1 app I wrote back in 2014/2015 which basically just returns from results from a DB query. One of the fields is called 'url' and I need to tweak it, since the URL in the DB is completely wrong. I've tried poking at it in my views/search.tt file, but I can't figure out the right syntax to make it work. My results loop looks like this: <% IF results.size %> Results (found <% results.size %> matches): <ul> <% FOREACH result IN results %> <li><% result.full_name.replace("((?i)$tobold)",'<b>$1</b>') %> <ul> <% FOREACH a IN result.account %> <li><a href="<% a.url() %>" target="_blank"> Account # <% a.account_number %>, <% IF a.volume() %> Volume: <% a.volume() %>, <% END %> <% IF a.box() %> Box: <% a.box() %>, <% END %> <% IF a.folder() %> Folder: <% a.folder() %>, <% END %> <% IF a.range() %> Range: <% a.range() %> <% END %> </a> <% newurl %><br> <% END %> </ul> <% END %> And I need to pull out the a.url, do some manipulations on it, then display it. I tried doing: <% FOREACH a IN result.account %> <% a.url = a.url.replace("http://gigi.mwa.org/netpub/server.np", "https://gigi.mwa.org/imagearchive/filename/") %> <li><a href="<% a.url() %>" target="_blank"> Account # <% a.account_number %>, <% IF a.volume() %> Volume: <% a.volume() %>, <% END %> <% IF a.box() %> Box: <% a.box() %>, <% END %> <% IF a.folder() %> Folder: <% a.folder() %>, <% END %> <% IF a.range() %> Range: <% a.range() %> <% END %> </a> <% newurl %><br> <% END %> </ul> But that didn't seem to work. Should I be doing this search and replace in the lib/Foo.pm file in the _perform_search function instead? And if so, how do I deal with the DBIx resultset properly so I can loop through and update these URLs? I really don't want to have to do this in SQL somewhere, though I guess I could be convinced to do this hack until the DB itself is updated. Thanks, John

1 1

Re: [dancer-users] Passing environment to dockerized Dancer2
by Mikko Johannes Koivunalho 20 Jul '21

20 Jul '21

> Dear Dancer users, > > I want to run my Dancer2 application in a docker but I want to run the > docker without mounting any local directories and I want to run the same > container in all environments, not compile it separately for different > environments. > > Problem: > How do I pass the environment dependent parts of the configuration to > the docker? I cannot use file `config_local.yaml` because then I would > need to copy that file somehow into the container when it starts. > > Possible solution: > I would pass parts of the configuration as environment variables, e.g. > /DB_HOST/ and/SITE_URL/, and then apply these over the values in > config.yaml when I am creating the endpoints. > > Suggestion: > Change the command `/config/` (or the underlying file reader function) > so that it can transpose environment variables into config strings, e.g. > plugins->database->connections->main->host = '${DB_HOST}'. I have built a solution to this. I have not yet submitted a Pull Request. I am using this solution in my build and web application. Please comment. Would this be a good pull request? What the PR does? 1) After reading the whole config from different files, it traverses the whole tree and if it sees a string with a substring like "${ENV:SOME_ENV}", it changes it to the corresponding environment variable, if that environment variable is defined. 2) Before doing 1), it checks if the config item "env_var_replace" is defined and is boolean true. If so, then it executes 1). Otherwise it skips. This is to maintain backwards compatibility. (I have tests for the actual PR. Not included here.) diff --git a/lib/Dancer2/Core/Role/ConfigReader.pm b/lib/Dancer2/Core/Role/ConfigReader.pm index 0d0bc1ab..b8e1fdaf 100644 --- a/lib/Dancer2/Core/Role/ConfigReader.pm +++ b/lib/Dancer2/Core/Role/ConfigReader.pm @@ -232,6 +232,12 @@ sub load_config_file { } # TODO handle mergeable entries + + if( defined $config->{'env_var_replace'} && $config->{'env_var_replace'} ) { + warn "Apply environment variables to config\n" if $ENV{DANCER_CONFIG_VERBOSE}; + $self->_replace_env_vars($config); + } + return $config; } @@ -275,6 +281,48 @@ sub _compile_config_entry { return $trigger->( $self, $value, $config ); } +# Attn. We are traversing along the original data structure all the time, +# using references, and changing values on the spot, not returning anything. +sub _replace_env_vars { + my ( $self, $entry ) = @_; + if( ref $entry ne 'HASH' && ref $entry ne 'ARRAY' ) { + croak 'Param entry is not HASH or ARRAY'; + } + if( ref $entry eq 'HASH' ) { + foreach my $value (values %{ $entry }) { + if( (ref $value) =~ m/(HASH|ARRAY)/msx ) { + $self->_replace_env_vars( $value ); + } elsif( (ref $value) =~ m/(CODE|REF|GLOB)/msx ) { + # Pretty much anything else except SCALAR. Do nothing + 1; + } else { + if( $value ) { + while( my ($k, $v) = each %ENV) { + $value =~ s/ \$ [{] ENV:$k [}] /$v/gmsx; + } + } + } + } + } else { + # ref $entry is 'ARRAY' + foreach my $value (@{ $entry }) { + if( (ref $value) =~ m/(HASH|ARRAY)/msx ) { + $self->_replace_env_vars( $value ); + } elsif( (ref $value) =~ m/(CODE|REF|GLOB)/msx ) { + # Pretty much anything else except SCALAR. Do nothing + 1; + } else { + if( $value ) { + while( my ($k, $v) = each %ENV) { + $value =~ s/ \$ [{] ENV:$k [}] /$v/gmsx; + } + } + } + } + } + return; +} + 1; __END__ -- Mikko Koivunalho LinkedIn: http://www.linkedin.com/in/MikkoKoivunalho AboutMe: http://about.me/mikkokoivunalho Blog: http://www.koivunalho.org/blogs/exercises-in-integration-and-delivery/

2 2