Re: [dancer-users] Passing environment to dockerized Dancer2
Dear Dancer users,
I want to run my Dancer2 application in a docker but I want to run the docker without mounting any local directories and I want to run the same container in all environments, not compile it separately for different environments.
Problem: How do I pass the environment dependent parts of the configuration to the docker? I cannot use file `config_local.yaml` because then I would need to copy that file somehow into the container when it starts.
Possible solution: I would pass parts of the configuration as environment variables, e.g. /DB_HOST/ and/SITE_URL/, and then apply these over the values in config.yaml when I am creating the endpoints.
Suggestion: Change the command `/config/` (or the underlying file reader function) so that it can transpose environment variables into config strings, e.g. plugins->database->connections->main->host = '${DB_HOST}'.
I have built a solution to this. I have not yet submitted a Pull Request. I am using this solution in my build and web application. Please comment. Would this be a good pull request? What the PR does? 1) After reading the whole config from different files, it traverses the whole tree and if it sees a string with a substring like "${ENV:SOME_ENV}", it changes it to the corresponding environment variable, if that environment variable is defined. 2) Before doing 1), it checks if the config item "env_var_replace" is defined and is boolean true. If so, then it executes 1). Otherwise it skips. This is to maintain backwards compatibility. (I have tests for the actual PR. Not included here.) diff --git a/lib/Dancer2/Core/Role/ConfigReader.pm b/lib/Dancer2/Core/Role/ConfigReader.pm index 0d0bc1ab..b8e1fdaf 100644 --- a/lib/Dancer2/Core/Role/ConfigReader.pm +++ b/lib/Dancer2/Core/Role/ConfigReader.pm @@ -232,6 +232,12 @@ sub load_config_file { } # TODO handle mergeable entries + + if( defined $config->{'env_var_replace'} && $config->{'env_var_replace'} ) { + warn "Apply environment variables to config\n" if $ENV{DANCER_CONFIG_VERBOSE}; + $self->_replace_env_vars($config); + } + return $config; } @@ -275,6 +281,48 @@ sub _compile_config_entry { return $trigger->( $self, $value, $config ); } +# Attn. We are traversing along the original data structure all the time, +# using references, and changing values on the spot, not returning anything. +sub _replace_env_vars { + my ( $self, $entry ) = @_; + if( ref $entry ne 'HASH' && ref $entry ne 'ARRAY' ) { + croak 'Param entry is not HASH or ARRAY'; + } + if( ref $entry eq 'HASH' ) { + foreach my $value (values %{ $entry }) { + if( (ref $value) =~ m/(HASH|ARRAY)/msx ) { + $self->_replace_env_vars( $value ); + } elsif( (ref $value) =~ m/(CODE|REF|GLOB)/msx ) { + # Pretty much anything else except SCALAR. Do nothing + 1; + } else { + if( $value ) { + while( my ($k, $v) = each %ENV) { + $value =~ s/ \$ [{] ENV:$k [}] /$v/gmsx; + } + } + } + } + } else { + # ref $entry is 'ARRAY' + foreach my $value (@{ $entry }) { + if( (ref $value) =~ m/(HASH|ARRAY)/msx ) { + $self->_replace_env_vars( $value ); + } elsif( (ref $value) =~ m/(CODE|REF|GLOB)/msx ) { + # Pretty much anything else except SCALAR. Do nothing + 1; + } else { + if( $value ) { + while( my ($k, $v) = each %ENV) { + $value =~ s/ \$ [{] ENV:$k [}] /$v/gmsx; + } + } + } + } + } + return; +} + 1; __END__ -- Mikko Koivunalho LinkedIn: http://www.linkedin.com/in/MikkoKoivunalho AboutMe: http://about.me/mikkokoivunalho Blog: http://www.koivunalho.org/blogs/exercises-in-integration-and-delivery/
On 19 Jul 2021, at 2:54 am, Mikko Johannes Koivunalho <mikko.koivunalho@iki.fi> wrote:
Problem: How do I pass the environment dependent parts of the configuration to the docker? I cannot use file `config_local.yaml` because then I would need to copy that file somehow into the container when it starts.
Possible solution: I would pass parts of the configuration as environment variables, e.g. /DB_HOST/ and/SITE_URL/, and then apply these over the values in config.yaml when I am creating the endpoints.
Suggestion: Change the command `/config/` (or the underlying file reader function) so that it can transpose environment variables into config strings, e.g. plugins->database->connections->main->host = '${DB_HOST}'.
I have built a solution to this. I have not yet submitted a Pull Request. I am using this solution in my build and web application. Please comment. Would this be a good pull request?
I’m not clear that this complexity is required give that the solution can be achieved using a startup script as per Racke’s previous reply. Also I’m assuming you’re not running your containers under Docker swarm or Kubernetes but if you are there are more advanced ways to achieve what I describe below. In case Racke’s approach wasn’t clear, what you do is add an entry point script to your Docker image build using the ENTRYPOINT instruction. e.g. startup.sh. This script will always run at container start up. The startup script would use the environment you had set to build an appropriate config_local.yaml inside your container before Dancer is invoked. There are lots of ways to do this, one simple and way from a shell script is to use envsubst. The last thing the startup script would do is start your Dancer app. If you want or need your container to run in read only mode then you can combine this technique with a tempfs mount for config_local.yaml. See: * https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#en... * https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.ht...
On 19 Jul 2021, at 2:54 am, Mikko Johannes Koivunalho <mikko.koivunalho@iki.fi> wrote:
Problem: How do I pass the environment dependent parts of the configuration to the docker? I cannot use file `config_local.yaml` because then I would need to copy that file somehow into the container when it starts.
Possible solution: I would pass parts of the configuration as environment variables, e.g. /DB_HOST/ and/SITE_URL/, and then apply these over the values in config.yaml when I am creating the endpoints.
Suggestion: Change the command `/config/` (or the underlying file reader function) so that it can transpose environment variables into config strings, e.g. plugins->database->connections->main->host = '${DB_HOST}'.
I have built a solution to this. I have not yet submitted a Pull Request. I am using this solution in my build and web application. Please comment. Would this be a good pull request?
I’m not clear that this complexity is required give that the solution can be achieved using a startup script as per Racke’s previous reply. Also I’m assuming you’re not running your containers under Docker swarm or Kubernetes but if you are there are more advanced ways to achieve what I describe below. In case Racke’s approach wasn’t clear, what you do is add an entry point script to your Docker image build using the ENTRYPOINT instruction. e.g. startup.sh. This script will always run at container start up. The startup script would use the environment you had set to build an appropriate config_local.yaml inside your container before Dancer is invoked. There are lots of ways to do this, one simple and way from a shell script is to use envsubst. The last thing the startup script would do is start your Dancer app. If you want or need your container to run in read only mode then you can combine this technique with a tempfs mount for config_local.yaml. See: * https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#en... * https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.ht...
participants (2)
-
Adam Clarke -
Mikko Johannes Koivunalho