Solved... (it was an invisible character, but see below) -- On Nov 14, 2011, at 6:13 AM, David Precious wrote:

On Monday 14 November 2011 03:09:01 Puneet Kishor wrote:

...

This is bizarre.

<snipped fine-looking config>

...

The above config file works just fine on my laptop, but fails on my server with the following message

$perl bin/app.pl Use of uninitialized value in numeric gt (>) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/ Loader.pm line 718, [...] Loader.pm line 407, <$IN> line 1. Unable to parse the configuration file: /Users/punkish/Sites/app/config.yml: YAML Error: Invalid element in map Code: YAML_LOAD_ERR_BAD_MAP_ELEMENT Line: 9 Document: 1 at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML. pm line 36 at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/Dance r/Config.pm line 204

[...]

Hmm.

Do you know what version of YAML.pm you have installed on your server? If it's not the latest, could you try updating it, and see if the problem persists?

Can you see if you can parse the YAML directly, without Dancer's involvement?

E.g.:

perl -MData::Dump=pp -MYAML -E 'say pp(YAML::LoadFile("config.yml"));'

Update YAML from 0.73 to 0.77 punkish@strata ~/Sites/pbdb$cpanm YAML --> Working on YAML Fetching http://search.cpan.org/CPAN/authors/id/I/IN/INGY/YAML-0.77.tar.gz ... OK Configuring YAML-0.77 ... OK Building and testing YAML-0.77 ... OK Successfully installed YAML-0.77 (upgraded from 0.73) 1 distribution installed Checked directly with YAML... no joy punkish@strata ~/Sites/pbdb$perl -MData::Dump=pp -MYAML -E 'say pp(YAML::LoadFile("config.yml"));' Use of uninitialized value in numeric gt (>) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 716, <$IN> line 1. Use of uninitialized value in numeric gt (>) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 717, <$IN> line 1. Use of uninitialized value in numeric gt (>) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 717, <$IN> line 1. Use of uninitialized value in numeric ne (!=) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 719, <$IN> line 1. Use of uninitialized value in subtraction (-) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 724, <$IN> line 1. Use of uninitialized value in numeric eq (==) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 359, <$IN> line 1. Use of uninitialized value in numeric eq (==) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 359, <$IN> line 1. Use of uninitialized value in numeric eq (==) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 359, <$IN> line 1. Use of uninitialized value in substitution (s///) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 329, <$IN> line 1. Use of uninitialized value in substitution (s///) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 336, <$IN> line 1. Use of uninitialized value in substitution (s///) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 340, <$IN> line 1. Use of uninitialized value in substitution (s///) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 405, <$IN> line 1. Use of uninitialized value in substitution (s///) at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 405, <$IN> line 1. YAML Error: Invalid element in map Code: YAML_LOAD_ERR_BAD_MAP_ELEMENT Line: 9 Document: 1 at /Users/punkish/perl5/perlbrew/perls/perl-5.14.1/lib/site_perl/5.14.1/YAML/Loader.pm line 352 Opened the file again in vi and `:set list`, and bingo! there was a character that looked like ^I, a result of pressing 'tab' Seems like YAML doesn't like tabs. I deleted that character, and now everything is well. Many thanks, -- Puneet Kishor

On Mon, Nov 14, 2011 at 8:11 AM, David Precious <davidp@preshweb.co.uk> wrote:

On Monday 14 November 2011 15:33:19 Mr. Puneet Kishor wrote:

...

Solved... (it was an invisible character, but see below) -- [...] Opened the file again in vi and `:set list`, and bingo! there was a character that looked like ^I, a result of pressing 'tab'

Seems like YAML doesn't like tabs. I deleted that character, and now everything is well.

Ah, good stuff, glad to hear you got it sorted.

I'm beginning to wonder if we should add a little magic to Dancer to catch YAML parsing errors, and, at the very least, output a message making it clearer that the problem is not with Dancer but that YAML.pm couldn't parse the YAML and provide a link to the YAML spec, and perhaps also, look for any tabs in the file (before any non-whitespace chars) and, if found, output a "It looks like you've got tabs in the file; don't do that" message.

I agree to some degree with previous messages on the subject of YAML parsing failures that it's not Dancer's problem, but if we can make things easier for the user at little cost, I think we should.

So, what I propose is, if the parsing fails, log a message something like:

 "It wasn't possible to parse your config.yml file.  This file needs to be valid YAML in order to be parsed by YAML.pm - please see the YAML specifications for details.  In particular, YAML is sensitive to indentation - use a consistent number of spaces, do not use tabs."

I think this type of message would be useful, but...

Also, if the parsing failed, open config.yml and skim over it, and report any lines which included tabs, with the line number.

This would be a fairly small amount of code, and would only come into play if the parsing of config.yml failed (in which case, the app is unlikely to start anyway, so it's not like it's extra complexity / expense in the majority of cases).

It seems like it would be better to contribute to the YAML module itself so it could report errors like this, as opposed to doing post-processing on an already-failed file. Also, aren't tabs allowed inside quoted values? By the time you take into account all the valid possibilities, you will have rewritten a YAML parser.

If other devs agree that makes sense, I'll rock up something like this, initially for Dancer then see about porting it forwards to Dancer2.

I was going to append "when I have a circular tuit", but unfortunately I no longer have that excuse, thanks to the guys at LPW2011:

http://www.preshweb.co.uk/downloads/tuit/lpw2011-tuit-bothsides.jpg

:)

(I should have picked up a whole bag ;) )

-- David Precious  ("bigpresh") http://www.preshweb.co.uk/

  "Programming is like sex. One mistake and you have to support   it for the rest of your life". (Michael Sinz) _______________________________________________ Dancer-users mailing list Dancer-users@perldancer.org http://www.backup-manager.org/cgi-bin/listinfo/dancer-users

Is indenting with tabs a problem specific to Dancer users? I would say not, unless we count "not knowing YAML" as a likely property of Dancer users Therefore, I would think if it is worth implementing this, it is worth doing so in the YAML module. If "not knowing YAML" is a likely property of Dancer users, then updating the documentation would seem a simpler solution than writing a custom error checker feature. Interestingly, the documentation already states that config.yml is a YAML file. Config.pm currently reads: A better way of defining settings exists: using YAML file. For this to be possible, you have to install the L<YAML> module. If a file named B<config.yml> exists in the application directory, it will be loaded, as a setting group. It could read (note "a YAML file" on first line): A better way of defining settings exists: using a YAML file. For this to be possible, you have to install the L<YAML> module. If a file named B<config.yml> exists in the application directory, it will be loaded, as a setting group. If the file exists but is not valid YAML (for example, if it contains unquoted tab characters or does not end in a new line), YAML - and Dancer - will die. As an aside, I don't understand the phrase "as a setting group". Is it a group of settings or a group which is setting something else? Does the group have an identity of some kind, distinct from other setting groups (if so, how is it accessed?), or is it simply an anonymous list of key/value pairs which will be mixed in with other settings? Are there restrictions on the YAML files which are accepted - e.g. do they need to evaluate to a hash-like structure or an array-like structure? (I'm guessing hash) What is the priority of settings loaded through config.yml - are they overwritten by 'set' commands executed after "use Dancer;"? (I'm guessing 'yes') Daniel From: David Precious <davidp@preshweb.co.uk> To: brian@massassi.com Cc: dancer-users@perldancer.org Date: 14/11/2011 16:27 Subject: Re: [Dancer-users] User-friendly errors on YAML parsing failures (was: Re: dancer croaking with a strange error in config.yml) Sent by: dancer-users-bounces@perldancer.org On Monday 14 November 2011 16:16:29 Brian E. Lozier wrote:

On Mon, Nov 14, 2011 at 8:11 AM, David Precious <davidp@preshweb.co.uk>

...

Also, if the parsing failed, open config.yml and skim over it, and report any lines which included tabs, with the line number.

This would be a fairly small amount of code, and would only come into play if the parsing of config.yml failed (in which case, the app is unlikely to start anyway, so it's not like it's extra complexity / expense in the majority of cases).

It seems like it would be better to contribute to the YAML module itself so it could report errors like this, as opposed to doing post-processing on an already-failed file.

Hmm, yeah, that could make sense - although I think YAML.pm is to be replaced with essentially YAML::Any at some point. Getting all the parsers to output a friendly message is likely harder than just outputting a fairly generic message ourselves. We could have the message refer to an FAQ entry which lists various common mistakes in YAML for more details, perhaps.

Also, aren't tabs allowed inside quoted values? By the time you take into account all the valid possibilities, you will have rewritten a YAML parser.

Yeah, that's why I mentioned earlier in the message "look for any tabs in the file (before any non-whitespace chars)" - yes, tabs within quoted values should be valid, so I'd look only for e.g. /^\s*\t/ or similar. Cheers Dave P _______________________________________________ Dancer-users mailing list Dancer-users@perldancer.org http://www.backup-manager.org/cgi-bin/listinfo/dancer-users

On Nov 14, 2011, at 10:11 AM, David Precious wrote:

On Monday 14 November 2011 15:33:19 Mr. Puneet Kishor wrote:

...

Solved... (it was an invisible character, but see below) -- [...] Opened the file again in vi and `:set list`, and bingo! there was a character that looked like ^I, a result of pressing 'tab'

Seems like YAML doesn't like tabs. I deleted that character, and now everything is well.

Ah, good stuff, glad to hear you got it sorted.

I'm beginning to wonder if we should add a little magic to Dancer to catch YAML parsing errors, and, at the very least, output a message making it clearer that the problem is not with Dancer but that YAML.pm couldn't parse the YAML and provide a link to the YAML spec, and perhaps also, look for any tabs in the file (before any non-whitespace chars) and, if found, output a "It looks like you've got tabs in the file; don't do that" message.

..

In my view, none of the above is necessary. What is more important is to have this information in the config file docs "NO TABS; FINAL NEWLINE" along with a sample config file. Now that I know that hidden tabs were causing the program to croak, I will make sure they never occur again, or, if they do by mistake, then I know how to fix the error. -- Puneet Kishor

On Tue, Nov 15, 2011 at 11:42 AM, damien krotkine <dkrotkine@gmail.com>wrote:

try to parse the config file. If it fails, try to parse the error message sent by YAML, and extract the line / column from the message. It's usually reported accurately. Then 2 cases : if Carp::Verbose is false (general case), perform a croak (not a confess), with a human message saying " failed to parse config file '$filename', at line 42, column 12. Take extra care to spaces and tabs, blabla". If Carp::Verbose is true, then do a confess instead, appending the full YAML message.

Disagree. We shouldn't be trying to parse error messages of other modules. You cannot be sure it will be consistent between versions or between engines or between modules. We should be outputting a "Reading your configuration file failed. This was the error message: $msg\n" Something like that, no more.

On Tue, Nov 15, 2011 at 11:46:01AM +0200, sawyer x wrote:

On Tue, Nov 15, 2011 at 11:42 AM, damien krotkine <dkrotkine@gmail.com>wrote:

...

try to parse the config file. If it fails, try to parse the error message sent by YAML, and extract the line / column from the message. It's usually reported accurately. Then 2 cases : if Carp::Verbose is false (general case), perform a croak (not a confess), with a human message saying " failed to parse config file '$filename', at line 42, column 12. Take extra care to spaces and tabs, blabla". If Carp::Verbose is true, then do a confess instead, appending the full YAML message.

Disagree. We shouldn't be trying to parse error messages of other modules. You cannot be sure it will be consistent between versions or between engines or between modules.

We should be outputting a "Reading your configuration file failed. This was the error message: $msg\n" Something like that, no more.

I agree with Sawyer. This is also the simplest code change that can possibly be done. Along with the doc changes discussed before (either in Dancer::Config or in a FAQ entry), that should solve a great deal of YAML-related issues. -- Philippe Bruhat (BooK) In war, the only winners are those who sell the weapons. (Moral from Groo #3 (Image))