Hi guys, I apologise in advance that this is rather vague, but I'm hoping that someone might be able to point me in the right direction of where I should be looking. I'm getting some strange problems with Dancer2 and the "auto serializer". I have set serializer => 'JSON'; in my app, but under some circumstances it is trying to serialize HTML content, with the subsequent errors that you would expect of such an action. The problems seem to be related to my error handling (using Log::Report). My implementation is currently rather poor and complicated, so I won't go into the details here, but I wondered whether someone can point me to the relevant parts of the Dancer2 code that I should start poking around in (in particular the code which detects whether to serialize the data). I've looked at serialize() in Serializer::Mutuble, but as far as I can tell this function is not being called. Where else should I look? If I remove the aforementioned serializer statement then everything works as expected. Thanks, Andy
On Thu, Oct 9, 2014 at 1:47 PM, Andrew Beverley <andy@andybev.com> wrote:
Hi guys,
Hi Andrew. :)
I apologise in advance that this is rather vague, but I'm hoping that someone might be able to point me in the right direction of where I should be looking.
Not at all! That is an excellent question.
[...] so I won't go into the details here, but I wondered whether someone can point me to the relevant parts of the Dancer2 code that I should start poking around in (in particular the code which detects whether to serialize the data).
You can see that Dancer2::Core::Request checks for a supported content type. The line is: return unless $self->serializer->support_content_type($content_type); However, in Dancer2::Core::Response, it doesn't check for a supported content type in the request in order to serialize a response back. This would be the correct behavior since a serializer basically says "I would like to serialize my responses" and not "I would like to serialize my responses IF the request was made with a header". The reason it ignores any request header in the response is because it is, by definition, a *request* header. This means it only relates to the request, not the response. If you would like to return HTML and serialized information, you have two options: 1. Separate the code that accepts and returns serialized data into two different Dancer2 apps. 2. Don't use a serializer, and call the to_json and from_json when *you* decide it's the right time for it. I hope this answer helps.
On Thu, 2014-10-09 at 19:43 +0200, Sawyer X wrote:
Not at all! That is an excellent question.
Great, thanks for the prompt reply.
You can see that Dancer2::Core::Request checks for a supported content type. The line is:
return unless $self->serializer->support_content_type($content_type);
However, in Dancer2::Core::Response, it doesn't check for a supported content type in the request in order to serialize a response back. This would be the correct behavior since a serializer basically says "I would like to serialize my responses" and not "I would like to serialize my responses IF the request was made with a header". The reason it ignores any request header in the response is because it is, by definition, a *request* header. This means it only relates to the request, not the response.
Right, got it. Thanks. Basically, when I was receiving requests for JSON from the browser, I was forwarding to a normal HTML page in the case of an exception. This was causing the problems with the HTML being serialized.
If you would like to return HTML and serialized information, you have two options: 1. Separate the code that accepts and returns serialized data into two different Dancer2 apps. 2. Don't use a serializer, and call the to_json and from_json when *you* decide it's the right time for it.
What I've done is to detect whether the request is for serialized data, and if it is then not to forward the HTML page in the case of an exception. One thing I did find is that request->content_type was empty when I called it (in the controller after an exception). So instead, I used if (request->serializer =~ /JSON/) That seems a bit messy. Is there a better way? Thanks again, Andy
On Thu, 2014-10-09 at 12:47 +0100, Andrew Beverley wrote:
Hi guys,
I apologise in advance that this is rather vague, but I'm hoping that someone might be able to point me in the right direction of where I should be looking.
I'm getting some strange problems with Dancer2 and the "auto serializer". I have
set serializer => 'JSON';
in my app, but under some circumstances it is trying to serialize HTML content, with the subsequent errors that you would expect of such an action.
The problems seem to be related to my error handling (using Log::Report). My implementation is currently rather poor and complicated, so I won't go into the details here, but I wondered whether someone can point me to the relevant parts of the Dancer2 code that I should start poking around in (in particular the code which detects whether to serialize the data). I've looked at serialize() in Serializer::Mutuble, but as far as I can tell this function is not being called. Where else should I look?
If I remove the aforementioned serializer statement then everything works as expected.
Just thought I'd follow up on this. It turned out that the problem was that my text to render to the screen was a reference to an object that normally stringified into the required text (Log::Report::Message). However, because the serializer was simply checking for a reference, the object didn't have a chance to stringify itself, and instead an attempt was made to convert the object into JSON. The fix was easy (just stringify first), but I did wonder whether a better error message could be displayed somehow, as it took me rather a long time to debug. I came to the conclusion that it couldn't, but if anyone has any ideas then please let me know. Andy
On Tue, Nov 11, 2014 at 7:06 PM, Andrew Beverley <andy@andybev.com> wrote:
[...] Just thought I'd follow up on this. It turned out that the problem was that my text to render to the screen was a reference to an object that normally stringified into the required text (Log::Report::Message). However, because the serializer was simply checking for a reference, the object didn't have a chance to stringify itself, and instead an attempt was made to convert the object into JSON.
The fix was easy (just stringify first), but I did wonder whether a better error message could be displayed somehow, as it took me rather a long time to debug. I came to the conclusion that it couldn't, but if anyone has any ideas then please let me know.
I have two comments left: 1. I don't think API requests should be conflated with regular page rendering. This should be a clear "I send you JSON, I receive JSON" and a clear "I send a request from the browser to render output for the user and expect to receive HTML back". There is one exception in which you might want to make a JSON request return a rendered HTML which you could stick in the page ("template 'parts/box' => {...}, { layout => undef };") but that's something which would then also get serialized and returned back. Imagine an API on the frontend having to account for either JSON or HTML, not knowing how an error would be returned. It could be a JSON structure with the error or an HTML which itself is an error. It's gonna have a bad time. :) 2. If you found the error unhelpful, we should definitely fix it. That's about it. :) If you could open an issue on Github so we can easily add all the context and track it there, we would be more than happy to help. You could make suggestions for what you think a better error message could be. Thanks! :)
On Wed, 2014-11-12 at 10:23 +0100, Sawyer X wrote:
I have two comments left: 1. I don't think API requests should be conflated with regular page rendering. This should be a clear "I send you JSON, I receive JSON" and a clear "I send a request from the browser to render output for the user and expect to receive HTML back". There is one exception in which you might want to make a JSON request return a rendered HTML which you could stick in the page ("template 'parts/box' => {...}, { layout => undef };") but that's something which would then also get serialized and returned back. Imagine an API on the frontend having to account for either JSON or HTML, not knowing how an error would be returned. It could be a JSON structure with the error or an HTML which itself is an error. It's gonna have a bad time. :)
Okay, I think I may be using this wrong. I have the auto-serializer set to "JSON", which from what I can tell, will look at every response, and always attempt to serialize the output as long as it is passed a ref, regardless of the request. So when I'm returning HTML to a HTML request, it will still try and serialize the data if that HTML happens to be in a reference (even if it stringifies). I am assuming that I should instead set the serializer to "Mutable", which will only try to serialize the appropriate requests.
2. If you found the error unhelpful, we should definitely fix it.
I guess my point was that I couldn't think of any way it could be more helpful, otherwise I would have made a pull request ;-)
If you could open an issue on Github so we can easily add all the context and track it there, we would be more than happy to help.
Thanks. If my comments above stand that I should be (in general) using the Mutable serializer instead of the JSON serializer, than I'll be happy to submit some documentation patches. Thanks, Andy
On Wed, Nov 19, 2014 at 1:27 PM, Andrew Beverley <andy@andybev.com> wrote:
On Wed, 2014-11-12 at 10:23 +0100, Sawyer X wrote:
I have two comments left: 1. I don't think API requests should be conflated with regular page rendering. This should be a clear "I send you JSON, I receive JSON" and a clear "I send a request from the browser to render output for the user and expect to receive HTML back". There is one exception in which you might want to make a JSON request return a rendered HTML which you could stick in the page ("template 'parts/box' => {...}, { layout => undef };") but that's something which would then also get serialized and returned back. Imagine an API on the frontend having to account for either JSON or HTML, not knowing how an error would be returned. It could be a JSON structure with the error or an HTML which itself is an error. It's gonna have a bad time. :)
Okay, I think I may be using this wrong. I have the auto-serializer set to "JSON", which from what I can tell, will look at every response, and always attempt to serialize the output as long as it is passed a ref, regardless of the request. So when I'm returning HTML to a HTML request, it will still try and serialize the data if that HTML happens to be in a reference (even if it stringifies).
This used to be true, but recent version changed it and for good cause. Originally serializers would check if it's a reference or not, which is wrong because some serializers can also serialize scalars which aren't references. This meant we're both crippling new serializers we haven't thought of (Dancer2::Serializer::CBOR, for instance) and providing inconsistent behavior to you, the user. Latest version of Dancer2 will *always* call the serializer to deserialize and serialize. If you provide bad input, you'll get an error. If your users send a bad request, it should result in an error as well. While those errors should do the right thing (render an HTML or return a serialized error), you can still hook up to errors, both in the app and the serializer. We still have two additional features we want to implement regarding those (that Mickey Nasriachi and I worked on yesterday): having a default serialized output (for cascading recursive serialization errors) and using Throwable internally.
I am assuming that I should instead set the serializer to "Mutable", which will only try to serialize the appropriate requests.
Mutable (which itself could use a once-over and has associated issues on GH) should be able to handle multiple serializers. Paths which have serializers should be expected to always try to deserialize and always return serialized output.
2. If you found the error unhelpful, we should definitely fix it.
I guess my point was that I couldn't think of any way it could be more helpful, otherwise I would have made a pull request ;-)
Fair enough. :)
Thanks,
All the best! Sawyer.
participants (2)
-
Andrew Beverley -
Sawyer X