Hey fellow hackers! TL;DR; I rewrote the documentation and would be happy for comments. Following is an explanation. If you're short on time, here's the PR link: https://github.com/PerlDancer/Dancer2/pull/1069 It might be easier to view only the docs themselves, available here: https://github.com/PerlDancer/Dancer2/blob/docs/from-scratch/lib/Dancer2.pm#... Long version: Documentation is hard. You probably know this. It's difficult to convey both how something works and how to use it. When working on a framework, we often find ourselves explaining where it fits in and how, and what is the problem the framework solves. In our context this might explain the HTTP protocol, how to implement a web service, or how to deploy an existing application or merge your new application within existing code. A while ago, following conversations on the topic, I decided to take a stab at rewriting the entire documentation. My goals were simpe (and daring): * Explain what Dancer2 does * Explain how Dancer2 is composed * Explain how to set it up, configure it, and use it * Cover all the important keywords I had made the following assumptions: * I focus on best practices * I do not cover at length what I don't think should be done (call it "bad practices") * I provide follow-up links (or at least try) where I prefer not to go into detail * I do not explain how HTTP works or how web applications work I would be grateful for any review of this work you can provide before we merge it as the new official documentation. Here are some leading questions to help you think through this: * Is this documentation too long? * Is this documentation not enough? * Does it explain things the documentation usually does not? * Do you think you understand more of how Dancer2 works now? * Does this clarify parts that were unclear before? * Any big pieces missing? You can reply on the list or directly to me. * The link to the Pull Request: https://github.com/PerlDancer/Dancer2/pull/1069 * The link to the documentation directly: https://github.com/PerlDancer/Dancer2/blob/docs/from-scratch/lib/Dancer2.pm#... Thank you ever so much. :) Sawyer.
On Apr 7, 2016, at 5:13 AM, Sawyer X <xsawyerx@gmail.com> wrote:
I rewrote the documentation
*Which* documentation? One of the biggest problems I’ve had with Dancer’s docs from day 1 is the distinction between https://metacpan.org/pod/Dancer2 https://metacpan.org/pod/distribution/Dancer2/lib/Dancer2/Cookbook.pod https://metacpan.org/pod/distribution/Dancer2/lib/Dancer2/Manual.pod https://metacpan.org/pod/distribution/Dancer2/lib/Dancer2/Tutorial.pod Ideally, all of this would be a single document, possibly in multiple chapters named after topics. As it stands, I end up having to search all of them to find any given topic, because they’re all named generically.
and would be happy for comments.
perlcritic --brutal ? :)
It's difficult to convey both how something works and how to use it.
That’s why I recommend two top-level documents: a user manual and a reference manual. The reference manual exists to explain class interfaces and such. It is meant to be read at point-of-need, in any order. The existing POD docs do this adequately today. The user manual is mainly prose, intended to be read in a specific order, up to the point that the reader loses interest. Thus, the easiest and most central topics come first, and later topics build on earlier material. The problem I identified above is largely because you’re trying to use POD — a tool intended more for reference docs — to generate a user manual. For example, POD doesn’t let you have a “chapter 2” that follows “chapter 1” unless you do something ugly like: Dancer2::UserMan::1_Introduction.pm Dancer2::UserMan::2_Tutorial.pm Dancer2::UserMan::3_... That’s why we have Doxygen *and* Docbook, Epydoc *and* AsciiDoc, etc. Different tools for different purposes.
* Is this documentation too long?
Kind of. You’re forced to put all of the material in a single POD because that’s the only way to get a strict ordering within POD, short of the naming hack I gave above. Tools like Docbook and AsciiDoc have ways to define chapters which end up in separate HTML files, with hyperlinks chaining them in the correct order. Each file would have a name that describes the chapter, so you could predict its contents from its name.
* Is this documentation not enough?
I haven’t read it yet, but perhaps not. The freedom to write arbitrary-length books in your user manual, chunked into reasonably-sized chapters, offers the freedom to write only as much as necessary, and no more. When everything is in a single page, you start asking questions like this, wondering if you should write less text because it makes your single page too long.
* Does it explain things the documentation usually does not? * Do you think you understand more of how Dancer2 works now? * Does this clarify parts that were unclear before? * Any big pieces missing?
I wanted to get the above out before I read everything. Expect at least one more reply.
Thank you ever so much. :)
No, thank *you*. :)
Lots of comments, wow! :) On Fri, Apr 8, 2016 at 12:54 AM, Warren Young <wyml@etr-usa.com> wrote:
On Apr 7, 2016, at 5:13 AM, Sawyer X <xsawyerx@gmail.com> wrote:
I rewrote the documentation
*Which* documentation?
Shortly replying to this entire paragraph: the documentation I'm replacing is the Manual. It wasn't deleted yet. It might be deleted in the same PR once we feel comfortable with it, or it might be kept until a later commit. Depending on whether the old Manual still has anything we want covered. A reference should still exist, possibly in the Manual, possibly outside of it. This is not what this is about.
One of the biggest problems I’ve had with Dancer’s docs from day 1 is the distinction between
https://metacpan.org/pod/Dancer2 https://metacpan.org/pod/distribution/Dancer2/lib/Dancer2/Cookbook.pod https://metacpan.org/pod/distribution/Dancer2/lib/Dancer2/Manual.pod https://metacpan.org/pod/distribution/Dancer2/lib/Dancer2/Tutorial.pod
Each part exists for a reason, but I agree they conflate and are confusing. * Dancer2.pm was the main hub which should also be a very very light introduction (if any) and instead redirect you to the Manual. * Dancer2::Manual was to be a proper manual explaining how to use Dancer2, as prose. It will be superseded by this new documentation. At the end of the Manual there is a short reference, and that should be kept (maybe there, maybe elsewhere). * Dancer2::Tutorial is supposed to be an example on how to write an application with Dancer2. Instead of explaining the structure (Manual) and available keywords (reference), it actually leads you step by step on an application. Many people prefer learning this way. We discussed separating this into a class of sample applications. "Here's how to write X", "Here's how to write Y". * Dancer2::Cookbook contains interesting (and arguably useful) patterns users have. It literally started with people documenting practices they have which are supported but not keywords in and of themselves. ("Here's how you can define a default route", "Here's how you use the WRAPPER directive in TT"). Some of these should be removed because they are (or should be) covered in the Manual. Some really are things that are just good to write down somewhere because people try to do them (the TT one is a good example). I think we can all agree documentation is hard, but we need to start somewhere. Looking at the entire picture now and fixing it all at once (which is what I understand you're interested in) is not feasible. Many people tried, they all failed. I'm explicitly attempting to not do that.
and would be happy for comments.
perlcritic --brutal ? :)
Yes. But try to keep it practical. I want comments for the documentation that exists, not for the entire documentation in Dancer, all the available parts, reorganizing the entire thing, different available formats for documentation, etc. We're doing this step by step. This is not the last and only step, but it's a big one. If we draw back to "wait, let's go over *all* the documentation (including available documentation formats) and redo it", we won't actually move forward.
It's difficult to convey both how something works and how to use it.
That’s why I recommend two top-level documents: a user manual and a reference manual.
The reference manual exists to explain class interfaces and such. It is meant to be read at point-of-need, in any order. The existing POD docs do this adequately today.
The user manual is mainly prose, intended to be read in a specific order, up to the point that the reader loses interest. Thus, the easiest and most central topics come first, and later topics build on earlier material.
That's not a bad idea, however the Cookbook was deemed more than useful in many situations and many people prefer a tutorial when learning. In contrast, when asked about the docs, some people said, "You need a tutorial! More tutorials! Only tutorials!" - Different people, different thoughts on how to do it. Let's start with what we have in front of us.
The problem I identified above is largely because you’re trying to use POD — a tool intended more for reference docs — to generate a user manual.
For example, POD doesn’t let you have a “chapter 2” that follows “chapter 1” unless you do something ugly like:
Dancer2::UserMan::1_Introduction.pm Dancer2::UserMan::2_Tutorial.pm Dancer2::UserMan::3_...
That’s why we have Doxygen *and* Docbook, Epydoc *and* AsciiDoc, etc. Different tools for different purposes.
But assuming we're staying within the realm of Pod for now...
* Is this documentation too long?
Kind of.
You’re forced to put all of the material in a single POD because that’s the only way to get a strict ordering within POD, short of the naming hack I gave above.
Tools like Docbook and AsciiDoc have ways to define chapters which end up in separate HTML files, with hyperlinks chaining them in the correct order. Each file would have a name that describes the chapter, so you could predict its contents from its name.
Understood. But considering we're not moving to LaTeX... let's move on from this point.
* Is this documentation not enough?
I haven’t read it yet, but perhaps not.
The freedom to write arbitrary-length books in your user manual, chunked into reasonably-sized chapters, offers the freedom to write only as much as necessary, and no more.
When everything is in a single page, you start asking questions like this, wondering if you should write less text because it makes your single page too long.
I see you're very focused on this, but this is something we're not going to try and fix with this PR. So, please, let's move past this.
On Apr 7, 2016, at 5:13 AM, Sawyer X <xsawyerx@gmail.com> wrote:
* I focus on best practices * I do not cover at length what I don't think should be done (call it "bad practices”)
One example of this must be the insistence on ‘route_parameters’ and such. I think it's a mistake not to use “params” almost completely through the tutorial. Only once routes, sessions, etc. are all understood should you show an example of a case where you need to be specific about the kind of parameter you want. *Then* introduce *_parameters(). If you can’t show such an example, don’t talk about the distinction at all. You may come to believe, as I do, that the distinction doesn’t matter for a great many apps. I’m glad that Dancer2 provides a way for me to be specific, but I’d rather not court RSI just to be needlessly precise. I think the material presently “Parameters” should be *last*, being less important than Control Flow, Keeping Information, and Hooks. (Yes, I mean take it out of “Handler”, and make it a peer to it.) It should not have any “recap” because it should introduce the topic rather than recapitulate an earlier introduction. It needs a motivating example at the end, to show an example of a case where using “params” gives the wrong result, so you need to use one of the *_parameters() keywords.
* I provide follow-up links (or at least try) where I prefer not to go into detail
Yes, good, gooood. Use the power of the dark s…um, HTML.
* Does it explain things the documentation usually does not?
I think you get down into the details too early. You jump from “dancer2 -a” to talking about engines, routes, and applications, then talk about how to write your own app without dancer2 -a. That forces the reader to try to cram the D2 architecture into their head before they’ve even seen “hello world” run. Instead, I think you should go from “dancer2 -a Foo” to running it. Have the user modify a few things and see how that changes the running app. That gives the user a chance to poke around in lib/Foo.pm. Along the way, describe routes. After that, have the user create an intentional error in lib/Foo.pm, observe the result in the browser, and then switch the environment to production — introducing the user to config.yml — to see how that changes the error as shown in the browser. Then have them switch an engine, such as from Dancer2::Template::Simple to the full Template::Toolkit, and change something in the views file and the pm file that requires Template::Toolkit. And only after all of that, talk about the concept of multiple apps. Maybe have the user create lib/Bar.pm in a separate namespace, and show that the two apps are separate at runtime, not just on disk. I realize that the notion of apps is a huge difference between D1 and D2, and that it is foremost in your mind as an architect of Dancer, but it is a fairly advanced concept that the user may not even have to deal with, at least at first. If there were a Dancer book, I wouldn’t expect apps to appear until chapter 7 or so. I wouldn’t talk about things like Starman at all. Just point to the Deployment document and be done with it. If you go with my userman idea and collect that document into the new one, it would be a fairly late chapter, near the end of the book. Similarly, I think “Vars” probably shouldn’t be discussed until after both “forward” and Hooks. Use those jumping mechanisms as a motivation for “var” and “vars”, which is indeed the reason for the keywords.
* Do you think you understand more of how Dancer2 works now?
I think I’ve been using Dancer too long to be surprised by any of this. :)
* Does this clarify parts that were unclear before?
I’m glad that you talk about ‘prefix’ earlier than in other Dancer docs I’ve read. Only the most trivial Dancer app ends up with a flat route hierarchy, and thus has no use for ‘prefix’.
* Any big pieces missing?
As I said in the previous email, I think several existing documents should merge into this, as separate chapters. (And those possibly broken up into multiple chapters themselves.) Once all of the primarily-prose information is organized that way, look through the outbound links to see if there is any other similar material that isn’t already linked to. At minimum, there must be a bunch of Advent Calendar articles that are worth linking to. This list should be carefully curated, not just a copy of the table of contents here: https://metacpan.org/release/Dancer2
On Fri, Apr 8, 2016 at 1:50 AM, Warren Young <wyml@etr-usa.com> wrote:
On Apr 7, 2016, at 5:13 AM, Sawyer X <xsawyerx@gmail.com> wrote:
* I focus on best practices * I do not cover at length what I don't think should be done (call it "bad practices”)
One example of this must be the insistence on ‘route_parameters’ and such. I think it's a mistake not to use “params” almost completely through the tutorial.
Other than "I don't want extra typing of characters", can you explain why it's a mistake? Having handled parameters extensively, I have a strong understanding of the problems with "params". I would be happy to understand why you're in favor of it, other than RSI.
Only once routes, sessions, etc. are all understood should you show an example of a case where you need to be specific about the kind of parameter you want. *Then* introduce *_parameters().
The goal was to introduce you from the top to using parameters based on where they come from. Instead of saying "data!", we say "this data!". This is on purpose. I presume that since you think "params()" should be used instead in general, you probably don't see why I prefer from the top to using specific parameters.
If you can’t show such an example, don’t talk about the distinction at all. You may come to believe, as I do, that the distinction doesn’t matter for a great many apps.
I don't believe that. :) (The greater the apps I worked on, the more I don't believe that.)
I think the material presently “Parameters” should be *last*, being less important than Control Flow, Keeping Information, and Hooks. (Yes, I mean take it out of “Handler”, and make it a peer to it.) It should not have any “recap” because it should introduce the topic rather than recapitulate an earlier introduction.
So the options here are: * Current: parameters, control flow, rest. * Your #1 suggestion: control flow, keeping information, hooks, etc. and then parameters. * Your #2 suggestion: parameters a little, control flow, keeping information, hooks, etc. parameters again. Did I understand this correctly? I would advise against parameters at the end because that's the first thing people want to see. Explaining to someone "let me show you control flows and hooks, we'll get to the core of what you're doing later". The second suggestion is more plausible, but that would still be splitting a concept which I think is simple and better to explain all at once. I would be more than happy to see a suggestion on how you would do it.
It needs a motivating example at the end, to show an example of a case where using “params” gives the wrong result, so you need to use one of the *_parameters() keywords.
I think you're missing the place I'm coming from then. You judge this from the perspective of "I like params() and I want you to explain why I shouldn't use it". I wanted to explicitly start from "here's how you should do it". I don't want the documentation to be an argument or a justification, it should be a documentation. If you want, I have written about this here: https://github.com/PerlDancer/Dancer2/pull/988 https://github.com/PerlDancer/Dancer2/pull/931 https://github.com/PerlDancer/Dancer2/pull/993 And the original article introducing our new approach: http://bulknews.typepad.com/blog/2009/12/perl-why-parameters-sucks-and-what-... Instead of arguing why one or the other should be used, I want to focus on what should be used and not on what shouldn't be or on why. That was one of the principles in writing this documentation.
* Does it explain things the documentation usually does not?
I think you get down into the details too early. You jump from “dancer2 -a” to talking about engines, routes, and applications, then talk about how to write your own app without dancer2 -a. That forces the reader to try to cram the D2 architecture into their head before they’ve even seen “hello world” run.
True. This was on purpose and perhaps one of the bigger things that might need to be changed. Let me explain why I did it this way. When running Dancer courses with Mickey Nasriachi (another core team member), we found that much of the confusion people had with Dancer was how it's structured and what things mean. When we said "a Dancer App", people asked what that is. How is it different from a web app. When we said "engine", people asked what those were and how that mixed with the App. Eventually, it actually became easier to explain the structure and then show how to use it. In practice, with groups, it seemed like it helped grasp things easier because they fit into a structure instead of general things like "routes" and "apps".
Instead, I think you should go from “dancer2 -a Foo” to running it. Have the user modify a few things and see how that changes the running app. That gives the user a chance to poke around in lib/Foo.pm. Along the way, describe routes.
After that, have the user create an intentional error in lib/Foo.pm, observe the result in the browser, and then switch the environment to production — introducing the user to config.yml — to see how that changes the error as shown in the browser.
Then have them switch an engine, such as from Dancer2::Template::Simple to the full Template::Toolkit, and change something in the views file and the pm file that requires Template::Toolkit.
And only after all of that, talk about the concept of multiple apps. Maybe have the user create lib/Bar.pm in a separate namespace, and show that the two apps are separate at runtime, not just on disk.
That's not a bad idea. I can do this in person, I'm not sure how to do this in a manual. I can't imagine a manual saying "okay, now please change a few things, and try again". That won't work. Can you formalize an example of how this would work in documentation?
I realize that the notion of apps is a huge difference between D1 and D2, and that it is foremost in your mind as an architect of Dancer, but it is a fairly advanced concept that the user may not even have to deal with, at least at first. If there were a Dancer book, I wouldn’t expect apps to appear until chapter 7 or so.
This was deemed to be working better from practicing it with groups.
I wouldn’t talk about things like Starman at all. Just point to the Deployment document and be done with it. If you go with my userman idea and collect that document into the new one, it would be a fairly late chapter, near the end of the book.
Interesting. Could you expand, please?
Similarly, I think “Vars” probably shouldn’t be discussed until after both “forward” and Hooks. Use those jumping mechanisms as a motivation for “var” and “vars”, which is indeed the reason for the keywords.
Could you please expand?
* Does this clarify parts that were unclear before?
I’m glad that you talk about ‘prefix’ earlier than in other Dancer docs I’ve read. Only the most trivial Dancer app ends up with a flat route hierarchy, and thus has no use for ‘prefix’.
And this is why I need to explain "appname" as well, because they are both forms of expanding an application. This is why a Dancer App needs to be explained earlier too, because then what does "appname" mean? That's actually a very commonly used feature. Thank you for all the feedback. Sawyer.
On Apr 8, 2016, at 1:39 AM, Sawyer X <xsawyerx@gmail.com> wrote:
On Fri, Apr 8, 2016 at 1:50 AM, Warren Young <wyml@etr-usa.com> wrote:
On Apr 7, 2016, at 5:13 AM, Sawyer X <xsawyerx@gmail.com> wrote:
* I focus on best practices
One example of this must be the insistence on ‘route_parameters’ and such. I think it's a mistake not to use “params” almost completely through the tutorial.
Other than "I don't want extra typing of characters", can you explain why it's a mistake?
Because I have yet to come across a case where it matters. In many apps, the code calling Dancer routes is either other Dancer routes (e.g. views which contain URLs pointing to other parts of the site) or JS code that calls back into the Dancer app, such as via Ajax. In both of these broad cases, the caller and the callee are written by the same person, so you aren’t going to get a mismatch in intent. If your app does allow outsiders to make calls into your Dancer app, such as via an API, what does it matter if the Dancer code says: post `/api/foo’ => { my $f = param ‘bar’; }; and the docs say you should call it as POST /api/foo?bar=x but the caller uses HTML form parameters instead? The data was transferred and understood. The only way I can see that it matters is if you write something like post `/api/foo/:bar’ => { my $f = param ‘bar’; }; and then the caller does something horrible like call POST /api/foo/x *and* pass a “bar” HTML form variable, leading to unclear intent. But in that case, it is the caller that is confused, and the only problem is that the caller might not get the answer they expected. But that’s the sort of thing you should expect when the API contract is violated, so why is it Dancer’s problem?
Having handled parameters extensively, I have a strong understanding of the problems with "params”.
As I challenged you in the previous reply, provide a motivating example if you expect people to type ~3x as many characters on every parameter access. Even if you come up with such a thing, I could only justify following that example in the few cases where I could see it was necessary a priori. That is, call ‘param’ or ‘params’ most of the time, except in this one weird case where the distinction matters.
I think the material presently “Parameters” should be *last*,
So the options here are: * Current: parameters, control flow, rest. * Your #1 suggestion: control flow, keeping information, hooks, etc. and then parameters. * Your #2 suggestion: parameters a little, control flow, keeping information, hooks, etc. parameters again.
Did I understand this correctly?
I would advise against parameters at the end because that's the first thing people want to see.
That’s why I say “params” first, and “*_parameters” last. You need to access your passed-in data early, but you don’t need to distinguish between the source of the data until much later, if ever.
When running Dancer courses with Mickey Nasriachi (another core team member), we found that much of the confusion people had with Dancer was how it's structured and what things mean.
Fine, but do you really need to get into architectural details in paragraph 12? You don’t teach someone about functional programming by first teaching them the lambda calculus. You don’t teach someone electronics by first deriving Maxwell’s equations from first principles. (And yes, I’m aware that there are ivory tower pedagogues guilty of both crimes!)
Instead, I think you should go from “dancer2 -a Foo” to running it…
I can do this in person, I'm not sure how to do this in a manual.
Screenshots and console output. PNGs and codeblocks.
I can't imagine a manual saying "okay, now please change a few things, and try again". That won't work.
Works for me: http://tangentsoft.net/mysql++/doc/html/userman/tutorial.html But part of that is because I’m using Docbook for that. (Incidentally, I’m knocking down your “LaTeX” strawman from the previous reply here. I never said “LaTeX”.)
Can you formalize an example of how this would work in documentation?
The source code for that page of my MySQL++ manual is here: http://svn.gna.org/viewcvs/mysqlpp/trunk/doc/userman/tutorial.dbx?view=marku... I’m not advocating DocBook specifically. I’m just pointing out that POD is scarcely more powerful than the “man” package for nroff, and it was intended for a different purpose. If you want to write an article or book, there are more appropriate tools.
I wouldn’t talk about things like Starman at all. Just point to the Deployment document and be done with it. If you go with my userman idea and collect that document into the new one, it would be a fairly late chapter, near the end of the book.
Interesting. Could you expand, please?
I’m saying that Starman and Memcached and Nginx and… don’t matter until months down the development pipeline, if at all. But here you’re talking about them in paragraph N where N < 100. If you’re still not convinced, explain to me how you program differently knowing that your session engine will be Dancer2::Session::Foo instead of Dancer2::Session::Simple? That’s the whole point of abstraction layers: you can switch between them at will, for the most part. That’s also why I talk about showing a switch of template engines early on: because although Dancer has a template engine abstraction layer, it is between Dancer core and the template engine, not between the template engine and the programmer. Thus, the programmer needs to select their template engine early on. This is not the case with session engines, caching systems, proxy layers, etc.
Similarly, I think “Vars” probably shouldn’t be discussed until after both “forward” and Hooks. Use those jumping mechanisms as a motivation for “var” and “vars”, which is indeed the reason for the keywords.
Could you please expand?
I mean you’re currently covering “var” between “forward” and “Hooks”, apparently because you want to group Vars with Cookies. There’s nothing saying you must talk about Vars and Cookies at the same time, so why not move Vars until you’ve covered all of the keywords that require the use of Vars?
And this is why I need to explain "appname" as well, because they are both forms of expanding an application. This is why a Dancer App needs to be explained earlier too, because then what does "appname" mean? That's actually a very commonly used feature.
Yes, but is it common to need to know about it in the first 30 minutes of your Dancer training? I think you might have lost perspective, coming from so many years of Dancer expertise. You’re sitting there looking at what the app needs to look like 6 months down the line when it’s deployed and are trying to explain all the architectural details will be involved along the way, but your poor reader hasn’t seen Dancer before today, and is wondering how they can get Hello World talking to the Internet before lunch.
participants (2)
-
Sawyer X -
Warren Young