Dev and Release workflow ( was: Re: Invalid value for log config )
*top-posting* OK here is a first version of my opinion. You may not like it :) I'm playing a bit of devil's advocate too. We are talking about Dancer : a cool project that is very easy to use, to start with, and to build your web platform on. I suggest we try to have the same goal for the development and release environment. So there should be no thinking needed when getting a production release on CPAN, and there should be freedom to hack Dancer code 1/ About CPAN releases : I don't think it's useful to have even and odd releases visible for user (aka on CPAN). Either you have something stable you are happy to ship for production use, and you ship it on CPAN, or you don't ship it at all. Having multiple versions on CPAN with odd numbers, etc, will make it more difficult for users to know which one is good for them. Especially because we don't have strict deadlines. So basically, you may end up in 2014 having 1.4 good and stable since feb 2011 but with an old set of features, and 1.5 containing a bunch of new features, but considered unstable. Basically that's the debian approach (*ouch* don't *ouch* hit me! ), and release management and cycles will become a nightmare. So I'm in favor for 1.2 and 1.3 being both stable versions. And only these to be released on CPAN. 2/ About github : The problems we have currently is that : - the dev manager (sawyer for now I think - congrats) has to choose what to pick up from developers branches. He has to handle conflicts, test failing and so on, if the branch has been made long time ago - the developer is frustrated because he doesn't see his code merged in devel before long ( he's being said "after we release 1.2", "after advent calendar", "after we have talked about it", etc) To fix these issues, I propose that we keep devel and master as we have them, but add a new *experimental* branch. It's a big open sandbox : - this experimental branch is recreated from devel each time a new stable version is done, or when the dev manager thinks it's too messy - Dancer developers are trusted : they can push new branches and *merge* them into experimental, as long as they don't use rebase, and that the test suite passes - the dev manager picks up branches he likes from the experimental branch When new versions are released, the sandbox is cleaned up, experimental branch is wiped out and recreated from devel, and a next cycle can occur. Benefits : - developers can see their work in action in the code. Other dev will get the features immediately, they will react quickly : either they like it, or they don't, so they argue with the developer that merged the change. But they are not frustrated anymore - dev manager has less work to do regarding conflicts and merge issues, because developers take care of test passing when they merge - dev manager has an easy way to clean up things : wipe out experimental and recreate it Because we use git, developers don't loose their work when experimental is wiped out : they have their own branches. All they have to do is to rebase them on the new experimental branch, and fix tests / conflicts if any. This way of doing is from a simplified version of previous work experience. Just my 2 cents, may be useful for the discussion. Remember, developers freedom is important :) On 17 December 2010 11:10, Alexis Sukrieh <sukria@sukria.net> wrote:
Le 16/12/2010 20:16, sawyer x a écrit :
Our deprecation policy is to warn of deprecated behavior in the current
release version in which it was deprecated (1.2x, 1.3x, whatever), and die fatally in a following major release.
Indeed.
This could be added in Dancer 1.3x, including the deprecation warning.
Then removed entirely in 1.4x.
I agree, using the 1.3 releases serie for deprecating stuf inn 1.2 sounds good. Making them fatal error in 1.4 makes sense.
This strategy sounds good: release series with even number (1.2, 1.4, 1.6, ...) always contain stable features, intermediate series can deprecate things in the previous serie (1.3 can deprecate, with warnings, stuff in 1.2).
I like that.
1,2x was released only recently, so I'm not sure if we already want to
deprecate stuff for the big chunk of users we now added (thanks to many marketing efforts by the entire community).
As explained, I'd rather never deprecate anything in a stable serie, 1.2xxx is meant to provide bugfixes, documentation updates to the 1.2000 release, only. New features should be added to 1.3xxx in my opinion. And this is where deprecation can occur.
So:
1.2xxx => stable codebase, no deprecation, no new features 1.3xxx => development codebase, deprecation, new features
I welcome your points of view on this strategy.
This also reminds me we need a new permanent branch on the repo, like experiemntal, where we will build releases for 1.3
I use such a layout at work, and it works pretty well (with git-flow configured in two different ways):
2.0000 -[ master ]------------------------------|----------------------->
-[ devel ]--+----------------------------+----------------------> \ / \-[ topic/XXXXX ]--------/
This layout is good for maintaining one release cycle, but if we want to provide development releases in parallel (1.3xxx) we need one more branch. Otherwise it will be veru difficult to maintain the 1.2 serie.
So the layout could become somthing like this:
2.0000 -[ rel-stable ]-----------------------|----------------------->
-[ dev-stable ]--+----------------------------+----------------------> \ / \-[ topic/XXXXX ]--------/
2.000 ... 3.0001 -[ rel-unstable ]-----------------|------------|----------->
-[ dev-unstable ]--+----------------------------+----------------------> \ / \-[ topic/XXXXX ]--------/
The rel- branches are used by git-flow to produce releases, the dev- ones are used as integration branches.
My main concer is how can we publish those releases on CPAN...
Either way, there's no reason why not to start using this structure manually and implement the new strictures in a new branch to be merged on whichever release in which we want it. I'll try to get on it sometime soon.
Yes, that's another way of doing it, maybe more simple: use some parallel branches and merge them when we want, but I fear this will become very hard to maintain over time, if we want 1.2xxx to remain stable.
Feel free to coment on that idea.
Regards
-- Alexis Sukrieh
_______________________________________________ Dancer-users mailing list Dancer-users@perldancer.org http://www.backup-manager.org/cgi-bin/listinfo/dancer-users
On Fri, Dec 17, 2010 at 4:00 PM, damien krotkine <dkrotkine@gmail.com>wrote:
*top-posting*
OMG! :)
1/ About CPAN releases :
[...]
So I'm in favor for 1.2 and 1.3 being both stable versions. And only these to be released on CPAN.
Agreed! See my last reply on this subject (under the original topic).
2/ About github :
[...]
Still seems like tricky business to me. Of course, I could be wrong. I think that once we remove the major backlog we'll be able to clean up the pull requests and branches nicely. What I believe we need are reviewers. Basically nowadays I review everything and unofficially delegate things to Alexis, Franck, David, yourself, others. Most of these are either in inline responses to people (which isn't good because the intended reviewer could miss it) or on IRC/mail - not a very good way of doing it. No one is actually an official reviewer that knows that what they do. I think that if some people were just reviewers (even if it's just for specific stuff they know about), it'll be easier to skim through the pull requests and merge things better. Also, if we'll be able to add new stuff and deprecate old behavior in the same release series (see your number 1 above, "About CPAN releases"), it'll be MUCH easier to integrate stuff. I'm still waiting on the forward() feature, for example. Sawyer.
Hi, I don't have much time these days to follow everything that happens around Dancer and I must say I'm very glad of that, because it meas the project I've started in summer 2009 starts to grow by itself. As a project leader, nothing can be more pleasant. I take the opportunity to thank very much all the contributors that worked on Dancer, and more precisely the members of the core team who do a great job: SawyerX, Franck Cuny and David Precious. On the other hand, we have a very important choice to make, about how Dancer's developement and releases will be handled. I'm not pleased with the startegy we have yet, because it's not working. Yes, it is *not*. First, what we have: Dancer 1.2xxx is the current "stable" release of Dancer, and we announced in the first calendar article we will _provide stability support_ for it. We want to provide new features, refactoring and experimental stuff in the 1.3xxx series. If we choose to do that, I'm sorry to say we have no choice but creating a new branch to reflect changes that will have to occur in the 1.2 serie. Let me take an example: Let's say someone has 1.2003 in production, they ar happy with it and want to make sure nothing breaks in their app. So they trusted us when we said 1.2xxx will remain stable (only bugfix updates). Then, we upload 1.3001 to CPAN. OK, now imagine a critical bug is found in 1.2003. ... How do we fx it in GitHub? Our master branch now reflects the 1.3001 release. That sucks. We have to create a branch from the 1.2003 tag, maybe "1.2-maint" and start producing bugfix release from there. ==> My point is that we CANNOT provide stability release for 1.2 WITHOUT this new branch. It's just impossible. If we solve that, we also have a situtation regarding CPAN uploads: if 1.3001 is uploaded on CPAN, how do I publish 1.2003 releases? Again, all this situation is there because we said publicly in the first place that 1.2xxx will be maintained for stability. I think it's a good idea to do that, it shows that Dancer takes production matters seriously, but if we want to do that, we have to understand all that it means. Furthermore, I think I have a good overview of the situation here, because my job at @paidwork was exactly that: providing a set of tools and startegy for release cycle with maintenance branches, based on git + git-flow. So I think I see all the implications. If we want to do that, the git solution is the following layout: master ------------> last release (1.3001) devel ------------> integration branch for building master releases(see git-flow) topic/* -----------> temporary features branch for devel (see git-flow) This layout works great for one release cycle, but as explained before, we want to provide 2. So we should add: maint/1.2 ----------> originate from the tag 1.2003 and contains new releases of 1.2 This branch only contains merge of topic branches that are bug fix or updates for the 1.2 serie. They are also backported in devel as well. Trust me, this works and is not as complicated as you may think in the first place, with a core-team that know it well. Remember we have GitHub pull-requests with us, for handling contributions (this just concerns the job of Dancer's integrators, namely the core-team). As the project-leader of Dancer, I'm saying all this not to be the "bad-guy in the party" or anything, but just because I really appreciate all the work we did together, and I don't want us to go in a wall with a mess in our development process. And I think that we are, if we dont take good decisions now. I hope things are clearer now, and that you get my point of view. Note that I don't have a solution for CPAN uploads, but dams told me the Catalyst team has a similar concern, maybe we should look at how they do. Best Regards, -- Alexis Sukrieh
On 18 December 2010 12:31, Alexis Sukrieh <sukria@sukria.net> wrote: On the other hand, we have a very important choice to make, about how
Dancer's developement and releases will be handled. I'm not pleased with the startegy we have yet, because it's not working. Yes, it is *not*.
First, what we have:
Dancer 1.2xxx is the current "stable" release of Dancer, and we announced in the first calendar article we will _provide stability support_ for it.
We want to provide new features, refactoring and experimental stuff in the 1.3xxx series.
If we choose to do that, I'm sorry to say we have no choice but creating a new branch to reflect changes that will have to occur in the 1.2 serie.
I think everybody agree about that. A branch is created from the master release, to apply bug fixes and release minor versions.
Let me take an example:
Let's say someone has 1.2003 in production, they ar happy with it and want to make sure nothing breaks in their app. So they trusted us when we said 1.2xxx will remain stable (only bugfix updates).
Then, we upload 1.3001 to CPAN.
OK, now imagine a critical bug is found in 1.2003.
...
How do we fx it in GitHub? Our master branch now reflects the 1.3001 release. That sucks. We have to create a branch from the 1.2003 tag, maybe "1.2-maint" and start producing bugfix release from there.
==> My point is that we CANNOT provide stability release for 1.2 WITHOUT this new branch. It's just impossible.
Agreed, but again, I think there is nothing wrong about creating a new branch from 1.2. I think it's a good thing. I am completely happy with this model as long as the branch is created *from* 1.2 to apply maintenance fixes. It's different from having an other unlimited "branch" without starting point, like "unstable-dev".
If we solve that, we also have a situtation regarding CPAN uploads: if 1.3001 is uploaded on CPAN, how do I publish 1.2003 releases?
I don't think there is a
Again, all this situation is there because we said publicly in the first place that 1.2xxx will be maintained for stability. I think it's a good idea to do that, it shows that Dancer takes production matters seriously, but if we want to do that, we have to understand all that it means.
Furthermore, I think I have a good overview of the situation here, because my job at @paidwork was exactly that: providing a set of tools and startegy for release cycle with maintenance branches, based on git + git-flow. So I think I see all the implications.
If we want to do that, the git solution is the following layout:
master ------------> last release (1.3001)
devel ------------> integration branch for building master releases(see git-flow)
topic/* -----------> temporary features branch for devel (see git-flow)
This layout works great for one release cycle, but as explained before, we want to provide 2.
So we should add:
maint/1.2 ----------> originate from the tag 1.2003 and contains new releases of 1.2 This branch only contains merge of topic branches that are bug fix or updates for the 1.2 serie. They are also backported in devel as well.
Trust me, this works and is not as complicated as you may think in the first place, with a core-team that know it well.
Remember we have GitHub pull-requests with us, for handling contributions (this just concerns the job of Dancer's integrators, namely the core-team).
As the project-leader of Dancer, I'm saying all this not to be the "bad-guy in the party" or anything, but just because I really appreciate all the work we did together, and I don't want us to go in a wall with a mess in our development process.
And I think that we are, if we dont take good decisions now.
I hope things are clearer now, and that you get my point of view.
Note that I don't have a solution for CPAN uploads, but dams told me the Catalyst team has a similar concern, maybe we should look at how they do.
Best Regards,
-- Alexis Sukrieh
Sorry, message was sent before completion... Following up : On 18 December 2010 15:14, damien krotkine <dkrotkine@gmail.com> wrote:
On 18 December 2010 12:31, Alexis Sukrieh <sukria@sukria.net> wrote:
If we solve that, we also have a situtation regarding CPAN uploads: if 1.3001 is uploaded on CPAN, how do I publish 1.2003 releases?
I don't think there is any CPAN issue with that : the catalyst CPAN versions mix 5.8 and 5.7 (maint. versions) altogether. If you have 1.3 out, and need to fix 1.2, branch on github from master/tag-1.2 and release on CPAN 1.201 You may need to release 1.301 as well, so that even when sorted by date, 1.301 comes first. But you can definitely mix 1.2xx and 1.3xx on CPAN As for the special development release on CPAN, I don't see any benefit from using it, but maybe there is ?
If we want to do that, the git solution is the following layout:
*snip snap*
I hope things are clearer now, and that you get my point of view.
It's very clear, and I fully agree and support the workflow you presents. And if we have to hack gitflow or speed up to implement github, let's do that :)
Note that I don't have a solution for CPAN uploads, but dams told me the Catalyst team has a similar concern, maybe we should look at how they do.
We can definitely ask them. I'll ask t0m, he owes me some beers :) dams
I am pretty new to github and many other things, but I think it would be a good idea if the conclusion would find its way on the github wiki together with a few words on how to create a pull request. If there is a conclusion already. On Sat, Dec 18, 2010 at 9:23 AM, damien krotkine <dkrotkine@gmail.com> wrote:
Sorry, message was sent before completion... Following up :
On 18 December 2010 15:14, damien krotkine <dkrotkine@gmail.com> wrote:
On 18 December 2010 12:31, Alexis Sukrieh <sukria@sukria.net> wrote:
If we solve that, we also have a situtation regarding CPAN uploads: if 1.3001 is uploaded on CPAN, how do I publish 1.2003 releases?
I don't think there is any CPAN issue with that : the catalyst CPAN versions mix 5.8 and 5.7 (maint. versions) altogether. If you have 1.3 out, and need to fix 1.2, branch on github from master/tag-1.2 and release on CPAN 1.201 You may need to release 1.301 as well, so that even when sorted by date, 1.301 comes first. But you can definitely mix 1.2xx and 1.3xx on CPAN As for the special development release on CPAN, I don't see any benefit from using it, but maybe there is ?
If we want to do that, the git solution is the following layout:
*snip snap*
I hope things are clearer now, and that you get my point of view.
It's very clear, and I fully agree and support the workflow you presents. And if we have to hack gitflow or speed up to implement github, let's do that :)
Note that I don't have a solution for CPAN uploads, but dams told me the Catalyst team has a similar concern, maybe we should look at how they do.
We can definitely ask them. I'll ask t0m, he owes me some beers :) dams
_______________________________________________ Dancer-users mailing list Dancer-users@perldancer.org http://www.backup-manager.org/cgi-bin/listinfo/dancer-users
Le 18/12/2010 15:40, Maurice Mengel a écrit :
I am pretty new to github and many other things, but I think it would be a good idea if the conclusion would find its way on the github wiki together with a few words on how to create a pull request.
If there is a conclusion already.
The conclusions are summarized in the Development.pod file: https://github.com/sukria/Dancer/blob/devel/lib/Dancer/Development.pod Regards, -- Alexis Sukrieh
On 18 December 2010 10:55, sawyer x <xsawyerx@gmail.com> wrote:
On Fri, Dec 17, 2010 at 4:00 PM, damien krotkine <dkrotkine@gmail.com>wrote:
*top-posting*
OMG! :)
OMGZ!!!!111oneone :)
1/ About CPAN releases :
[...]
So I'm in favor for 1.2 and 1.3 being both stable versions. And only these to be released on CPAN.
Agreed! See my last reply on this subject (under the original topic).
I've checked on CPAN and it seems that it's not a problem to release 1.201 after 1.3 has been released. If you install a module from CPAN, you'll get the highest version by default. But if you ask CPAN the list of versions, you'll get them sorted by dates, which would work fine for us I think.
2/ About github :
[...]
Still seems like tricky business to me. Of course, I could be wrong. I think that once we remove the major backlog we'll be able to clean up the pull requests and branches nicely.
What I believe we need are reviewers. Basically nowadays I review everything and unofficially delegate things to Alexis, Franck, David, yourself, others. Most of these are either in inline responses to people (which isn't good because the intended reviewer could miss it) or on IRC/mail - not a very good way of doing it.
No one is actually an official reviewer that knows that what they do.
I agree to the extend that I think there are now 4 people (members of the core team) which are more and more seen as reviewer/integrators from the community, and that's good. Maybe there is a need of a bit more time and some communication on the ml and in the dev documentation, to make it more clear.
I think that if some people were just reviewers (even if it's just for specific stuff they know about), it'll be easier to skim through the pull requests and merge things better.
Also, if we'll be able to add new stuff and deprecate old behavior in the same release series (see your number 1 above, "About CPAN releases"), it'll be MUCH easier to integrate stuff. I'm still waiting on the forward() feature, for example.
Well, my opinion is that it's important to be strict with regards to versions : 1.2xx should be API compatible, as you said. So you can't deprecate and add features in these versions. However, maybe we need a shorter release cycle. While discussing on IRC, I mentioned one way of doing it : the release manager would aim at a time frame (say, a release every month). Nothing strict, but people know that they have to hurry to cleanup their code if they want their feature in the next release. And if their feature missed it, it's potentially only 1 month to wait, which is short enough. So they get motivated. Currently, there is no time frame, so you're not motivated to finish your feature, and you're a bit in the dark to know when a given feature will be included (like "forward"). That said, the release manager should have total control, and be able to say "it's too unstable, we'll release later", or "I remove these features, they are not good enough" About my "experimental" branch, after some discussion on IRC, I think it has be proven to be a bad idea. dams
participants (4)
-
Alexis Sukrieh -
damien krotkine -
Maurice Mengel -
sawyer x