Friday, January 25, 2008

What to do?

Grant Goodyear had a pretty good blog post summarizing the current situation from his perspective. I am glad that he is diving in and tackling Foundation issues, and I'm also glad to see that the Gentoo releng team, Donnie Berkholz and others are stepping forward to improve the quality and quantity of communication emanating from the Gentoo project as well as Gentoo community interaction as a whole. Those are all good signs, and I hope they are signs of long-term positive change.

My perspective at this stage is that there is effort being made to address some of the serious issues surrounding Gentoo.

Now, what about my offer? I made my offer to return, primarily to run the Foundation and provide some direction and a more community-oriented angle for the project, and it wasn't accepted, or at the very least not accepted in the time frame I specified. However, now there are people working to address the very concerns that led to my offer in the first place. So I'm quite satisfied with how things are playing out.

I should also mention this - I care about Gentoo, but I did not look forward to returning to the project to clean up various messes. It's just something I felt obligated to do if no one else was going to do it. So I do appreciate those who are doing the work that I would have otherwise been doing myself.

I also realize that many Gentoo users would like me to return to lead the project. I realize that there is a lot of energy out there in the Gentoo user community and a desire for change, for Gentoo to have a rebirth of sorts, and frankly I understand and in many ways share that desire. This isn't a desire for me to be in control - it's just a desire for Gentoo to be more than it is right now.

And now we come to the title of this blog post - what to do? What is the next step?

On one hand, the Gentoo project has recently been held accountable by its user community in a pretty remarkable way. I think this is healthy and commendable. And now, we are seeing that developers on the Gentoo project are making efforts to be more accountable to their user community. I think the proper response to this is to be supportive and encouraging, and yes, even genuinely appreciative of these efforts by the Gentoo project, and encourage this process to continue.

On the other hand, many users are wanting major change, not just little incremental improvements here and there. They're fed up. They want to clean some house, or maybe fork the project. The perspective of this group is that we can do better - way better - than what's been the norm over the last few years. They have basically given up on Gentoo as it currently exists and they are tremendously motivated for some kind of regeneration or even a revolution.

I have thought about this carefully, and this is what I would like to do. Well, first, here's what I don't want to do.

I really don't want to be part of the official Gentoo project if I can avoid it, whether as a developer or in some kind of leadership role. It's not something I would enjoy long-term. Yes, there are things I like about the project but there are still too many things I don't. There are things I could try to fix as a part of the official team but I am not convinced it would be very productive or a place that I would really fit in. In other words, I'm not a big fan of the current culture. However, I support the recent signs of change with Gentoo and hope that they will continue.

I also don't want to fork the project and create a "new" Gentoo, and try to convince everyone to join my project and use my new improved version. The fork option only really appeals to me in a petty and wrong way, so it is clearly not the right thing to do. Too much time would be spent on organizational issues rather than actually improving things technically and from a community perspective, so it would be wrong in a couple of ways. And it would split the community into two rival camps, and it could get ugly at times. Some things could get better, but some other things could get quite a bit worse.

But what I do want to do is to see Gentoo get better. I actually think I can do a lot more by working independently on some technical and community initiatives to benefit the larger Gentoo community.

That will probably involve me importing the Portage tree into a git, mercurial or bazaar repository sometime soon, not for the purposes of forking but for allowing me and others to work on improving Gentoo more easily and independently from the main project. The goal is to improve the technology and improve collaboration in the larger Gentoo community, a vastly under-tapped resource. This is an effort of collaboration, not competition, and one that does not exclude the official Gentoo project either.

I think this will result in much more positive change than a fork. And I'm at peace and comfortable with this direction. I'll share more about these efforts in future posts. Right now, I need to talk with a bunch more people.

30 comments:

mvolat said...

I agree with you, Daniel : this seems to be a very reasonnable path to follow.

As it was discussed in your previous entry, a fork just for the sake of revenge would not have been a good thing. And even so, I feel that too much people call for the fork as a magic solution, but by putting too much hope in it, I fear that you would have to face their disappointment -- and many critics.

It's good to see that you haven't given up with gentoo too, I must admit that if I were you, I surely would have lost all will to work on this project.

Concerning your work on portage, this is something I really look forward to. I was septic on the paludis project even if a C++ version of portage coded from scratch looked good in theory. Let's see what you can bring us :)

Anonymous said...

hell, that would be awesome daniel :D
i think you have measured everything properly.
i'm really anxious to see open minded people to work again with passion and innovate for the good of gentoo =D
I dont want to push you or anything but... start asap setting everything up so we can begin the development! XD

Matthew said...

Thank you for updating the community on not just your decisions, but the reasons for those decisions. I know many of us (users) greatly appreciate the supportive attitude you've assumed toward every contingent (devs, users who want a fork, users who want you to clean gentoo house yourself, users/devs who want you to stay away). While I would have liked to see a Gentoo Gen2 (I know, not terribly clever) fork, and still would, your points are well taken and I imagine the current direction you've chosen will benefit the community as a whole.

Thanks again.

Anonymous said...

Well said.

Tavis said...

Daniel,
Thank you for bringing these issues to a head and helping the developers, trustees and the broader userbase focus on fixing them.

I think you've chosen the right path. The status quo would have assured Gentoo's continued slow decline, but a fork would have been much worse. Furthermore, I suspect it would have cost you a lot personally.

It seems like the project's decision making process is deadlocked at times and that it's too difficult for users to make small contributions. A distributed portage repository (mercurial's my pref) would help, but it needs to be coupled with a clear path for getting local changes tested and then pulled into the official repository.

Another area that needs improvement is the updating of stability keywords for non-core ebuilds, particularly for those that few other packages depend on. This seems to stall frequently. I think it would be worthwhile implementing a mechanism that allows registered users to vote for a particular ebuild being marked stable on their platform. Bugs are usually reported for unstable ebuilds, but I don't see any way for users to tell others that they've tested an ebuild and found it stable. I'm imagining a command like 'ebuild vote-stable'.

These votes could then be queried by tools like equery and eix. We should also be able to list/query bug reports and bug confirmations related to a particular package/ebuild via a command line tool.

Having this information accessible on the command line during the installation process would make it much easier for users to choose which version of a package they should install.

A similar mechanism could be used to vote for a third party overlay to be included in the core repository.

Tavis Rudd

shrewd.user said...

i don't think a fork would be the worst thing in the world, look how it turned out for compiz / beryl.

I'm a mere gentoo user and i don't really understand the politics behind it, but i agree with your sentiments, the politics and processes and leadership needs to be as open and transparent as the source code.

gentoofan23 said...

Daniel, I heartily agree that forking Gentoo would be irresponsible in many ways.However, I especially like the idea of copying the portage tree to a git(i like subversion better though) repository and working through there. I would definitely help, as the reason I am not exactly a gentoo dev is that CVS is too slow for my net connection.

Thanks,
Thomas

Anonymous said...

Daniel, I think you know best for you. At present I can't fault your logic.

I personally am somewhat let down, even as you say many more are motivated the old attitudes still prevail. I'm still stinging from the last time a dev told me in response to a bug report I filed 'get over it and move on'.

A piece of me got ripped out that day and the very thought of your return kindled hope.

Maybe I'll get what I need not just the way I expected it.

Thanks Daniel for sticking your neck out and asking to take time from work if necessary. These are revolutionary times and I am proud to call you a revolutionary.

All The Best

Anonymous said...

What to do next? I have an idea although it's too general.

I always thought there must be something beyond portage and Gentoo; something better.

I don't think Linux distributions today work as a whole like an organism. They are rather disconnected pieces that are supposed to work together and they do up to a point but things eventually get messy and break trying to maintain the distro.

There must be something that works in a more integrated fashion like the parts of a cell do in biology.

Maybe you can call it the Cell distro.

I don't have ideas what it is but I know there must be something better. Maybe something that integrates smaller pieces of the software rather than the packages as they written now. Most of the software and packages have too much in common. And no I'm not talking about libraries that provide common code although that could be part of it.

Of course this is way beyond package management and has more to do with the theoretical relationships of the code.

mikecore said...

You are a better man than I am. I would have forked. But again I also understand you have a life outside of linux that is probably more important then Gentoo or a fork. Good luck with everything. I on the
other hand stated I would not come back unless you did ( leadership wise ) and I meant that. one last time thank you again!

oli said...

Very wise spoken...Thank's for your great work in the past, present and future!!!!!!!!! Best Regards from Germany!!!! Oli

Rudi said...

Thanks, Daniel! That is exactly what I was hoping for you to do.
I'm glad you didn't listen to the fanatics.

Menelkir said...

Hi Man

I agree with you in parts.

The fork isn't a direct solution, I know. But in this actual situation, gentoo will die or will be more elitist than ever.

There's a lot of things to do, and no one cares because "I am a leet kid and I know what I do.. oh! its Daniel! you're not welcome here", this is ridiculous. This is not gentoo. This is a bunch of 12 year minded leet-kids trying to be more "The one".

If you make a fork, I'm with you, and not just me, you see in the other post, there's a lot of people that can help, that want help and aren't happy with actual situation.

If you take the leadership (not probably) we also be with you, because there's a lot of things to do, and a lot of mess to clean. For example, look at livecd, this shit works just 20% of time and no one fix the thing yet.

lucre said...

Great first step for bringing out the quality that is embedded in gentoo.

Kevin said...

I suppose we all have to learn the lesson eventually - I learned it in a different field of endeavor. The lesson is that setting up political processes invariably results in unresponsive behavior and a tendency for people to sit in sinecures. I saw it coming when you left Gentoo initially and it's proven out in practice.

Bottom line is that I think I have to learn another distro and move in that direction since I have no confidence in the leadership of Gentoo to keep the distro on the straight and narrow.

I really appreciate all your work on Gentoo. Discovering it six years ago was a magic moment and during your tenure and for a considerable time afterward I was nothing but pleased with the distribution. I learned a lot about Linux also.

Thank you, Daniel.

seek said...

I was shocked to hear that Gentoo still uses Subversion for the first time. Switching to a distributed CMS will increase the amound of collaboration in Gentoo, as well as the overall efficiency.

Linus Torvalds gave a great presentation about Git at google techtalks. He did not only describe Git there, but also the way people work on the Linux kernel. It would be great to see this kind of collaboration in Gentoo too.

thomasn said...

What absolutely excellent news — you have absolutely made my day. Thank you so much for this decision: I was anticipating with some dread either a fork, with all its associated inefficiencies and and rigid polarisations, or a choice by you to move on and leave Gentoo behind, which would (from my extremely limited understanding of the situation) not bring any noticeable benefit to the Gentoo community. If you and apparently-level-headed individuals like Grant Goodyear can help involve a wider group of contributors this will surely be to everyone's benefit.

Suggestions: please do consider bzr as an alternative to git/hg - useful comparison of git, hg and bzr if you're interested.

And might it be possible to build bridges between official Gentoo docs and the wiki? At the moment it's easy enough to update the wiki (surprise!) but much more time is required to contribute anything, even typo fixes, to official docs.

Looking forward to another happy 5 years of using Gentoo!

-- Thomas.

Anonymous said...

I'm disappointed, but I respect your decision. I think Gentoo will go back to the trajectory of slow decay it was on before you took an interest.

Thank you for your efforts. The last few weeks have been the best for Gentoo in the last year.

Anonymous said...

I call shenanigans, that these changes had nothing to do with your posts. They may have been on a path but I think you lit a bit of a fire under some rears.

I would like to see a fork, but one purely aimed at hardened-enterprise, with stability in mind. I know myself and quite a few others could care less about bleeding edge, but stability and security.

Additionally I hate the idea of overlays, what kind of QA am I to expect from one? It's like rpmfind all over again.

ColdWind said...

This was discusses some time ago, here's a GLEP draft about it:
http://dev.gentoo.org/~antarus/projects/gleps/glep-0666.txt

Anonymous said...

Daniel - you might give thought to talking to the dev community regarding your ideas. I think that most devs would be supportive of them. There are a couple of existing initiatives like sunrise and the various overlays that TRY to do what I think you intend - maybe the time has come to rethink the whole approach.

Perhaps we need multiple tiers of QA. A core of officially-maintained packages similar to what is in portage now. They could be depended on to be around, and they don't get marked stable until a maintainer and an arch team agree they're ready for mass use. Then you could have something more distributed for all those niche apps that it is hard to find maintainers for. People could essentially vote on how stable they are, and maybe a web-of-trust of some sort could be used to automatically accept upgrades on packages.

I find that as an arch maintainer I'm always hammered with stable requests for packages that are very difficult to adequately test. I genuinely want to try to get them stabilized, but I don't want to break systems because of inadequate testing. Consider packages like dialup network access - I don't even have a modem on any boxes running gentoo - and I don't just want to deploy an upgrade that could sever somebody else's only connection to the net (which would be a pain for them to fix).

Again, I think that most devs would work with you on some of your ideas. Sure, you can just do them on your own, and you might get some official blessing. But maybe bounce some of your ideas off the -dev or -project mailing lists and at least see what ideas come up. Worst case you can just ignore all the replies, but maybe somebody will give you some good advice.

Most devs aren't blind to gentoo's shortcomings. However, many of these issues aren't trivial to fix, and nobody wants to sink the ship while trying to redesign it. Ideas are always welcome!

Anonymous said...

Daniel,

thanks!

PS. git!

Anonymous said...

What "forking a project means" if not importing to another repository and starting work on it with another team of developers? I feel a bit lost here. I think that it is a kind of a psychological game "I did not want to fork but I was eventually forced to". Do not hesitate to do the right thing and start creating a new distro no matter if you start from the current Gentoo or from scratch. Competition is not a bad thing and currently Gentoo is the only project that addresses those people who want a source distribution or to be more precise a metadistribution or a system builder (OK the only one that works well). There is enough room for another competing project in this area. More over situation is healthy only if there are at least 2 competing products to choose from. Actually you have done an excellent job creating Gentoo so I believe you are able to do that again and there is a good chance it will be even better as you have much more experience now. And if current Gentoo has to face a real competition it will be forced to improve too. The worst thing you can do is doing nothing as your potential will be lost for the open source community.

Edomaur said...

I'd like to see Mercurial used as a distributed source manager, but it's probably because I use it daily and find it really usefull and very fast.

There is also this idea : what if Gentoo use hg or bzr instead of rsync to update the Portage tree ?

Anonymous said...

if i'm not mistaken, gentoo has its roots in freebsd, and, in my humble opinion, that's a very good thing; what about making "ebuilds" resemble *more* the freebsd ports? the KISS axiom should rule everything, starting with the most important thing in a linux distro: the package management!
i think a "make"-style descentralized ports system (eventually backed up by something like sqlite for the USE-flags and other user "tunable" things) could make a big difference!
is it possible to see something like this in your fork, Daniel?
if the answer is YES i think you got my whole attention!

Anonymous said...

I was the one who proposed a fully new perception on Gentoo development with the business as a strong point in the comments of one of the last blog entries.

With your last blog entries you built up hope in the community, Mr Robbins. Many users were looking to some possible, interesting future.

Now, I am - and I think many community members are also - very disappointed that you just want to change a source code control system (from svn to git/bzr/whatever) for the portage tree. What should this solve, please?

Maybe it could solve a development problem in the "small".

But it will never solve the Gentoo problems "in the marketplace": You know, I am speaking of the marketplace of free - as in speak - distributions as they are also in competition for devels, users, some spotlight in the press and so on.
Gentoo is bad here, as many distros are getting much more users than Gentoo.

And many free distros are today also in the business space: They are used for really important tasks.
Although Gentoo has some prof. users out there I never read that Gentoo is seriously supported by some company - not even those small 1 to 3 employee companies supprt Gentoo -.

You, Sir, had the opportunity to change all that.

But with using git instead of SCCS
you will not solve all those "big" problems.

I am very disappointed. :-(((


(With all respect, Mr Robbins, but if these were (all) the changes you spoke of in your last blog entries on which the hopes of the community built up I now think it is better that you don't fork Gentoo.)

Thanks for reading my comment.

Js said...

Hi Daniel,

That was one of the gustiest moves I ever saw.
Unfortunately all what has happened is just some water being pumped out of the haul of this sinking ship...
Only the fools stay on board which also has its benefits I guess:) Most unfortunate, what a waste..

Good Luck in future endeavors !

me & the other reasonable people.

Anonymous said...

Daniel,

As much as I appreciate everything you have done and said I still feel Gentoo has now entered a descending spiral that will eventually see it joining many other Linux distro's that have come and gone. How can anyone honestly trust the Gentoo board and the up-start developers now after everything that has happened?

Unfortunately people have long term memories and it will take years before the Gentoo community will be able to convince people to come back if at all. It's just human nature.

There are distro's out there such as Ubuntu that have stepped up to the mark and proven they can produce a quality product and provide good community support to assist those who have issues. While Ubuntu was making strides, Gentoo was too concerned about destroying itself by not listening to the community anymore. In the end everything you worked hard to produced has slowly been destroyed and I very much doubt it will ever recover.

The only real chance Gentoo had was by YOU coming back. So many people believe in you and what your capable of doing. They do not believe in the current board or most of the developers. As much as they may hate to hear it the facts remain that Gentoo is dwindling away because of the mess it's currently in which has been caused by the board and developers. I have NO faith in the current board regardless of what they say they will do. We have all heard it too many times in the past.

I have found Ubuntu to be a MUCH better distro than Gentoo (in it's current state) and will never go back. And believe me, I'm not the only person who thinks like this. Gentoo has lost many users, particularly those who were around from the beginning.

Anyway all the best with your own endeavors. I still hope that one day you will either fork Gentoo or start a new distro not out of spite but for the very same reasons you started Gentoo in the first place :)

Daniel Robbins said...

Misc comments to various people:

Well, we will see how Gentoo does. I'll reiterate that I think we need to encourage the good things that are now being done to improve Gentoo, and let the people who are doing them know that their work is appreciated. Whether it is enough for you, for others, or for changing the trajectory of the distribution as a whole is something I don't know. But I remain optimistic.

Also, there are some commenters who seem to have misunderstood my mentioning of git/mercurial/bzr to mean that I have a srategy of using or promoting these tools, and by doing so I magically expect something with Gentoo to get fixed. That was not what I was trying to say. I'm going to be looking at how to collaborate with various projects and I'm hoping that the fruits of our collaboration will make a positive difference for Gentoo. git/mercurial/bzr (and our own repository) are simply tools that would allowing this collaboration to happen more easily.

Another person didn't understand how cloning the portage tree is not a fork. Well, for all I know, some of our work might be accepted upstream into the official Portage tree. Even if it is not, there several Gentoo-based projects that already exist. The point I was making is that I would rather collaborate and support these projects than create a project of my own. And these Gentoo-based projects are not considered forks, but rather a realization of the Gentoo meta-distribution concept.

The main point is that I am hoping to find ways to work with the larger Gentoo community.

d2_racing said...

Hi Daniel, first I want to thank you for what you have done with Gentoo.I hope that you will have fun an another distribution and maybe you will endup with Sabayon :)

Good luck.

For thoses who don't know, there a news on distrowatch about that : http://lists.sabayonlinux.org/pipermail/devel-sabayonlinux.org/2008-January/000094.html