Mik Kersten's Posts

Proposal to move Hudson to Eclipse

Wednesday, May 4th, 2011

Some of the most successful open source projects have histories that transcend organizational boundaries. My first experience with this was AspectJ, which we launched as an independent open source portal out of Xerox PARC in 2000. In 2003 our DARPA funding dried up, but the user community was still growing. We moved the project to Eclipse, the leadership moved from PARC to IBM, then to SpringSource. Not one of the original committers remains. But Eclipse has allowed the project to thrive throughout these waves of change in commercial interests, community leadership and intellectual property (IP) ownership.

Today Oracle proposed to move Hudson to Eclipse. As a board member and long-time committer, I have an inherent bias towards Eclipse being a great place to grow frameworks and tools. But I also believe Eclipse’s track record is a strong indication of the foundation’s effectiveness at combining the interests of multiple vendors and community of plug-in builders and contributors, to the net benefit of all involved.

Oracle owns the Hudson IP which they acquired via Sun’s initial investment in the project. IP ownership is a key factor that drives companies to innovate, in open or source and otherwise. Open source projects additionally need governance that combines the interests of vendors and of the community. In moving the IP and governance of Hudson to Eclipse, Oracle has done the right thing for the long term success of this very popular Continuous Integration (CI) tool.

Jenkins exercised the very important open source community right to fork, but in the process split the community. I in no way want to diminish what Kohsuke Kawaguchi created, and I have a deep and personal appreciation for the labour of love that open source projects like this require. But FUD ensued around the state of CI, and today’s announcement of moving the project to a neutral body marks major progress.

Consider the alternatives. As we learned with the rapid exodus off CruiseControl to Hudson, CI tools are a well understood space and easy enough to migrate between. If the differences between Hudson and Jenkins had grown sufficiently large and there was overall confusion and friction among the developer and corporate communities, this would have increased the demand for a new CI solution.

At Tasktop we follow the Application Lifecycle Management (ALM) stack needs of our customers, integrating open source, legacy and enterprise ALM tools rather than pushing a single stack. Since the announcement of the fork, we have been witnessing our customers’ frustration from the lack of a clear path forward from the current fragmentation and from the fear of downstream incompatibilities, or of betting on the wrong horse. While we are happy seeing a heterogeneous and best-of-breed ALM ecosystem thrive, we are less happy about all of the duplicated effort this would involve, especially since there is so much work left to do on REST APIs of Hudson and its plug-ins, as well as in providing the IDE integration and ALM traceability needed to make Hudson a key component in modern ALM stacks. It would be counter-productive to split efforts in evolving the CI interoperability layer that we have been creating in Mylyn, which enables both IDE integration and traceability across builds, source code and tasks. Eclipse is a tried-and-true place to evolve this level of tool support around ALM tools such as Hudson, and we are looking forward to collaborating around the convergent evolution of Mylyn, Hudson and Git/EGit and other key ALM technologies.

While there may be many questions about this move, the proposal phase of the Eclipse Development Process makes the path forward clear. The next stage is soliciting input from the community-at-large. As I see Eclipse as a great home for this technology, I have agreed to mentor the project and look forward to the community discussions around this proposal and the increasingly central role of continuous integration in the ALM stack.

Watch Tasktop webinars

Prediction #1: Task-focused collaboration transforms knowledge work, starting with developers

Thursday, April 28th, 2011

With this final and belated prediction in the series, I have taken the liberty of looking beyond 2011 speculating on the coming decade. I’ll start with a trip down memory lane in order describe how we got into this mess of information overload and what we can do about it.

It all goes back to files. The concept of a file is one of the simplest and most widely used abstractions in user interface design. Files are how we share documents, source code, and online content. Part of the success of UNIX and C that started in 1969, and lives on today in Linux, came from the elegance that files provided to the programming model. Those of us who started our careers programming C may still be amazed at seeing UNIX-style file paths now engrained in pop culture in the form of URLs. In addition to being a useful construct for programmers and operating systems, the notion of files became the primary way to organize information in the IDE, on the desktop, and on the web.

The problem with files is that we have way too many of them. In 1981, the graphical user interface was created at Xerox PARC in order to help us manage large volumes of files more easily than we could on in the filing cabinet or command-line interface. That same desktop metaphor was imitated first by the Mac, then by Windows. With a few decades of Moore’s law driving graphics performance improvements the windowed desktop has become a lot prettier. But the window-based desktops have experienced only incremental improvements, and the amount of information that we now need to process has exploded. The document hierarchies that we access daily have grown to tens of thousands of files. For developers, tens of thousands of files turned into hundreds of thousands. Yet the file and folder metaphor for browsing them remained the same. We were spending more time hunting for and clicking for information than creating it. The desktop metaphor collapsed under the weight of all of those files.

In late 1990s search-based navigation started getting embedded into both our desktop and online information retrieval experience. We were no longer bound to constant browsing for files through deep hierarchies or Yahoo-style categorizations of tagged content. Computers, network infrastructure, indexing and ranking technologies had become fast enough to place at our fingertips the thousands of files on our desktop, millions of classes accessible in our IDEs, and billions of pages on the Internet. When you consider the sheer speed of access that we have via Google’s Instant Search or Eclipse’s Open Type, this is nothing short of remarkable. But then we started seeing a familiar pattern emerging again: we were spending more time searching for information than we were spending on the collaborative and creative endeavours that are productive and rewarding. The repeated, wasteful clicking and keystrokes used to constantly browse and re-browse through information had now turned into repeated, wasteful finding and re-finding for the same information. We increased the volume of information that we could work with effectively by an order of magnitude or two, but the level of information relevant to our day-to-day work grew at an even higher rate.

The information overload problem that we are facing goes back to this notion of files that we have been relying on so heavily. Files encapsulate content. We tag them, bookmark them, index them and search them. Some of us are filers and obsess with having our content neatly organized and desktops de-cluttered. Most of us are pilers, with our computers’ desktops as cluttered as our physical ones. Either way, we are all playing a losing battle. The fundamental problem is that no matter how we organize and structure our documents and source code, our daily tasks crosscut that structure. We need a new approach to managing knowledge work in order to survive the ongoing deluge of information and increased demands for collaboration. The graphical user interface and search-based navigation will continue to be a key part of user interface design, but improvements on that front will only provide diminishing returns.

Tasktop task structure

The primary unit of knowledge work is the task. Tasks have a structure that is defined by the process that we use in our knowledge production activities. For software, this could be an Agile process, with tasks being recorded in a project tracking tool and taking the form of user stories and defects. For the sales part of the organization, tasks are captured as opportunities in Salesforce. The game changer created by the task-focused interface [http://en.wikipedia.org/wiki/Task-focused_interface] is to leverage these entrenched tracking tools to make tasks the primary anchor for the conversation and context around knowledge work. This concept means that the primary way that we access the information in the IDE and desktop user interface becomes the task, not the file. All conversation is available through a single point of access, no matter how many different repositories, email, or social communication channels it appears on. Even more importantly, all knowledge accessed and created as part of the task is tracked automatically, meaning that we offload our brains and let the computer do the work of ranking the information that’s relevant to what we are doing. Multi-tasking becomes a one-click operation, meaning that we don’t have to pay the price of context switching in the presence of constant interruptions. The near-term benefits of this approach are the reduction of information overload and multitasking overhead. The long-term benefits are a new level of knowledge capture and sharing that will eventually transform the productivity of the knowledge worker by an order of magnitude.

In 2011, tasks will be well on their way to becoming an entrenched unit of abstraction in the software development of leading IT-centric organizations. Every change made to a system will be automatically linked back to the task that it originated from as task activation becomes an integral part of developer workflow. This task-level traceability will be connected to continuous integration systems, which will position tasks as the primary unit of production in the continuous integration loop. This will dramatically improve development transparency across heterogenous ALM stacks, and plug Agile project tracking into the development loop, while empowering developers with the tools that they need to manage changes in very complex systems.

In 2012, this automatic capture of context will be leveraged by other stakeholders in the software delivery loop, including support staff, DevOps and QA. Stakeholder-specific context tracking will tie together the activities of developers, operations and quality assurance engineers; this will help to unify all of the participants in the deployment, diagnosis, and development of an application. Materializing a release-specific workspace to investigate a defect in a deployed application, along with all of the context from the application operators and help desk staff, will become as easy as clicking the “activate” button on a task.

2013 will bring tasks-focused collaboration higher up the management stack. Requirements tracking, project, and portfolio management tools will become directly linked to development activities happening at the production level. The production level of software development activity will finally have a high-fidelity link to the planning level, allowing for a new level of predictability and productivity in the software lifecycle. In their support of this movement, ALM tools will learn to speak the language of development tasks and then get out of the developer’s way.

What follows in 2014 is an infection to other parts of the enterprise. With project and portfolio management activities planned via tasks, email will finally start being displaced by task-focused collaboration as the mechanism for tracking work. One-click multitaskingâ„¢, task-focused collaboration and automatic knowledge capture will start to be tailored to the other departments in the organization.

In 2015, the single integrated Task List and task-focused interface will layer so seamlessly over desktop and mobile operating systems that consumers will start seeing the same level of productivity and collaboration benefits. By 2020, it will be effortless to assign your grandmother a task to bake you cookies, no matter where she is located, and have the recipe automatically captured as part of the context in order to later delegate the same task to your robot assistant before flying to work in your car.

As William Gibson stated, “the future is already here – it’s just not very evenly distributed”. In writing this blog series, every bit of content that I produced and accessed has been automatically captured as part of my task context, and each of the many revisions and bits of feedback were tracked on the task. When I re-activated this task after a series of interruptions, my desktop-based Word and web-based WordPress editors were instantly restored. In the same way, every change that any committer has made in the Eclipse Mylyn codebase is automatically linked to the Bugzilla bug describing the social conversation around the change, then linked to the corresponding build and release. Tasks and contexts externalize our memory and make it much easier for both our forgetful selves and for newcomers to help us evolve our software.

To my surprise and concern, Tasktop is telling me that I have spent over 88 hours researching, collaborating on, and writing this prediction series on Agile, ALM and developer tools. All of this information was automatically captured and will be accessible to me in 2015 when I attempt to explain away any inaccuracies in my predictions.

“The best way to predict the future is to invent it” said Alan Kay at Xerox PARC in 1971, inspiring Smalltalk and paving the road for the graphical user interface. Our vision at Tasktop, as embodied in the “less is more” logo , is to transform knowledge work around the focus and flow required to get creative work done in the presence of ever-growing volumes of infromation. As with other innovations, like the file hierarchy and desktop metaphor, this move to task-focused collaboration will originate from developers and move upwards and outwards in the organization. We look forward to working with you in making this vision a reality, starting with the transformation of the Agile ALM stack and developer tools. <=>

Watch Tasktop webinars

Interview on ALM getting a bad name, how Agile and open source are changing that

Friday, April 1st, 2011

Here is a short interview from The Server Side conference, with Jan Stafford of Software Quality Insights, on how ALM got a bad name from heavyweight methodologies like RUP, and why the move to Agile, open source and developer-centric tools is changing that. Yes, I am in fact holding an invisible crystal ball, which is the very same ball that I’ve been using for my Agile/ALM/dev predictions series. It has been malfunctioning for the past couple of weeks, but has started working again, so stay posted for prediction #1.

Watch Tasktop webinars

Tasktop 2.0 released with Eclipse Mylyn 3.5, brings sanity to your heterogeneous ALM stack

Tuesday, March 22nd, 2011

If you are involved in large-scale software development, you may be alarmed to hear leading analysts discussing the growing number of Agile rollouts that are failing. Several factors are contributing to this. Legacy and in-house ALM tools have proven to be sticky and hard to migrate from. Their content, customizations and workflows encapsulate product history. Agile tools have moved in, but have not been adequately connected to developer workflow or to enterprise ALM tools and processes. Lightweight open source ALM tools are becoming favoured by developers and being adopted in a bottom-up fashion, but these are disconnected from both enterprise and Agile ALM tools. The resulting difficulty of modernizing and integrating the increasingly heterogeneous ALM stack is a key cause of the failing Agile deployments and ALM stack modernization efforts.

For the past three years, the Tasktop 1.x train for releases was focused on providing developers with the integrations that they need to work with Agile and ALM tools within their home environment, the IDE. Our goal is to bring the productivity benefits of the task-focused collaboration paradigm to as many developers as possible. On the Mylyn side, this has meant collaborating with Agile and ALM vendors and contributors in order to grow the frameworks that have become the most widely extended APIs for ALM integration. With last year’s restructuring of Mylyn to become a top level Eclipse project, these frameworks grew to encompass SCM tools, Build and Continuous Integration tools and code reviews. The Tasktop 2.0 release extends this ecosystem with new Mylyn connectors and new features that support the range of our integrations, including Task Federationâ„¢ and cross-repository Agile planning.

New Mylyn Connectors

For the 1.x release, our obsession with improving the developer’s workday meant providing integration with the huge breadth of ALM tools in use today. This meant getting involved with each of the corresponding open source projects and partnering with the best-of-breed Agile and enterprise ALM vendors in order to enhance their web service APIs and establish the long-term support needed for this level of interoperability. The figure below highlights the new integrations made available as part of Tasktop 2.0 in green, with the most notable being:

  greenbullet_icon HP Agile Accelerator (Tasktop Enterprise): This builds on our existing support for HP Quality Center and ALM in order to provide both the task management and IDE-based Agile planning facilities. Learn more.
  greenbullet_icon Accept360 (Tasktop Enterprise): Brings requirements and other product management artifacts to the developer’s desktop in order to better connect development to planning activities. Learn more.
  greenbullet_icon SmartBear CodeCollaborator (Tasktop Enterprise): This is the first full-featured Mylyn connector for a code review tool, SmartBear’s popular CodeCollaborator. Learn more.
  greenbullet_icon Polarion ALM (Tasktop Pro): Polarion is the vendor behind the Eclipse Subversive SVN client project and we now have full support for the Track & Wiki and ALM solutions. Learn more.
  greenbullet_icon Hudson/Jenkins (Eclipse Mylyn): For the Mylyn 3.5 release we created a Hudson integration that layers on top of Mylyn’s new Builds framework. On top of this Tasktop provides traceability with tasks that exist in ALM tools such as HP Quality Center. Learn more.

Tasktop for Visual Studio

Many of our customers have been asking for support for their teams and colleagues who are based in Visual Studio. Organizations are building both Java and .NET based applications that need common lifecycle management tools. The Tasktop 2.0 release includes the GA of Tasktop for Visual Studio, which brings the ecosystem of our task connectors into the Visual Studio IDE. Whereas previously only basic integration existed for task management within Visual Studio beyond Microsoft’s TFS product, you now get Mylyn’s functionality including instant opening of tasks, offline access, the Task List and a full-featured and WPF-based task editor.

  greenbullet_icon Supported Connectors (Tasktop Certified): Accept360, Atlassian JIRA, CollabNet TeamForge & ScrumWorks, Edgewall Trac, Google Gmail, HP Quality Center & ALM, IBM Rational ClearQuest, Microsoft Outlook, Microsoft TFS, Mozilla Bugzilla, ThoughtWorks Mingle, Polarion ALM, VersionOne. Adding support for the remaining Tasktop Certified connectors is underway.

Learn more about Tasktop for Visual Studio.

Task Federation, for Developers

The big news in Tasktop 2.0 is not the new connectors, but what we have created on top of this ecosystem in order to solve the problem of ALM stack heterogeneity. Tasktop 2.0 includes a Task Federation framework that provides a layer of insulation between the various stakeholders in the software lifecycle and the implementation details of the organization’s ALM stack. Task Federation manifests differently for each set of stakeholders.

Developers are often stuck working with multiple ALM tools. You may be working on two different projects, each of which has a different ALM tool. Or you may be adopting Agile, but still needing to use the company standard defect tracker. Those of us depending on open source frameworks and SDKs will have dependencies between our internal task and those managed in an external repository such as Eclipse.org or Apache. Tasktop 2.0 provides full support for linking dependencies between the various task repositories that define your software’s evolution, and new features such as cross-repository Agile planning and task synchronization build on this support.

Task Federation, for Agile Planning

The biggest single new feature in Tasktop 2.0 is the release of the Agile Planner that we launched as beta at the Agile 2010 conference. Just as we do not replace your existing ALM servers, we are not replacing your existing Agile planning tool. Instead, we augment it with our unique interoperability features. The Tasktop Planner provides:

  greenbullet_icon Task Federation: The most important feature of the planner is its ability to show plans that span repositories. Planning your user stories in an Agile tool but using HP Quality Center for defects or an open source issue tracker is no longer a problem. Depending on another team or an open source project? These dependencies now show right in your release plans since Tasktop can display and reconcile state across repositories. All of the cross-repository state is stored in your existing ALM tools as hyperlinks and metadata.
  greenbullet_icon IDE Integration: This is the first full-featured cross-repository Agile Planning tool within the IDE. You get a fully native and offline-capable Scrum style task board and a release planner. Relevant content from your web-based planning tool, such as burn down charts, is available when connected via our embedding of web UI gadgets. For product owners and project leads, the planner is also available in the Tasktop desktop edition.
  greenbullet_icon Developer Focus: Whereas most tools focus on project-level views, the Tasktop Planner supports developer centric workflows such as highlighting everything assigned to you for the current Sprint or populating your Task List with user stories assigned to you for an iteration. Everything you need for your daily Scrum or release planning is always at your fingertips.
  greenbullet_icon Automatic Time Tracking: The Tasktop Planner uses time values from Tasktop’s automatic time tracking facility to make Agile estimation dramatically more realistic
  greenbullet_icon Supported Tools: Agile plans can be displayed from HP’s Agile Accelerator and Rally. More tools will be supported in subsequent releases. All Tasktop Certified connectors are supported as targets for cross-repository plans.

Task Federation, for the ALM Tools Manager

In addition to providing developers and plan stakeholders with a unified view of the various tools and servers that define their planning activities, Task Federation can also be deployed to further unify an ALM stack. Say that you are deploying IBM Rational Team Concert (RTC) for Agile planning, but have realized that HP Quality Center (QC) is completely entrenched in your defect and quality management process. You can deploy a Task Federation setup that bi-directionally synchronizes all the key tasks and fields between RTC and QC so that RTC has the state needed for planning, while QC happily remains your system of record for quality management. Since each Tasktop client includes the full federation functionality and uses Mylyn’s hardened bi-directional task synchronization and conflict resolution interface, it is possible to deploy this solution in a scalable and secure fashion without requiring any new servers.

Learn more about Task Federation

Task-Focused Continuous Integration

As always, our commercial efforts in Tasktop have been driving framework and open source tool improvements in Eclipse Mylyn. The most notable of these is the new Hudson tooling, which also supports Jenkins. You can now work with Hudson day-to-day entirely from within the Eclipse IDE, monitor and run builds, and inspect results. Accessing build information is instant and available offline. One of the most useful automation features is the ability to run tests that failed on the server within your workspace, with just one click. But the most profound feature here is the first phase of our new support for ALM traceability. If your developers are using Tasktop, thanks to the magic of task activation and automatic change set tracking, you now have perfect traceability between your Hudson builds and the tasks that changed for a given build, providing an unprecedented level of traceability between your tasks, source and builds.

Learn more about Mylyn 3.5


With this release of Tasktop 2.0, we are taking a big step forward in bringing a new level of interoperability and sanity to your Agile deployment and ALM stack, helping developers focus on coding while giving teams a much more complete and real-time view of plans and progress.

  greenbullet_icon Read the Tasktop 2.0 New & Noteworthy
  greenbullet_icon Sign up for my webinar on Tasktop 2.0

Watch Tasktop webinars

Eclipse Foundation Board of Directors elections

Monday, March 7th, 2011

Voting for the Eclipse Foundation board member elections ends this Friday. If there is something to be learned from this year’s Hudson vs. Jenkins duel, it is the importance of the governance model of popular open source projects. The Eclipse Foundation’s governance has been effective, continues to evolve, and is not something that we should take for granted. As the technology and business landscape around Eclipse change, the leadership and direction of the platform and ecosystem must adapt. What makes Eclipse’s governance so unique is that it is designed to combine both commercial and community interests in the tools and frameworks hosted on Eclipse. Vendors are able to help direct Eclipse by voting with their membership level and with their election of sustaining member representatives to the board. Community members vote with their contributions, and have the opportunity to elect their representatives to the board. The result is nearly two dozen board members with a wide diversity of opinions, guided and reconciled by Mike Milinkovich and functioning as exemplary not-for-profit software organization that helps shape the broad innovation network that is Eclipse. My vision for Ecipse is included below, and those of the other candidates are listed on eclipse.org.


Eclipse has matured and cemented its role as a platform for innovation. As its Sustaining Members, we play a key role in driving the direction of Eclipse. Our contributions in the form of code, evangelism, as well as Eclipse-based products and services have made Eclipse the vibrant multi-vendor ecosystem that it is today.

In joining Eclipse as one of the first non-IBM committers, I have been engaged with both the technology community and commercialization aspects of Eclipse since the outset in 2001. For the past year I have been your representative on the Eclipse Board of directors, previously served as committer representative, lead the now top-level Mylyn project, and am currently serving on the Architecture and Planning Councils.

I have worked directly with many Solution Member companies and watched some Eclipse business models flourish while others have failed. Starting a profitable company around Eclipse has given me a pragmatic perspective on how to participate in the platform while growing both product and service revenues on Eclipse-based offerings. For the past year I served as your representative on the Eclipse board, and for this next year I plan on continuing to apply my experience and ongoing dialog with members in order to help make your Eclipse-based efforts successful.

Succeeding in Eclipse means striking a balance between the member, committer and user communities. As your board representative, my priorities will be to:

  greenbullet_icon Facilitate reaching users: Marketing in a vendor-neutral open source ecosystem can be far from obvious, and reaching the very broad Eclipse user base is needed for both the commercial success of members and of Eclipse as a whole. I have been closely involved in improving the install story for Eclipse solutions and the upcoming Eclipse Marketplace client. These efforts will provide the next generation of opportunities to get your solution into Eclipse users’ hands.
  greenbullet_icon Lower the barrier to engaging with projects: With product releases and other pressures, too often the step of getting directly involved with an Eclipse project that you build on does not make the cost/benefit cut. But not getting involved has the longer-term cost of incompatibilities and overhead. Lowering the bar for direct participation will provide members with better access to the collaboration and contributor community that make Eclipse a great open source ecosystem. On the Mylyn project we have had over 900 bugs and feature requests resolved by community contributions. I will use my experience of making this work for Mylyn to help lower the overhead of participation with existing projects so that similar benefits are accessible to more members.
  greenbullet_icon Listen to members: My job will be to represent your needs on the board, and I will work with the other representatives to actively seek out feedback from all Solution Members. I helped make the needs of individual committers heard as a committer representative on the Eclipse board, and will ensure that your needs are represented in discussions and decisions made at the board level.

We made good progress on these priorities in the past year, and I hope to continue representing you and helping you make the most out of your investment and engagement in Eclipse in 2011.

About the candidate

Dr. Mik Kersten is the CEO of Tasktop Technologies and lead of the Eclipse Mylyn project. Mik is a popular speaker on Eclipse at conferences in the United States, Germany, and worldwide. As a long-time Eclipse evangelist, Mik is a frequent source for media and analysts on matters related to Eclipse and developer tools. Highlights of Mik’s contributions to Eclipse include:

  greenbullet_icon Creator and lead of the Eclipse Mylyn top-level project (since 2005)
  greenbullet_icon Active member of the Eclipse Architecture Council (since 2007)
  greenbullet_icon Member of the Eclipse Planning Council (since 2010)
  greenbullet_icon Elected committer representative on the Eclipse Board of Directors in (2008/2009)
  greenbullet_icon Only Eclipse evangelist to be voted a JavaOne Rock Star (2008 and 2009)
  greenbullet_icon Only Eclipse content author on the top authors of the decade list of IBM developerWorks Java
  greenbullet_icon PhD in Computer Science from the University of British Columbia that lead to the invention of Mylyn’s task-focused interface (2006)
  greenbullet_icon Co-creator of the Eclipse AspectJ and AJDT projects (2002) based on his work creating of the first Aspect-Oriented Programming (AOP) tools, Xerox PARC (2000)

Related publications

  greenbullet_icon Symbian Blog: Mik Kersten on Transparency
  greenbullet_icon Tasktop Blog: Growing open source ecosystems: the install story
  greenbullet_icon Tasktop Blog: Tasktop working with Microsoft to improve Eclipse on Windows 7
  greenbullet_icon How Software is Built: Interview with Mik Kersten (highest ranked in series)
  greenbullet_icon Tasktop Blog: Tips on paying for free software
  greenbullet_icon Tasktop Blog: Platform for Innovation, Part 1: Openness & Modularity

Read more about Mik Kersten on Wikipedia.

Watch Tasktop webinars

Prediction #2: ALM tools become the gateway drug for hooking developers on cloud and PaaS

Thursday, March 3rd, 2011

The productization of DOS by Microsoft in the 1980s kicked off the trend of platform vendors clamouring for developer mindshare. Apple’s App Store may only generate 1% of the company’s profit, but it drives the platform that sells Apple hardware. One of the best examples of the value of a developer platform is Microsoft’s Visual Basic (VB), which is yet to see a rival in terms of breadth and pace of adoption. In 2010, VB ranked as the 5th most popular programming language on the Tiobe index, still well ahead of iOS’s Objective-C. Microsoft’s investment in developers goes well beyond paying homage to the company’s roots, since Microsoft knows the sheer number of Windows license sales that can be attributed to each developer that they won over with VB.

Developers drive the success of platforms. Whenever a platform shift occurs, such as the shift from proprietary to open source or from desktop to mobile, interest in gaining developer mindshare is renewed. While we may all be overloaded with atmospheric-condition-related acronyms, we are in the middle of a fundamental shift in enterprise software platforms. The VMware hypervisor has replaced Dell blade server as the hot new technology. Application architectures are being transformed by REST-based services. Over the course of the year, support for multi-tenanting Java applications will result in more significant changes to the programming model that any of the incremental features coming in Java 7. The shift to cloud and Platform as a Service (PaaS) has ramifications beyond the change in the deployment model. One of the most important changes coming down the pipeline is the convergence of open source, PaaS and ALM. The continuous integration and delivery loop, becoming increasingly popular in Agile deployments, will put a new set of requirements on both ALM tools and the connection between the running application and the developer’s workbench.

ALM Deployment Disconnect

All of this is good news for developers. There will be an increase in competition, choice, and budgets which are dedicated to bringing developers to new application hosting destinations and PaaS solutions. Over the course of the year, early adopters of PaaS will help to harden these new offerings in order to get them ready for the on-premise and private cloud primetime. If you are an early adopter and running a greenfield project, you’re bound to see some cool hosting and services thrown your way. But there are a few things you will need to watch out for when migrating to the bleeding edge. Beware the snake-oil pitch that moving your Java app into the cloud will be as easy as choosing a hosting provider. When migrating existing applications, you are likely to discover that achieving a scalable and cost effective architecture will involve significant efforts such as reconsidering your data storage, implementing multi-tenancy and porting the application to new ready-made services. This migration is similar to moving from a rural homestead to a downtown condo—it will take more than just a moving truck, your lifestyle will change, and you will spend less time worrying about your plumbing.

We have transitioned from thinking about applications as static entities that we build and deploy to living entities whose evolution is punctuated by releases and sprints. Given the turnkey automation promised by PaaS, developers will demand integration between the running application and the lifecycle tools used to manage it. This will drive one of the most important developer-centric trends in ALM, the convergence of application hosting and monitoring with ALM. The core ALM services of change management and continuous integration will become the hub between the developer’s desktop and the running application, with tasks as the unit of abstraction. An application failure or problem will automatically create a defect that includes the application state that is related to the problem encountered. The product owner will schedule that defect into the appropriate sprint, and when the developer picks it up, all of the code relevant to the defect will be instantly retrieved from the SCM system and brought into focus. The commit of the fix will automatically spin up a build in a staging environment, deploy it, and then pass it on to production or Ops when tests pass. This future needs a new level or coordination between the core services of the ALM stack and PaaS offerings.

Tasktop Continuous Deployment

Vendors that tie together the programming model, frameworks, virtualization, hosting and ALM will be the winners in the battle for developer mindshare. To capture a meaningful snapshot of a hosted application, the runtime must understand the programming model. To present this to the developer, the IDE tools must be capable of be effective in displaying that information alongside the code. And at each step of the way, it’s the ALM tools that will facilitate communication between the various stakeholders in the process: monitoring by Ops identifies a problem, that problem’s state is captured in the issue tracker, the issue pops up on the developer’s desktop, and then all subsequent changes are attached and captured on the corresponding task and reported in the build that was triggered. Workflows of this sort will require significant changes from ALM stacks, which have generally ignored the details of application build and deployment to date.

ALM services will also facilitate the migration of applications to the cloud. As we saw from the $212M acquisition of Heroku, supporting developer workflow provides a great on-ramp for application hosting. In the land of enterprise Java this is considerably more complex than a Git-based push and deploy of Ruby apps. But the need to bring this kind of simplicity to the Java application lifecycle is the same, and tasks provide the link necessary to scale continuous integration and delivery to large Java applications with a broader range of stakeholders ranging from dev, to Ops and QA.

Connecting the IDE to an ALM hub and using the hosted source and builds to drive deployment will provide a convenient on-ramp to application hosting offerings, and will significantly reduce the configuration and administrative burden currently placed on developers. While it can be fun to play around with configuring deployment environments and continuous integration servers, this level of busy work gets very tedious after a few months. A new level of automation is needed to help us focus on delivering application features and business value in the face of increasing platform complexity. Source, issue tracking and continuous integration will be a welcome gateway drug to cloud and PaaS application hosting. While having an app store claim 30% of all enterprise application revenues is not something we are likely to see this year, a Steve Jobs-like focus on turnkey simplicity of deployment and lifecycle management will help win the hearts and minds of developers making the move to the cloud.

Watch Tasktop webinars

Prediction #3: Git and distributed version control invade the enterprise, widespread panic and benefits follow

Wednesday, February 23rd, 2011

Three years ago, in his presentation on the Git Distributed Version Control Systems (DVCS), Linus Torvalds provoked developers by declaring that “if you actually like using CVS… you should be in some mental institution.” Since then, awareness of and discussion around both DVCS and mental health have grown rapidly. For many considering a migration to DVCS, discussions of trade-offs are filling coffee breaks. After the first migration steps are taken, the conversations get more heated and circle around getting DVCS best practices under control. For teams and organizations modernizing their ALM stack, this year will involve deciding whether to begin moving to a DVCS system. This post discusses the fundamental benefits that DVCS will bring to your Agile ALM stack, and identifies the gaps in tools and best practices that need to be addressed in order to avoid having the inmates get overly annoyed with the management of their asylum.

In 2009, the Eclipse Community Survey showed that 20% of respondents used CVS and 58% used Subversion (SVN), with CVS coming in at 24% for companies over 5,000 employees. While DVCS usage has been growing, the portion claimed by CVS and SVN has also grown in the 2010 survey. The mental model of CVS and SVN is entrenched for the average enterprise developer and builds on decades of the knowledge, tools and best practices that have built up around centralized version control. Tools built on top of CVS and SVN, such as the Eclipse and Mylyn team support, have also modernized these dated protocols and servers. But what the rapid rise of Git in open source has made clear is the limitations of traditional file-based change tracking and the centralized model for commit access control. The current adoption rate of Git is sufficient evidence that DVCS works and scales for open source development. The question then turns to how those benefits can be realized by your teams and organizations.

The key benefit of DVCS is that the collaboration rules and conventions captured by the version control system align with the trust network that forms organically between developers. The centralized version control model relies on component boundaries and restrictions on committer rights for managing the division of labour required for software configuration management (SCM) of complex systems. This perspective falls into the same trap as thinking that a modular, object-oriented decomposition of software is sufficient to manage the complexity of a large system. It is not. Dependency injection, aspect-oriented and declarative programming are key components of the modern programming model. In other words, concerns that crosscut the hierarchical decomposition of any large system need to be captured in order to evolve the implementation of that system effectively.

Tasks, whether in the form of user stories, features or defect fixes, are what define the planning and lifecycle management of large applications. Tasks crosscut the system’s modularity by capturing the discussions between developers and end users around the software, not the API and component boundaries of the software. Aggregations of tasks into user stories, plans and releases drive the evolution of the software over time and define the boundaries that naturally form between developers. Centralized version control systems have focused entirely on changes and components, and ignored this more social dimension of coding. Managing a large system meant breaking a system into components and managing access control with commit rights. This aligned version control with the system’s modularity, not with the way that developers work on the system in terms of experimenting with and collaborating on changes to the system. The crosscutting nature of developer collaboration and review activities needed to be captured and facilitated, and DVCS provides us just that.

In the video above, a visualization of directory structure corresponds to the modularity of the open Mylyn project. After watching the video for a few moments, we see that the activity of the developers themselves layers on top of the software’s modularity and has a structure of its own. This structure corresponds to the tasks, change sets and patch reviews done by the Mylyn committers. By facilitating the common operations needed to support this workflow between developers, such as task-based branching and merging, DVCS changes the game by making it easy for developers to capture their activity within the SCM system itself rather than tracking it through external tools, or not capturing it at all. By expressing the collaboration, review and trust patterns common with open source developers, DVCS has fundamentally transformed SCM.

That’s the good news. The bad news is that it will take the better part of this decade for many organizations to consume the benefits of DVCS. That timeframe may seem pessimistic given that some large organizations are already in the midst of deploying Git and Mercurial. But DVCS is a discontinuous innovation. It both provides a transformational benefit over centralized version control and requires a fundamental shift in developer practices that previously used a considerably simpler model. Consider the introduction of object-oriented programming (OOP). The core ideas were already baked in the 1970s, but it took decades and truckloads of books to turn the ideas into a scalable development practice in the enterprise. OOP introduced polymorphism and type hierarchies, neither of which supported the older style search and grep-based code navigation. A whole new breed of tool support needed to be created in order for developers to get a handle on navigating OOP system structure.

While the switch from centralized to distributed version control is a smaller paradigm shift than the switch from procedural to object-oriented programming, the scope of the change should not be underestimated. Problems will arise in three categories. First, the mental switch required from the average developer, accustomed to committing to the main line of the code with branches being few and far between, will be a source of friction and costly in terms of developer training. Second, tool support for discontinuous integrations tends to lag the initial deployments. For the near future, developers accustomed to full-featured graphical or IDE-based clients will find themselves back in the command-line when working with Git and Mercurial. Those with some muscle memory for command-line version control operations will still need to retrain their brains, since Git’s command line options are more complex. Finally, the biggest challenge organizations adopting DVCS in 2011 will face is a lack of clear best practices. If you are launching a new project with the social dynamics of the Linux Kernel, you’re all set and don’t need to look beyond the best practices that Linus has forged. But if you are a medium or large organization, you will need to define the various practices around branching, merging, rebasing, reviewing and staging that are needed to effectively map tools like Git onto your development processes. The higher complexity of DVCS introduces a broader variability of best practices. Today we are using different Git practices for developing the Mylyn and the Tasktop codebases.

Organizations pushing the bleeding edge of Git adoption will discover that these problems cascade. Neither the change in developer mind-set nor best practices are encapsulated by the Git tool support available today. This means that many developers will struggle to get a handle on needing to gain a sufficient level of Git expertise and best practices, and these will have to be learned individually rather than being captured in the repeatable process provided by tool support. That approach scales to a few dozen motivated developers, but much less so with a few thousand. The top technical people at these organizations will invariably push for DVCS migration, provided they see the fundamental productivity benefits as attainable. But until DVCS is further along the technology adoption lifecycle, for most organizations the move must be made stepwise, starting with non-core and greenfield projects establishing the best practices and conventions. On Tasktop and Mylyn development we have migrated 1/5th and 1/3rd of the codebases respectively, allowing ourselves time to refine the best practices suitable for each of those, and add the EGit support needed for our developers and contributors to be productive.

By architecting version control around the social structure of the software development process, distributed version control has aligned the Agile and task-focused planning process with the versioning and change operations that we use to manage our software’s history. Since both the task-focused and the DVCS movement have focused on the natural cadence of coding and developer collaboration, there has been convergent evolution between tools like Mylyn and Git. In the not-too distant future, the developer will activate a task, the appropriate Gerrit change or topic branch will be created, stashed when the task is deactivated, and a continuous integration build for the topic branch spun up, with failures percolated back to the developer’s desktop as updates to the task. As DVCS best practices evolve, we will see more of this natural mirroring between the Agile planning loop and the DVCS mechanisms converging, lined up with tool-supported best practices.

Successful open source projects have a community and contribution driven planning loop, and can already reap the benefits of DVCS. But the enterprise software planning loop is more complex. In his dogmatic focus on open source software, when proclaiming that everyone using centralized version control was insane, Linus ignored the work required to bring the benefits of open source technologies to the software development community at large. The average developer’s training needs to be accounted for, best practices for incorporating DVCS into the planning loop must be defined, and tool-based automation is needed to bridge the gaps.

Watch Tasktop webinars

Prediction #4: The single vendor ALM stack becomes extinct in organizations with more than two developers

Monday, February 14th, 2011

Development managers at large organizations with monolithic application lifecycle management (ALM) stacks once had it good. ALM components were well integrated, played nicely with one another, and when they didn’t, there was someone to call. But lightweight issue trackers started to move into the organization, popularized by the need for developer-centric collaboration facilities. At a cheap or free price point, these were often deployed without the corporate IT department’s knowledge or approval. Then the Agile project tracking tools moved in, and with the excitement around Agile, team and product leads pushed them through without consideration of standardizing across the organization. In the meantime, quality management tools like HP Quality Center remained so entrenched in their domain and so valued by management that they did not budge when the newer tools moved in. As a result, the large organization’s tool stack resembles an ALM history museum, ranging from the old in-house defect tracker that’s still running, to the freshly-installed Kanban tools intended to address disillusionment with Scrum.

No two ALM stacks look alike. The success of open-source ALM components has driven diversity into even the smallest software shops, and the new norm for the ALM tool stack is heterogeneity. While this brings the benefits of choosing your best-of-breed solution, it also means dealing with new complexity in terms of navigating the landscape of tools and options when considering how to modernize your ALM stack to support Agile. This post is a brief guide to helping you navigate that complexity by focusing on the options you have for modernizing the most core part of your ALM stack: the issue, project and change management layers. Tool support on this front boils down to capturing different stages and granularity levels of software development tasks, much as Salesforce has succeeded at capturing and tracking tasks across the various stages and stakeholders in the sales process. At the base level, we have tasks related to the code itself, consisting of defects, feature requests and tests. This is the realm of the issue trackers. At the next level, tools abstract over development-specific items to capture tasks relevant to management and planning, such as user stories and requirements. The new breed of Agile tools is popular here. Then, there are the enterprise ALM tools that can capture and track across products, releases and portfolios of projects.

In 2010, some ALM surveys included over 100 different issue trackers and change management tools to choose from. This enormous diversity has driven innovation, as vendors strive to differentiate their offerings in order to win over customers and to add value over the steadily rising bar of commoditization coming from open source. Last year we saw the price points of basic issue tracker support for small teams drop to dumping levels in an effort by vendors to displace their competition. In terms of feature sets, issue trackers are starting to look very much alike and, as with cars, their most visible differences are colour combinations and dashboard designs. In 2011, the issue tracker will be well on its way to becoming a commodity. The key thing to look for is that your issue tracker supports developer collaboration along with good ease-of-use for comments and updates, and that it is integrated with the workflow of the developer. The issue tracker also needs to be integrated with your planning loop. If you are a small team or organization, that may be as simple as managing a field with your release or iteration or setting up a wiki-based Kanban board with hyperlinks to the issues. But once you start scaling beyond a small team, tool-based planning features are required to scale your planning process across teams and geographical locations.

The next layer in the ALM stack is the project planning and tracking tool. With Agile starting its move into the enterprise, competition between vendors in this area is becoming fierce. Numerous ISVs are clamouring to lead the trends, implementing the latest Agile and Lean fashions as quickly as the books and blogs defining them are published. A few vendors, including Rally, VersionOne and ThoughtWorks Studios, have differentiated themselves by combining Agile tools with thought leadership and training. This helps organizations adopt the cultural aspects of Agile while bringing the vendor’s tool-based support for Agile practices along for the ride. These vendors have been key to many of the Agile rollouts to date, and have driven much of the management level adoption of Agile and Lean methodologies. New players have appeared in this space, introduce a wide range of expertise, and provide a spectrum of tools from simple layering of functionality on top of the issue tracker to full-blown enterprise ALM solutions. Picking the best solution here often depends on the size of your development organizations, how opinionated you want your Agile tool to be in terms of enforcing a particular Agile workflow, and selecting a tool that integrates with the rest of your diverse ALM stack. The latter often proves to be the greatest challenge.

If your development process needs to plug into a project, product and portfolio management loop, and also requires a connection to quality and requirements management, you need to reach one more level up the stack into the realm of enterprise ALM. Just as lighter-weight tools offer increasing benefit the closer you get to the developer’s desktop, the visibility, predictability and planning needs of a large organization with thousands of developers are addressed by these more heavyweight enterprise ALM tools. When you have a team of five developers, requirements management can often be handled by a shared understanding of the customer and the problem space. When you have a team of five thousand developers, many of whom are remote or outsourced, it’s almost impossible to be effective at managing delivery without this level of tool support. For large organizations the crux of the problem tends to be that the open source, lightweight issue trackers and Agile tools they have deployed do not provide this level of ALM support. This is the reason why enterprise ALM tools such as IBM Rational Team Concert (RTC), Microsoft Team Foundation Server, and more recently HP ALM, are providing support for the entire change management cycle. The trouble with getting the full benefits of those tools is, once again, the heterogeneity in your stack.

At each of these levels of the ALM stack, the key is ensuring that in the presence of heterogeneity, the code being produced and deployed from the development level is connected to the goals and product strategy determined at the planning and management levels. Failing to automate the connectivity between these layers means that the organization falls back to less reliable communication formats such as excessively long email threads, more tedious meetings, and the wearing new channels into the office carpets as you walk back and forth between cubicles.

What’s needed to bring sanity to ALM stacks is a “task federation” layer. At small organizations, the top-most system of record for the planning loop becomes the issue tracker. At medium-sized organizations it’s the Agile project tracking tool. Issue tracking facilities in Agile planning tools will increasingly displace the standalone tracker, or in the cases where the issue tracker is sticky or provides additional value, a task federation layer will provide linking between the tracker of choice and the Agile planning tool. At large organizations, task federation will become a critical part of the planning loop. For example, if you are deploying IBM RTC as your ALM tool you probably have HP Quality Center deployed already and do not want to lose its benefits for quality management. Task federation provides you with the bi-directional synchronization of tasks between the two systems, while ensuring that a tool like RTC has all the information needed for planning and that the quality management tool is the system of record for tests and defects. Or you could deploy HP ALM’s Agile features in combination with open source issue tracking and change management. What’s important is that task federation provides you with the options needed to modernize and streamline your best-of-breed stack, while ensuring that development is connected to your ALM solution of choice. While no two ALM stacks are alike, in 2011 we will start seeing the developer and manager stakeholders insulated from the intricate details of ALM stack implementation, with ISVs taking on the burden of integration.

Watch Tasktop webinars

Prediction #5: Open source ALM tools continue to gain market share, give the development manager a migraine

Tuesday, February 8th, 2011

The influence of open source on software development is often measured by the impact of successful libraries and frameworks. It’s hard to imagine building a modern web application without open source components. A similar trend is now unfolding in the Application Lifecycle Management (ALM) space, driven by tools created by projects needing to support their own open source software delivery. While ALM tools are often associated with the heavyweight workflow characteristics of enterprise application development, successful open source projects are a great example of the transformation underway in ALM, towards lean methods and lightweight, developer-centric tools.

In contrast with the application development tools which we use for writing and debugging code, ALM tools assist us with an application’s evolution over time. At their core, ALM tools track tasks and changes, help manage builds and releases, and support the dialogue that defines an application’s evolution. This subset of ALM is sometimes referred to as Application Development Management (ADM). On top of this core feature set layer tools for project and product management. Large organizations add additional tools to the stack to handle program and project portfolio management.

Thanks to a combination of resource constraints, a preference for using open source technologies for open development, and the common desire for developers to sharpen and extend their own toolset, the past decade has delivered a critical mass of the open-source ALM tools. Consider the scale of ALM on Eclipse.org: 33M lines of code in the last coordinated release (733 installable features/components), 330K total Bugzilla reports (3K new each month), 15K Bugzilla users, 1K committers (400 active), 300 projects and 173 Hudson build jobs. Add to that dozens of interdependencies between Eclipse projects and other open source projects such as the popular Apache libraries. ALM on Eclipse is managed entirely with open source tools including Bugzilla, CVS, Git, Hudson, MediaWiki and Mylyn.

The 1,948 respondents to the 2010 Eclipse Community Survey provide an overview of the degree to which open source tools have percolated into commercial software development. Only a small fraction of the survey respondents were directly involved with Eclipse, and half were from organizations with over 100 employees. The striking pattern is that the core open source ALM tools, when combined, have the market lead in each of three key ALM categories visible in the figure below. In 2010 for these categories, open-source ALM has taken the market lead from closed-source solutions. While surveys of this sort are always skewed towards the type of developer who bothers to answer surveys, this result remains indicative of a shift in application development practices and open-source ALM market share. In 2011, I predict that this trend will continue and that open source tools will percolate into the ALM stacks of more conservative organizations. A degree or two of separation from their open source counterparts, many of those developers will not recognize the term DVCS before it is force fed to them.

The attraction to open source ALM is not just price point, but the amount of innovation that has been driven by open source developers building tools to support their own productivity patterns. The ecosystem of extensions that forms around popular open source projects is another key driver of adoption. Those ecosystems are also likely to produce the most interesting integrations, since open source has proven itself as the most effective mechanism for growing both the community and APIs needed for innovative extensions. Finally, organizations with long application and product lifecycles are attracted to open source ALM tools because a suitable open source license gives them confidence that they will be able to access and extend the knowledge base represented by their issue tracker ten years from now, when the landscape of ALM vendors will much different than it does today.

Open source ALM tools are built on developer-centric principles. Transparency is favoured over hierarchy, with every task and bug on Eclipse and Mozilla being editable by anyone. It relies on asynchronous collaboration and a consistent use of the task or issue tracker to capture all discussion relevant to changes in the software. It encourages lining up modularity and framework boundaries with team boundaries, allowing API layers to facilitate dependencies and collaboration. There are also things missing from the stack. Responsiveness to the community often takes precedence over planning, and after the fading away of XPlanner, there has been a distinct gap in project management features within the open source tool space. There is also no single integrated open source ALM stack, instead open source projects glue together their own best of breed solutions, and layer customizations on top, as is the case with the numerous tools built on the Eclipse Bugzilla repository. Integration with product, project and portfolio management tools is typically non-existent, as this is not something that even large open source projects need.

While open-source developers will continue working very happily with their increasingly slick tool set, this impedance mismatch with large-scale ALM implies major problems for organizations who are planning to get a lot of ALM functionality for free. There mismatch between both the toolset and the cultural aspects of open source ALM tools and what’s needed by the enterprise. Agile and lean development have the opportunity to bridge some of the cultural gap, but still have a considerable way to go in order to incorporate the lessons learned from open source. There is enough of a gap in the toolset that organizations already deploying open-source tools at the enterprise ALM scale have needed to set up their own ALM tool engineering teams. These teams create enterprise-level authentication and access control, provide third-party ALM tool integrations, and implement support for features such as linking existing Project Portfolio Management (PPM) tools. Due to the pace of change in open source ALM tools, they are fighting a losing battle. While wasteful, this exercise is currently necessary. Large organizations that fail to integrate already deployed open source tools into their existing ALM and PPM infrastructure will see a dramatic reduction in the predictability of their development process, since their process relies on a connectivity between development and planning tools that was present in the more traditional ALM tool stack.

There is hope. First, the benefits of open-source ALM tools are fundamental as the ease with which they allow developers to work makes us happier and more productive. The velocity of successful open-source projects demonstrates how good these tools are at supporting the delivery of high-value software that is responsive to the needs of the community and customer. On the flipside, Enterprise ALM tools provide management and planning facilities which are critical for predictability of delivery as well as the longer-term planning that is necessary for organizations to thrive. These two worlds must be integrated into a cohesive whole, especially as more Agile teams find themselves at the intersection of open source and enterprise ALM.

After I presented my keynote on open source ALM at W-JAX last November, a colleague from one of the world’s largest software companies commented that the same developers that worked on open-source projects were twice as productive as when they worked on closed source projects. We discussed the usual issues of motivation and incentive structure, but nailed down the key issue being the sheer friction generated by traditional ALM tools which has been removed from open source ALM tools. It is time to reconnect the software planning process to a new breed of open source ALM tools that support lean and high velocity delivery, connect them to the planning and management tools needed for software delivery at scale, and bring some of the benefits of open source development to the enterprise.

Watch Tasktop webinars

Prediction #6: Continuous integration becomes central to deployment, Jenkins attacks Hudson with a chicken

Thursday, February 3rd, 2011

How does a continuous integration (CI) tool named after a butler or two grab such a large market share when much more feature-rich and polished commercial counterparts exist? The naïve answer is that it’s free. If you dig a bit deeper, the success of Hudson can be seen as part of a larger trend in developer-centric application lifecycle tools. A growing number of open source tools have hit the mark in capturing developer collaboration and Agile lifecycle management practices. What has caused these tools to snowball in popularity is the rich ecosystems of extensions that they support. That combination has become lucrative to larger organizations wishing to increase the productivity of their developers. When an open source community starts smelling of money-making potential, a different breed of dog moves in. This can be a very good thing, as it is often the catalyst needed to move the project across the chasm to broad industry adoption.

The challenge for popular open source projects is to create a governance model that marries the interests of the project’s community with those of vendors supporting the project. The most successful open source projects are ones that manage this dichotomy without alienating either party. Spring, JBoss and MySQL won by having a single dedicated vendor managing commercial and community interests. Eclipse and Apache have created governance models that support the overlapping interests of multiple vendors. In the case of Hudson, with Kohsuke’s departure from Oracle, a split formed. In recent blog posts we’ve seen Kohsuke Kawaguchi, creator of Hudson, highlighting the community and the ecosystem of extensions that have made Hudson successful to date. On the public forums, Oracle’s Ted Farrell has emphasized the importance of versioning, consistency and stability, which are relevant for taking Hudson to the next level of enterprise adoption. Both of these concerns need to be addressed.

Over the coming year, extensible continuous integration is going to play a key role in both on-premise ALM stack modernizations and cloud-hosted ALM solutions. If you have a few grey hairs on your head, this may not sound new or noteworthy. A decade ago, we set up a CI build for AspectJ.org with CruiseControl and Ant and it is still running today. More modern Mylyn CI builds are very similar, but running Hudson and Tycho. What’s interesting is that over the past decade the continuous integration loop has become the underpinning of build automation in the Agile development process. Easy extensibility has made CI servers a convenient hub for plugging a variety of ALM tools into the Agile build loop. We are moving away from a world where realses, code metrics, testing tools and deployment destinations are being configured by each developer within his or her IDE. Instead, the CI server is becoming the hub that holds the authoritative build specs, with developers attaching to the portion of the ALM artefacts that they need to work on for the current release. Testing, quality, and metrics tools will increasingly use the CI server as the place to hook into the development process. In an upcoming prediction, I’ll discuss how combining task-focused workflow with continuous integration will prove to be a convenient abstraction for tool support that facilitates knowledge capture and sharing between Planning, Dev, Ops and QA.

All of this potential means that Hudson is a hot topic, and that Hudson’s consumers are scratching their heads on what to do about the Jenkins fork. Hudson’s success to date comes from the typical mix of passionate community leadership, open source extensibility and a vendor-sponsored marketing investment that helped create that community. The latter is often omitted from developers’ discussions, but now that we are in the later stages of open source maturity, a project’s access to marketing resources has become a key component to success. A case in point is the significant marketing budget of successful open source foundations, or that of the Mozilla Foundation’s, which is in the millions.

In open source, community-centric passions drive projects to their initial critical. Then comes the point at which enterprise support, stability, and the ability for other vendors to invest in a project become relevant. These help a project to graduate from an early adopter community to the pragmatists who can establish an open source project as a de facto standard. For that to happen, a project must be bigger than the cult of personality around its founder, just as any organization must be bigger than its leader to thrive. But with the heart of the contributor community in his hands, and his brain more wired to the technical vision of the project than any other person’s on the planet, the founder is often required to drive major downstream innovations.

In the year ahead, the existing consumers of Hudson will vote with their feet whether to go with the forked Jenkins codebase or with Hudson. The question will largely boil down to how stable the core of Hudson is, and whether upcoming changes to the core of Hudson that happen within Jenkins, and its key extensions, warrant a migration. In the closer-knit community there will be a tendency to Jenkins, as represented two hundred community votes, the majority of which were in favour of the fork. But in the much larger end-user community the default will be resistance to change in the absence of major and clear value add to making the move. We’ve seen this on Eclipse with the overly slow migration of many plug-ins to e4, which brings incremental value add and only benefit.

The large and more conservative consumers of Hudson will look to which corporate sponsor, Oracle for Hudson and the CloudBees startup for Jenkins, is providing better long-term assurance for their build infrastructure investment. There will be questions about Jenkins’ promise of neutrality. In the past, genuine open source project neutrality has only been achieved by large open source foundations with sufficiently diverse funding sources, and even then with mixed success. As such, while Jenkins is likely to see activity, Oracle’s Hudson is unlikely to go away and I predict that it will continue to be a major target of CI deployments through the year.

In the interim, as with the Emacs vs. XEmacs fork of the 90s, there will be some confusion and annoyance from those less interested in the drama. Thankfully John Ferguson Smart wrote his Continuous Integration with Hudson book in open source fashion, making it easier to refactor. If the divergence between the two projects is sufficient to fragment and fracture the community, there could be a large enough gap for a new open source CI system to join the scene, but it would take time for it to build up momentum. In the meantime, the usual API-level federation layer is likely to form so that plug-ins can support both tools, and we’re invariably going to be asked to create one in Mylyn. Most of the internet associates the name Jenkins with a certain Leeroy who went into a World of Warcraft battle wielding little more than an attitude and a chicken. I hope that in this battle less damage is done to the consumers of this otherwise winning CI technology.

Watch Tasktop webinars