Tasktop

Having just returned from the fantastic ALM Summit in Redmond it is even clearer to me that there is a lot to talk about when discussing ALM. The event in Redmond is focused on the Microsoft platform, but the discussions were far broader. Technology impacts such as cloud, mobile and open source coupled with process changes driven by Agile and Lean Startup mean the very fabric of ALM is changing. The announcement of TFS and GIT working in perfect harmony is illustrative of this shift. But even after 3 busy days there is still lots to talk about, which brings me to the ALM Connect, an event that myself, the Eclipse foundation and a team of great people have been working on. The event, which is scheduled for March 25th through 28th in Boston MA has all the ingredients for an amazing conference.

Deep Dive Into Content

All the sessions in the program are great, but I would like to draw your attention to a few that caught my eye during the call for papers.

• Moving towards ALM 3.0 by Forrester Analyst Jeffrey Hammond. Not only is Jeffrey a great speaker, but always fills his presentation with lots of data that you can use back in the office. In this talk Jeffery is highlighting the major shifts in the fabric of ALM. Looking at the data this talk is based on we might see a redefinition of the ALM category, which is very exciting!
• What ALM knowledge you can expect from CS graduates by Gary Pollice, professor at WPI. The emerging skills crisis in software engineering is going to affect us all so I am excited to hear how you can better hire CS graduates and make them more productive. This talk also helps to remind us that ALM is more than just tools, but includes processes and people.
• Continuous Integration at Google Scale by John Micco, Google. CI has emerged as the lifeblood of modern software delivery, and on paper seems easy, but for the majority of large organizations with complex builds, heavy dependencies and nasty test environments building a workable CI environment is difficult. In this talk John describes how a very complex CI environment can be built and maintained in a very changeable business.
• Building Mylyn 4.0 by Mik Kersten, CEO of Tasktop. Mylyn has defined how Eclipse developers interface with systems of record such as bug trackers and project manager tools, but the underlying model has not changed for over 5 years. Mik is going to describe what needs to change to get Mylyn ready for next generation ALM.

Bring your boss to ALM Connect Executive Event

ALM Connect will provide a rich set of ideas for practitioners to take back to their teams, but without management support many of those ideas will never be implemented.

Above: EclipseCon 2012 ALM Panel with Mik Kersten, Dave West, Melinda Ballou and James Governor

Solving this problem was the motivation of running an Executive event on Wednesday the 27th of March. This event is aimed at decision makers and is by invitation only.

Content highlights include:

• Twitter and Github kick off the event talking about the future of ALM and how the next generation of software development is being undertaken. These presentations will show how you can marry innovation, rapid delivery and complex development teams into an Agile delivery capability.
• ALM in action case studies. Still working on the fine print, but we will have two high profile companies who are going to present their experience with ALM and how they are using ALM to form a competitive advantage. These sessions are aimed at telling all the dirty secrets of ALM, the motivation for adopting ALM and the reality of ALM in their companies.
• The user is the center: Apps in the world of engagement by Lee Nackman from HP. Lee has been involved in ALM for many years and was one of the executives responsible for IBMs involvement with Eclipse. In this talk Lee will describe how systems of engagement have changed the face of ALM and how that is only to get worse. I expect some sneak previews of the HP’s future ALM strategy in this talk.
• Agile 2.0 software development in the era of the social graph by Israel Gat Fellow at the Cutter Consortium. Israel a leading management light on ALM and the economics of software delivery will describe the social side of development. Describing how social graphs and other mechanisms can be used to better manage and enable software delivery.
• Scrum – Success ends with middle management by Ken Schwaber co-creator of Scrum. Having Ken, one of the drivers of the Agile movement at the event will add a level of Agile pragmatism to the proceedings. In this talk Ken will present the audience with framework for taking Agile to the next level, but be warned this path is not for the faint of heart.
• The future of ALM panel – This session includes Sam Guckenheimer from MS, Lee Nackman, Jeffrey Hammond and Mik Kersten on what the future of ALM looks like and how it will affect the audience. I will moderate, so expect an exciting and thought provoking session.

If you are interested in attending the executive day, or have boss who is interested then please let us know. You may also wish to view the Executive Day landing page, or the event announcement from the Eclipse Foundation for a detailed agenda. I can promise the event will be exciting, thought provoking and inspiring. It is rare to get so many leaders on ALM in one room and at one event.

A room full of developers and students, cans of beer loosely scattered around the room, along with bags of chips, pop and pizza. It’s a setup that would make more sense for a party, were it not for everyone clustering around power cords and loud finger tapping of engineers ripping up their laptops.

Sticking to traditional hackathon culture, there was a whole lot of coding, lots of beer, and happy chit chat mixed with serious faces betraying some heavy problem solving, in a word: hackathon-fun!

In attendence: several experienced Eclipse submitters, students from SFU and UBC, and other Eclipse enthusiasts. Projects hacked on: JDT UI, Scripted, Orion and Mylyn (for project details see the wiki page).

It was great to have Ian Skerrett from the Eclipse Foundataion attend and hack away with all the others. Great many ideas were thrown around and many a bug got fixed. Newbies got to learn a lot about Gerrit & Mylyn and how to contribute to open source. Thanx to all who attended and made the night that much better!

See more photos from the event.

At this year’s Agile ALM Connect sub-conference of EclipseCon 2012 Mik Kersten moderated a very lively analyst panel with a few of the leading analysts in ALM: Melinda Ballou of IDC, James Governor of RedMonk and Dave West (then at Forrester). Enjoy.

We are happy to announce the availability of Mylyn 3.8 which was released as part of Eclipse Juno! This release includes new components for creating EPUB documents, Subclipse integration, user interface improvements and support for the latest Bugzilla, Hudson and Gerrit versions.

The Gerrit and Hudson connectors have come along way since we created the Mylyn Builds, Mylyn Reviews and Mylyn Versions projects two years ago, when Mylyn became a top-level project. I want to thank everyone who provided feedback in hundreds of Bugzilla tasks and helped evolve the connectors over three versions. With this release the APIs are reaching 1.0 and the projects have successfully graduated from Incubation status to mature projects.

To make these integrations that connect Eclipse to popular application lifecycle tools, available to a wider audience we added the Gerrit and Hudson connectors to several EPP packages. For instance, the RCP/RAP package now contains all components to complete a full Mylyn contribution cycle:

1. Querying the Eclipse.org Bugzilla using Mylyn Tasks
2. Activating a Bugzilla task using Mylyn Context
3. Committing and pushing a change to Git using EGit and Mylyn Versions
4. Monitoring the triggered build on the Eclipse.org Hudson using Mylyn Builds
5. Completing the resulting Gerrit code review using Mylyn Reviews

I showed this at the Eclipse DemoCamp in Zurich and Java Forum Stuttgart and never left the IDE throughout the demo. It’s great to see the tooling coming together with the Eclipse 4.2 based Juno packages covering all key aspect of the development workflow out of the box.

The connector reference implementations that are part of Mylyn enable us to validate our own frameworks but feedback that we get from users and adopters building on the APIs is also incredibly valuable to drive development. I want to thank Kiu Leung for implementing an initial prototype of a Koji connector based on Mylyn Builds and Robert Munteanu for creating an integration for ReviewBoard based on Mylyn Reviews fueling the growth of the connector ecosystem.

Likewise, the number of Mylyn projects under the application lifecycle banner has been growing. The Model Focusing Tools project provides focus when working with models reducing information overload. The initial 0.9 release is part of Juno and contains connectors for EMF, GMF, Ecore Tools and UML. Manuel Bork and Carsten Reckord from Yatta Solutions recently published an article that provides a great overview of the underlying technology.

A lot has already been said about Mylyn Intent which brings documentation, code, models and other programming artifacts closer together: “Based on Donald Knuth’s Literate Programmingconcepts and allowing collaborative work around design tasks, Intent will try to reconcile developers with Documentation.” Through tight IDE and workflow integration, Intent ensures that code and documentation stay synchronized. Intent contributes its 0.7 release to Juno.

Vex, a visual editor for XML, is another quickly evolving project under Mylyn Docs which has been releasing milestones regularly. The tooling now supports XML schema and XHTML as well as basic rendering of images. Vex is not yet part of Juno but available from a separate update site and hopefully will join the coordinated release Kepler next year.

Last, but not least, there is Reviews for Eclipse (R4E) which is a project under Mylyn Reviews. R4E supports any artifact, including models, as a review source and has built-in capabilities to conduct formalized reviews that follow IEEE standards. The first 0.10 release was published shortly before Juno and R4E is now on its way towards graduation and joining Kepler.

It’s amazing to be part of the continuously growing Mylyn community and I would to thank all the individuals and companies that have contributed to this release.

• Mylyn 3.8 New and Noteworthy

Tasktop Technologies, VMware and the Eclipse Foundation co-hosted the Eclipse Juno DemoCamp in Vancouver. The event was a blast with great presentations, cool prizes and lots of opportunities for networking and making new friends. The event started off with an introduction from VMware’s Kris De Volder (seen on the right) and David Green, Tasktop’s VP of Engineering (seen on the left), who recapped Tasktop’s development over the last year since the last Eclipse DemoCamp.

The intro was followed by the main presentations:

		Reverb: Dynamic Bookmarks for Developers (Nick Sawadsky, UBC, Software Practices Lab)
		Collage Framework and Code Markup Tools (Alex Bradley)
		AVA: Assembly Visualization and Analysis. (Jennifer Baldwin & Yvonne Coady, University of Victoria)
		Rigel: A new approach to JavaScript editing based on the Eclipse Orion Editor. (Andrew Eiseneberg, SpringSource a Division of VMWare)
		jimu: Build sophisticated Android apps in minutes, using building blocks. (Linton Ye, jimu Labs)

Seen below is a presentation by Yvonne Coady and Jennifer Baldwin (speaking over Skype) that proved that remote interaction can be engaging and fun:

In this photo, Andrew Eiseneberg of WMware, is introducing Rigel, a new way to do JavaScript editing:

The presentations closed with Linton Ye describing how jimu made Android app development easy:

After the event, the socializing continued at the nearby Library Square Pub. See More Photos from the event…

Every now and then you have a conversation that changes your view of the world. I’ve now had a dozen of those with one person, Dave West, in his role as Forrester analyst, VP, and Research Director. The common thread in our dialogue has been the need for application lifecycle glue that connects the software lifecycle stakeholders within the organization, as well as across the multi-company and increasingly open source based software supply ecosystem. Both of us realized that it would be more effective to make this vision a reality than to discuss it endlessly. So today, I’m thrilled to announce that we’ll be doing just that, with Dave West joining Tasktop as Chief Product Officer.

The very rapid growth that our products have seen lately is indicative of the need to look beyond any single tool in the evolving ALM stack, and consider the flow of information between the people that define the disciplines of the software lifecycle. Tasktop got to where we are today by placing a manic focus on the needs of the individual software developer, who was getting completely overloaded with the disconnected morass of ALM tools that failed to connect to the source code that defines delivery. That forced us to create a new model of social tasks that emphasized autonomy, transparency and integration across the increasingly diverse tool chain. With Tasktop Sync, our Task Federation has migrated from supporting Agile delivery on the developer’s desktop to connecting the rest of the software lifecycle in order to bring about a “Lean ALM”.

Driving a change in the way that software is built takes like-minded people filled with passion and purpose. Dave’s mission is to help people build software just a little bit better, and with our shared values, we expect that goal to materialize very quickly. In his role as an analyst, Dave has heard the software delivery needs and gaps of countless software organizations that build the products and services that we all rely on day-to-day. In his role as Chief Product Officer, responsible for transforming that need into our product vision and roadmap, you can expect Dave to accelerate our pace of customer-centric innovation even further as we work with our partners and open source community to connect the software lifecycle.

Read more in Dave’s post and Neelan’s post.

EclipseCon 2012 was my favorite to date, and I’ve been attending since the prototype—beers and demos at Thirsty Bear during JavaOne 2002. What made it so interesting was finally getting all the Eclipse devs in the same space as key folks from Agile and ALM. Developers are the engine of the software economy. But that engine is becoming part of such a complex ecosystem of vendors and open source that to scale software delivery, we need to break down organizational and departmental silos. We need to move towards what Forrester analyst Dave West has coined a Lean ALM. And that’s what my keynote was all about. Connecting devs to project managers, to testers, and eventually to @DEVOPS_BORAT.

Some have objected to my statement that Linus Torvalds’ bigger contribution to our planet is going to be Git, not Linux. Yes, Linux is everywhere. But Linux was a creative imitation, whereas I was focusing on the true innovations that are moving us towards the social code graph, and that’s precisely where Git fits in. Also, early in the talk I mention that Eclipse has gone from 1.5M to 2.5M downloads between January 2011 and January 2012. That’s monthly downloads, and with Vietnam surpassing Germany, a clear sign of the times.

Watch the keynote here, and I look forward to hearing your feedback and ideas.

As organizations increasingly become software driven, the role of the application lifecycle is taking a new meaning. Connecting stakeholders in the software lifecycle ceases being a nice to have and any gap in connectivity quickly becomes the bottleneck of software delivery. The organizations are now noticing the friction of having developers do duplicate data entry between their issue tracker and Agile tool, or testers and business analysts queuing up weeks of defects and requirements before handing them off to developers. The application lifecycle is only as efficient as its weakest link, and if that link is manual and based on large batches and handoffs, frustration for the individuals and large-scale inefficiencies result.

With Tasktop Sync we created the first general way to connect software delivery stakeholders working in best-of-breed tools across the application lifecycle management (ALM) stack. As we’ve been rolling out Tasktop Sync over the past year to IT organizations around the world, we’ve noticed a number of things. Organizations, especially those who have been around for a while, are trying desperately to apply ALM Automation across the enterprise. To do this, these organizations are having to inventory their tool sets and identify the information flows and the workflows between stakeholders and between their tools, usually for the very first time.

Often acting as a cross between marriage counselor and coach, the Tasktop expert’s first activity in a deployment is to gather they key stakeholders from management, quality assurance, development, and business analysis in a room with as big a white board as possible. In this meeting, the organization will identify the important tools used by each stakeholder, how information needs to flow between these tools, what are the key workflows within each stakeholder silo, and what activities kick off workflows in other silos. The edges connecting the ALM repositories turn out to be various kinds of tasks that represent the lines of collaboration between the stakeholders, and that are then mapped between the various vendors’ tools with Tasktop Sync’s real-time ALM artifact synchronization solution.

In the forthcoming release of Tasktop Sync, we have formalized the lessons learned of the past year with a new authoring tool called Sync Studio. Our expertise is now captured in visual tools for cross-ALM system task and workflow mapping, ALM architecture design, monitoring tools to ease integration maintenance and alert notifications for project and system administrators. To help IT organizations scale Tasktop Sync deployments and better manage the growing number of ALM systems in a typical tool stack, Sync Studio provides a whole new set of ALM infrastructure management tools. Capabilities include:

		A Unified View across the ALM Stack: Sync Studio presents ALM architects and administrators with a comprehensive and “live” architectural view of current tools and processes, and the associated interdependencies and roadblocks that need to be addressed.
		Visual Mapping for ALM Administrators: Sync Studio provides automated mapping capabilities for ALM administrators to author and configure task, data and workflow connectivity and integration between ALM servers.
		Cross-repository Monitoring and Administration: Sync Studio helps maintain the health and performance of enterprise-wide ALM architectures through the regular monitoring of inter-tool functionality and centralized administration of changes, maintenance, trouble-shooting and alert notifications.
		End-to-end Traceability for the Lifecycle: through its Task Federation platform, Sync Studio provides complete ALM traceability that is available through the visual mapping and visibility capabilities now available in the tool.

Tasktop Sync is being announced today as part of our coordinated Tasktop 2.3 release. A notable feature from Sync the instantaneous task querying needed for Sync’s conflict resolution, is getting pushed down into Eclipse Mylyn for the benefit of our developer users as we continue to build out both the Tasktop commercial tools and the underlying Mylyn frameworks needed to support Task Federation, both on the server side with Tasktop Sync and on the developer’s desktop with Tasktop Dev and Mylyn.

Tasktop Sync 2.3

		Sync Studio: Visual Mapping, Monitoring, Validation and Notifications
		Sync Server: Scalability & failover support
		New connectors: Accept 360, ThoughtWorks Mingle, full RTC Schema support

Tasktop Dev 2.3

		New OEM Edition for HP Quality Center
		Mylyn 3.7, including instant Task List search
		New connectors: Gerrit code reivew
		See New & Noteworthy for more

Contact us for a demo of Sync Studio.

When Mylyn was still known as Mylar, our Bugzilla listed a few hundred tasks for the project. Six years later, my Task List has accumulated over 10K tasks for Mylyn alone and contains twice as many tasks in total. To help manage large task lists like that, the just released Mylyn 3.7 enhances the filtering in the Task List with an incredibly useful feature for quickly finding tasks: a Lucene based index. With the latest version, it is now possible to scope searches and instantly query over the full comment streams of all tasks.

The powerful new search options are easily discoverable through content assist (Ctrl+space). I particularly like the ability to filter based on date range. If I know for instance that the task I am looking for was commented on recently I can now find the relevant tasks with a few key strokes:

There has been a lot of talk about Gerrit, and Eclipse projects are increasingly adopting code reviews as part of their workflow. The Gerrit connector, which has been evolving in the Mylyn Reviews project, is now included in Mylyn releases, the Juno repository, and available from connector discovery bringing the first-class IDE integration that Mylyn provides for tasks to code reviews.

We have incorporated a lot of feedback from early adopters and continuously improved the connector. It now supports Gerrit 2.2 (earlier versions have been reported to work as well) and ships with support for a variety of authentication mechanisms including Open ID. The connector is tightly integrated with the latest version of EGit and extends the import wizard for simple importing of projects. When a Gerrit server has been configured, the wizard provides a listing of all Git repositories from that server. This enables simple cloning and importing of available projects.

Another noteworthy enhancement is the improved compare editor integration. The editor now shows the structure of the review. You can navigate through patch sets and add comments inline.

A key feature of Gerrit is the ability to stage multiple iterations of a change. In practice each iteration, referred to as patch sets in Gerrit, equals a commit that is tracked in a separate branch. As in the Gerrit web interface, the review editor can now show differences between patch sets to review incremental changes.

Sometimes it’s seemingly simple enhancements that add a suprisingly big benefit. We have long recognized hyperlinks as a universal way to link artifacts and made them clickable in the Task Editor. This linking now works across builds, tasks and reviews.

For example, Loading the JUnit results or reviewing the console output of a failed build that is linked from a review comment is now literally two clicks away from the review editor.

If you are at EclipseCon in Reston don’t miss the opportunity to see Mylyn 3.7 in action. Visit Tasktop at booth #25 and also attend one of the great talks that will highlight new features. A full list of new features are described in the New & Noteworthy section.

		Read the Mylyn 3.7 New & Noteworthy
		Meet Tasktop at EclipseCon

(See also this article’s translation to Serbo-Croatian language by Vera Djuraskovic from Webhostinggeeks.com)

I joined Tasktop in part because I share the goal of increasing programmer productivity, especially by filtering out unimportant information.  I also liked how Tasktop is committed to being involved with and connected to the broader Eclipse community.

In this spirit, I suggested a feature to the EclEmma project: letting developers create and view incremental code coverage results.  This would let developers see a much smaller, but more relevant, set of classes and methods which they could then investigate.

It had bothered me how difficult it was to find where things happen in code, especially in large, unfamiliar code bases.  Yes, you can step through the code, but sometimes you have to step for a long time.  This is boring and tedious, and frequently you step one step too far and overshoot the place you wanted to see — losing the information about the values of the variables at the point you cared about.  If an asynchronous process gets spawned as the result of a Listener attached to a GUI element, it is nearly impossible to step through.

You can search for text on or around the GUI element to help you find where the actions related to that element are processed, but sometimes this isn’t practical.  Sometimes the GUI element has text that is so common that it is impractical to search for it, like “Finish” or “Next”.  Sometimes the GUI element doesn’t have text associated with it, like a button with a picture on it (and no tooltip).

In practice, I have observed that people usually guess at what words might be included in the class or method names, and then when they think they are close, simulate stepping through code by reading through it and making informed guesses about where the execution flow will go.  Unfortunately, frequently they guess wrong, usually in the choice of a starting point or the value of an if-condition.

Another particularly pernicious mistake is not realizing that you are tracing through a superclass of the class that is actually executed.  If you Command/control-click on a method name, Eclipse will preferentially take you to the implementation of that method in the same class; this means that if you ever trace into a superclass, Eclipse will tend to keep you in the superclass; realizing that you need to go back to the subclass is not always obvious.

I thus suggested to the EclEmma code coverage team that they add a feature to the code coverage tool to let users start and stop coverage so that they can see which code was executed for specific short periods of the execution.

With the 2.1 release of EclEmma, the EclEmma team has implemented incremental code coverage — a very useful feature!

How to use it

First, install EclEmma.

Next, go to Preferences > Java-Code Coverage, and check the “Reset execution data on dump”.

Open a Coverage view.  In the Coverage toolbar, there is a “Dump Execution Data” button.

Pressing the “Dump” button will now display coverage and reset the code coverage results.  Thus, if you press the dump button right *before* you do the action you are interested in (e.g. pressing a certain button), and then again right after you do the action, then the code coverage results will show only the exact classes and methods that were executed in response to that action.  Among other things, this means you won’t get misled to look in the superclass instead of the appropriate class.

If you need finer-grained information about which code executed, EclEmma colours your lines of code based on whether they were executed in that interval: green if they were fully executed, red if they were not executed at all, and yellow if they were partially executed (if, for example, the line uses a “?” ternary operator, as in the “drightSide” assignment below).

By default, EclEmma only shows coverage information for your code, not for all the libraries you bring in.  To change this, uncheck “Source folders only” in Preferences > Java > Code Coverage.

Note: while for code coverage, you probably want to see the coverage results for all classes and methods, when using incremental code coverage to locate places in code, you should select “Hide unused elements” from the Coverage toolbar’s drop-down menu.

Incremental code coverage is a very powerful technique: instead of wandering through thousands of classes and methods to find the handful of classes and methods that are interesting, you can spend a few minutes to get their names directly.

I do need to give a slight caveat: exceptions interfere with the code coverage instrumentation, interrupting the marking of that branch of code as executed.  That is a known limitation with the way that code coverage is done.  Thus if your code uses exceptions a lot, EclEmma might incorrectly say that a branch of code was not executed when it was in fact executed.   However, if EclEmma tells you that code was executed, it really was executed.

Using EclEmma in conjunction with Tasktop Dev or Mylyn is an exciting prospect.  Mylyn and Tasktop Dev tell you what you (or someone else, if you are looking at their context) had looked at; EclEmma gives you hints on what you should look at. We have only just started thinking about how those two could be combined, but are excited by the possibilities.

For further information, see EclEmma, Tasktop Dev, or Mylyn.

Note: the screenshots used from the open-source Java GIS tool GpsPrune, isolating the action of showing the scale legend.