Tasktop

When Mylyn was still known as Mylar, our Bugzilla listed a few hundred tasks for the project. Six years later, my Task List has accumulated over 10K tasks for Mylyn alone and contains twice as many tasks in total. To help manage large task lists like that, the just released Mylyn 3.7 enhances the filtering in the Task List with an incredibly useful feature for quickly finding tasks: a Lucene based index. With the latest version, it is now possible to scope searches and instantly query over the full comment streams of all tasks.

The powerful new search options are easily discoverable through content assist (Ctrl+space). I particularly like the ability to filter based on date range. If I know for instance that the task I am looking for was commented on recently I can now find the relevant tasks with a few key strokes:

There has been a lot of talk about Gerrit, and Eclipse projects are increasingly adopting code reviews as part of their workflow. The Gerrit connector, which has been evolving in the Mylyn Reviews project, is now included in Mylyn releases, the Juno repository, and available from connector discovery bringing the first-class IDE integration that Mylyn provides for tasks to code reviews.

We have incorporated a lot of feedback from early adopters and continuously improved the connector. It now supports Gerrit 2.2 (earlier versions have been reported to work as well) and ships with support for a variety of authentication mechanisms including Open ID. The connector is tightly integrated with the latest version of EGit and extends the import wizard for simple importing of projects. When a Gerrit server has been configured, the wizard provides a listing of all Git repositories from that server. This enables simple cloning and importing of available projects.

Another noteworthy enhancement is the improved compare editor integration. The editor now shows the structure of the review. You can navigate through patch sets and add comments inline.

A key feature of Gerrit is the ability to stage multiple iterations of a change. In practice each iteration, referred to as patch sets in Gerrit, equals a commit that is tracked in a separate branch. As in the Gerrit web interface, the review editor can now show differences between patch sets to review incremental changes.

Sometimes it’s seemingly simple enhancements that add a suprisingly big benefit. We have long recognized hyperlinks as a universal way to link artifacts and made them clickable in the Task Editor. This linking now works across builds, tasks and reviews.

For example, Loading the JUnit results or reviewing the console output of a failed build that is linked from a review comment is now literally two clicks away from the review editor.

If you are at EclipseCon in Reston don’t miss the opportunity to see Mylyn 3.7 in action. Visit Tasktop at booth #25 and also attend one of the great talks that will highlight new features. A full list of new features are described in the New & Noteworthy section.

		Read the Mylyn 3.7 New & Noteworthy
		Meet Tasktop at EclipseCon

Tasktop attended the Eclipse Foundation Island at CeBIT 2012, an event with massive attendance. Eclipse Island is part of the Open-Source forum which is one of a many booths in the hall which is one of out of two dozen halls, located a mere twenty minute walk from the main entrance. Still, a lot of folks were finding us to discuss Mylyn, Tasktop and Sync or ask where they can get the free Eclipse CDs.

When we weren’t busy demoing the latest Tasktop features we had a fun time explaining to 12 year olds that they need Git, Gerrit and Hudson if they want to be successful programmers. We’re watching for a flood of resumes around 2026. If you want to work with these tools now, check out our careers page for openings.

We had an awesome EclipseCon Europe with great talks and discussions that covered the latest developments in the Mylyn community. Fret not if you missed the conference. There will be more opportunities in November to see Mylyn in action.

I’ll be speaking at Devoxx in Antwerp, Belgium next week about connecting Agile, ALM and the IDE. The week after I’ll be heading to democamps in Germany to present the Git, Gerrit and Hudson integrations. The first demo is in Hamburg on November 23, followed by Kaiserlautern on November 24 and Kassel on November 29. Hope to see you there!

When Mylyn became a top-level project, the monolithic command line driven PDE build that was understood by one committer, executable on a single machine was no longer appropriate to support the modularization needs and distributed development of the project. Since we like to play with the cool kids we jumped onto the Maven/Tycho and Git bandwagon for a ride to the land where contributions merge without conflicts and sources build themselves. We are still on the road learning something new each time we hit the occasional bump. We already found out…

• …how to convert our reasonably linear CVS history into a Git repository that does not look like a blooming tree,
• …how to setup CI builds that finish within a couple of minutes,
• …how to aggregate tests results across build jobs,
• …how to create reproducible builds with strict target defintions,
• …how to manage releases across several Git repositories,

…and much more. While the new build system is not a silver bullet that made release engineering obsolete it’s a huge improvement over the previous configuration and enabled us to take advantage of tools like FindBugs or Gerrit that we could not easily integrate before.

If you are interested in learning from our experiences take a look at the video of the EclipseCon session.

If the length of the CC list is any indication of interest in a bug, a lot of committers are looking forward to Gerrit support at Eclipse.org. The webmasters are working hard on making that happen but for Mylyn we were so eager to integrate code reviews in our workflow that we went ahead and deployed a Gerrit instance outside of Eclipse.org. Although the server has only been running for a few weeks, over 50 reviews have already been opened by a dozen people. Read on for a description how to setup your workspace to start contributing to Mylyn.

The server setup with Gerrit 2.2.1 and Hudson 2.1.2 follows the configuration described in Alex Blewitt’s tutorial. Every time an update to a Gerrit change is pushed, a Hudson build is triggered, and the result is reported back on the code review. After having worked that way for a while I would not want to miss the benefits of quick validation of staged commits and the simplified feedback process for contributions and changes proposed by committers.

Registering a Gerrit Account

To use the review server a few steps are required:

1. Register an account at review.mylyn.org using an Open ID provider such as Google
2. Define a username
3. Upload your public ssh-key for authentication

Once the account has been created you need to contact me to get the account added to the trusted users group which has the required permissions to create code reviews.

Adding Gerrit as a Git Remote

On the client end, Gerrit is simply added as another remote to the local clone of a Mylyn Git repository. For example, the .git/config file for the Mylyn Tasks git repository would specify these two remotes:

[remote "origin"]
  fetch = +refs/heads/*:refs/remotes/origin/*
  url = git://git.eclipse.org/gitroot/mylyn/org.eclipse.mylyn.tasks.git
  push = HEAD:refs/heads/master

[remote "gerrit"]
  url = ssh://spingel@review.mylyn.org:29418/org.eclipse.mylyn.tasks.git
  fetch = +refs/heads/master:refs/remotes/gerrit/master
  push = HEAD:refs/for/master


If you have cloned any of the Mylyn repositories already, you can simply add the gerrit remote section. No need to clone again. Note that Gerrit uses a special ref for the push configuration called refs/for/master. This ensures that changes are not directly pushed into the master but staged in a branch first.

Additionally, I recommend adding these settings to the .git/config file:

[gerrit]
  createchangeid = true

[branch]
  autosetuprebase = always


The createchangeid flag causes EGit to automatically generate a Change-Id header for each commit. This id is used by Gerrit to track updates to existing code reviews. The autosetuprebase flag will cause all new remote tracking branches to be configured for rebase automatically which is useful when updating task branches.

Creating a Code Review

To push a new code review to review.mylyn.org, first create a local branch to track your change. This is good practice and makes it easy to modify the change based on feedback from the code review later.

$ git checkout -t origin/master -b bug#191522-provide-full-text-search


Now go ahead and commit to the branch.

The next step is to create the code review. Simply run this command:

$ git push gerrit


Instead of pushing to git.eclipse.org (origin) this will push the change to review.mylyn.org (gerrit). Note that Gerrit creates one code review per commit. In case you committed multiple times see below how to merge commits.

Updating a Code Review

If an existing code review needs be updated with additional changes or if it needs to be rebased against the current master, checkout the branch and pull:

$ git pull


Then make the desired changes and instead of creating a new commit, amend the previous commit. This retains the change id and allows updating of an existing code review:

$ git commit -a --amend


To make the update visible on the server push again:

$ git push gerrit


This creates another branch that is automatically attached to the same code review.

Pushing Changes to Eclipse.org

Once a code review is approved in Gerrit, changes are automatically merged into the master. Here comes the quirk: In the case of review.mylyn.org this does not mean much. The Git repositories on review.mylyn.org are mirrored hourly from git.eclipse.org overriding any changes that were merged by Gerrit. For good reason, only committers are allowed to push to git.eclipse.org hence a committer needs to run this simple command eventually to make changes visible in origin/master:

$ git push origin


Contributors need to go through the standard Eclipse contribution process. Once a code a review is approved a patch needs to be created and attached to a bug report to track the IP in accordance with the Eclipse process. This is very easy using EGit’s drag’n'drop support for commits.

Squashing Commits

If you committed multiple times you can always squash commits later using interactive rebase, e.g. to merge the last two commits into one commit use this command:

$ git rebase -i HEAD~2


This opens an editor to modify the commit history. If you change “pick” in the second line of the editor to “fixup”, that commit will be merged with the previous commit when you exit the editor.

Accessing Gerrit through HTTP

If you are not able to connect to review.mylyn.org through SSH on port 29418 you can instead setup a password for HTTP access. In this case the url in the remote configuration needs to be modified slightly:

[remote "gerrit"]
  url = http://spingel@review.mylyn.org/p/org.eclipse.mylyn.tasks
  fetch = +refs/heads/master:refs/remotes/gerrit/master
  push = HEAD:refs/for/master


More Information

The Mylyn contributor reference has a lot more information on how to get started. I am looking forward to your code reviews and contributions to Mylyn!

• Bugs marked as bugday
• Bugs marked as helpwanted

November will kick off with EclipseCon Europe in Ludwigsburg, Germany. It’s an excellent conference and a great opportunity to connect with the community. But, it gets even better: This year’s program is loaded with Mylyn related talks that cover a range of topics about exciting developments in Mylyn!

Wednesday

• Eclipse Hacker’s Guide to the Git Universe by Matthias Sohn, Benjamin Muskalla and others
• Experiences from migrating Mylyn to Git and Tycho by Steffen Pingel

Thursday

• What the heck are logical models? by Benjamin Muskalla and Mikael Barbero
• Collaborative Modeling: the New Deal by Mariot Chauvin, Benjamin Muskalla and Martin Fluegge
• Build your EPUBs with Eclipse, and read them too by Torkild U. Resheim
• Introducing the Visual Editor for XML by Florian Thienel

Friday

• Task-focused modeling with Mylyn, EMF and Papyrus by Benjamin Muskalla
• Pimp your productivity with Git, Gerrit, Hudson and Mylyn by Benjamin Muskalla and Steffen Pingel
• Mylyn Reviews – Introducing Review for Eclipse (R4E) by Alvaro Sanchez
• Create useful documentation with Mylyn Intent: a step further in Application Life-cycle Management by Alex Lagarde, William Piers and Cedric Brun

If you would like to meet us for a chat or grab a cold beverage, please connect to @bmuskalla or @steffen_pingel or catch us at one of the talks!

It’s always a good time when you meet people in person that you otherwise only interact with through the Eclipse.org Bugzilla. Recently, Matthias Sohn organized a week long hackathon at the SAP offices in Walldorf bringing together committers from EGit, Mylyn, Gerrit and the community. From Tasktop, Benjamin Muskalla and I joined the hackathon. We enjoyed lots of free SAP coffee and had a great time working with the rest of the team. We started the first day with a brainstorm session which resulted in a task board full of story cards.

My first goal was to get the Mylyn Gerrit connector working with the latest version of Gerrit. Since the early days of Mylyn we have worked bootstrapped feeling the same pains as everyone else using our tools. We know how important it is to eat our own dogfood and now that code reviews in Gerrit have (almost) become routine in our everyday workflows. Updating the tooling to work with the server used by the Mylyn project was an obvious step.

I’m happy that we made good progress during the week in Walldorf and the connector now works with Gerrit 2.2 while maintaining backwards compatibility with Gerrit 2.1.

Since the Mylyn instance is configured for OpenID authentication I also added support for that. It was an interesting exercise to extend the tasks framework to allow authentication through a browser window in a way that works across Windows, Linux and Mac.

Once the connector was working with the latest Gerrit, I took the opportunity to work with Dariusz from EGit Sychronize view fame. Dariusz knew instantly what classes to look at to enable navigating through patch sets from the Synchronize view.

The feature is still a work in progress but basic functionality is already available in the latest weekly build: The compare action in the review editor fetches the patch set and opens the corresponding changes in the Sychronize view.

Another useful enhancement to the Gerrit Connector was committed by Sascha Scholz who added a field for specifying free form queries.

We had lively discussions with Stefan Lay, Matthias Sohn, Benjamin Muskalla, Manuel Donninger and others around automating task-based branching. Not surprisingly everyone in the room had slightly different ways of working when it came to Git and there was no straight forward answer how to best automate this. I believe we settled on a nice workflow that is not intrusive:

1. On task deactivation, the checked out branches for all repositories connected to projects referenced in the task context are remembered
2. On task activation, branches that were checked out on last deactivation (if any) are automatically checked out again restoring the same workbench state

To support that Matthias committed a change in EGit that accelerates creating new branches by making the “Switch To” menu top-level in the popup menu of the Git repositories view. Additionally, Manuel proposed an enhancement to pre-populate the branch name based on the summary of the active task literally reducing the click count for creating a task branch to 4 clicks.

Based on the progress that Manuel had already made, a number of additional changes to persist branch information per task for Mylyn were proposed that are currently under review. If you have any input, please comment on bug 309578.

A number of other features that are beyond the scope of this post were being worked on throughout the week including significant performance improvements for EGit. Of course we didn’t limit our activities to coding but socialising was a big part of the hackathon.

Thanks to Matthias for organizing the meeting. Hopefully we’ll get a chance to repeat this in the future!

• Mylyn Weekly New & Noteworthy
• EGit 1.1 New & Noteworthy

Getting a contribution accepted into Eclipse is not a walk in the park. In the Mylyn project, each patch that comes in through Bugzilla is reviewed by a committer usually going through multiple iterations until it makes its way into Git or CVS. Mylyn has great support for handling patches, but this can still be a tedious process for both the contributor and committer when it comes to larger changes, due to a lack of automation of the contribution workflow. Contributors need to keep patches up to date, possibly juggling several outgoing change sets which can easily lead to incomplete or over inclusive patches. On the receiving end committers are burdened with validating patches and running tests which requires a number of manual steps. Conversations around patches happen through Bugzilla comments making it difficult to refer to particular lines.

These problems are not limited open source contributions. In many organizations, development follows similar work-flows often requiring use of several applications that are not integrated. This article introduces the new Mylyn connector for the open-source Gerrit code review system that helps automate onerous parts of the code review process enabling developers to focus on collaborative aspects and code rather than nit picking broken patches.

Gerrit is a web-based code review server for the distributed version control Git. When combined with Hudson/Jenkins it forms a powerful system that stages and validates every change before other developers have even noticed. With the integrations available in Mylyn 3.6 and Mylyn Reviews 0.8 developers can now seamlessly integrate these tools in their day-to-day work-flows.

Alex Blewitt has written an excellent tutorial how to configure Git, Gerrit and Hudson. In this article we will assume that the following servers have been setup as described in the tutorial:

• Gerrit 2.1.8   (Gerrit 2.2 is not yet supported)
• Hudson 2.0.1

To get started we need to install the following tools from their respective p2 repositories into Eclipse:

• EGit – http://download.eclipse.org/egit/updates
• Mylyn Builds: Hudson Connector – http://download.eclipse.org/mylyn/latest
• Mylyn Reviews: Gerrit Connector – http://download.eclipse.org/reviews/updates

To illustrate the interactions in a typical code review we’ll look at how Alice and Bob work on on a code change together. We’ll start with Alice who intends to implement an enhancement for an existing project that is already setup in Gerrit and Hudson.

Alice has cloned the Git repository that is managed by the Gerrit server into her workspace. In her environment the server is called dev.

By default EGit creates a configuration that pushes the local master branch to the remote master branch. For Gerrit, Alice needs to change the configuration to push the local master to refs/for/master instead. Gerrit uses that special location to stage changes for the master branch.

Alice is now ready to commit her enhancement and push it to Gerrit. The commit dialog has an option to automatically generate a Change-Id which is used by Gerrit to track the change. If Alice wants to update her commit later Gerrit can relate the changes based on that ID.

The confirmation dialog shows that the push to refs/for/master automatically created a new branch to track the changes for the code review.

In our setup we configured a Hudson server to monitor the review server for new branches. Every time a new branch is detected a build is automatically triggered. Alice now wants to verify that all tests pass in the build that was triggered by her push before involving Bob in the code review.

After Alice has configuring the build server in the Team Repositories view and subscribed to the corresponding Hudson job she can monitor the status in the Builds view without switching to the browser. Since Alice’s commit broke one of the integration tests the build status is red. This is not problematic since the commit is staged and not yet visible in the master branch.

It’s easy to drill down into the details of a failed build. Double clicking a job opens an editor that shows the failed tests, changed files and created artifacts. With a few clicks Alice can open test results in the JUnit view and re-run the failed test in her workspace.

Once Alice has fixed the bug she commits and pushes again to replace the previous change on the code review. To update the existing code review she amends the previous commit. That way, the commit message gets automatically populated with the Change-Id from the previous commit which identifies the code review. Through the ID Gerrit knows that the new commit replaces an existing change and is not a new code review.

Again, the build server picks up the changes and this time all tests pass.

Alice is now ready to pass the change on for review. She adds a task repository for the Gerrit server and a new query in the Task List.

The query is a personalized subscription to the review server. It brings in code reviews Alice is interested in into the IDE and enables her to work with them. Like any other Mylyn task, code reviews can be scheduled and activated embedding reviews seamlessly into the task-focused work day.

Code reviews open in an editor that is the hub for the conversation around a Gerrit change. The editor shows details about the work-flow state of a review, the people involved and the comments.

The review that is shown here has received one vote already. When the Hudson server executed tests it posted the results back to the review. A +1 in the Verified category indicates that the build passed.

To pass the code review a +2 is required in the Code Review category. To notify Bob that the change is ready for review Alice adds him as a reviewer in the people section.

Bob’s workspace is setup with the same projects and queries. When Bob task list refreshes a little popup notification is displayed and the review is decorated with a blue arrow in the task list.

Bob’s first step is to open the code review. The content under review is listed in the patch-set section. Each patch-set equals one commit to the Gerrit server and is tracked in a separate Git branch. Bob can now bring a patch set into his local workspace by using the fetch button under the patch set. This is useful to verify a fix or to try a feature through running changes locally. In this case Bob decides to review changes directly in the compare editor.

Double clicking a file in the patch-set section opens an editor that shows the current master on left and the proposed changes on the right. Comments can be added by selecting any line in the file and right clicking. Initially, comments are saved as drafts and are not visible to others.

After Bob has completed the review adding several comments he publishes them. The publish button opens a dialog that allows Bob to provide a rating.

With Bob’s +2 vote all requirements are satisfied and Bob submits the review as the last step. The most recent patch on the review is automatically committed to the master branch by Gerrit and becomes visible to all developers.

With these IDE integrations, Tasktop provides the same task-focused productivity benefits that developers already enjoy for tasks and SCM systems to code reviews and builds, capturing more of the developer’s workday and providing even better integration for mixed open source and commercial ALM stacks.

To see a demo watch the video at the end of Mik’s blog post on Mylyn 3.6.

When Mylyn was released as part of Eclipse Helios in June 2010, the project had 6 committers and resided under the Eclipse Tools umbrella. Since June, Mylyn has been promoted to a top-level project, 18 committers have joined and another half dozen are on the way, 350 bugs were resolved, and today we are happy to announce the Mylyn 3.5 release!

It all started as an idea on a University of British Columbia whiteboard in 2004. A year later Mik Kersten proposed the project (then called Mylar) on Eclipse.org which was created under Technology top-level project. With its 1.0 release in 2006 the project moved to Tools and only a year later published the 2.0 release with a matured API. In 2008, Mylyn 3.0 followed which laid the basis for a tremendous growth of the Mylyn ecosystem which now encompasses 50 extensions.

The Mylyn project has expanded along with the continuous growth of its user community. The WikiText and ReviewClipse components added new tooling and it became apparent that a single project was no longer sufficient to support the extended scope and growing interest for participation. Mylyn was promoted to a top-level project and divided into several sub-projects along its API boundaries and “Application Lifecycle Tools” was added to its name to reflect its new scope. Today’s Mylyn 3.5 release is the first under the new structure with frameworks and APIs for key ALM components: Context, Docs, Builds, Tasks, Reviews and Versions.

For a long time, it has been difficult to discover Alt+click for temporarily making filtered children visible in focused views. This was addressed in the latest release which adds a neat affordance that is displayed in focused navigator views on hover. It fully replaces the quirky Alt+click (also known as Alt+Shift+Ctrl+click to Linux users) mechanism with a plus icon that shows children of a node when clicked. Additionally, nodes are no longer collapsed while unfiltering hierarchies making it much simpler to navigate.

As with every release we have further streamlined the Task List experience. For Mylyn 3.5, we’ve added a new filter that helps control the number of incoming notifications. A key feature of Mylyn’s Task List is the ability to provide offline access to repository tasks. When a task is added to the offline store all its subtasks are retrieved as well to make them available for instant access. Querying repositories that make heavy use of task hierarchies can bring in a lot of subtasks some of which may not be interesting to the user. These subtasks that do not directly match the query can now be filtered using the Advanced Filter options in the Task List view. This is tremendously helpful for everyone managing a task list with many incomings.

The builds framework, which was released for the first time in Mylyn 3.5, provides support for continuous integration systems. The first reference implementation building on the APIs is the Hudson connector that seamlessly integrates Hudson or Jenkins builds in the Eclipse IDE. The connector was started as a Google Summer of Code project by Markus Knittig and enhanced by Torkild Resheim, Eike Stepper and others. These blog posts 1, 2, 3 highlight features of the integration.

There are lots of smaller goodies in the new release as well: The active task can now be shown on a widget that can be positioned on the window trim.

Custom work flow support was added to the Bugzilla connector based on contributions from Charley Wang. Frank Becker also updated the connector to support Bugzilla 4.0 and added an enhanced search UI for complex queries using boolean charts.

Thanks to everyone who participated in the release and contributed enhancements, bug fixes and feedback!

Read the Mylyn 3.5 New & Noteworthy

Those regularly checking for updates on the Mylyn weekly site may wonder why there were no changes for several weeks and come February there has been a spur of activity. Read on for an update on what has been happening behind the scenes and some of the cool new features available in Mylyn 3.5 RC2.

Several month ago we undertook an effort to make the Mylyn top-level project a reality. We started by splitting the code base that had evolved for over five years along its API boundaries into eight sub-projects each with its own Bugzilla product, source repository and project plan. Four sub-projects, Docs, Builds, Reviews and Versions, chose to host their sources in Git while Commons, Context, Incubator and Tasks have moved to a new CVS root with a migration to Git planned for later this year.

Having Git in the mix and a need for a modular build prompted us to replace a conglomerate of shell scripts, ant files and PDE build magic with a standard Tycho build. With that in place it was straight forward to run builds on the Eclipse.org Hudson server on a regular schedule where results are now accessible and visible to anyone.

Integration tests are running against all supported platforms, Eclipse 3.5, 3.6 and 3.7, and work is underway to also fully support Eclipse 4.1. Tests are split into different suites for regular JUnit plug-in tests, SWTBot UI tests and standalone tests that run on a plain VM without requiring an OSGi container. That way, if a commit accidentally introduces an undesired UI or platform dependency or uses an API not available on older Eclipse platforms, we find out before a build is released.

I’m happy that the resolution of bug 328383 marks the completion of the restructuring. After releasing Mylyn 3.4.3 as part of Helios SR2 last Friday we are now focusing with full force on resolving the remaining bugs for the Mylyn 3.5 release in March.

A very cool new features is a hover affordance that is displayed in focused navigator views. It replaces the quirky Alt+click (also known as Alt+Shift+Ctrl+click to Linux users) mechanism to show children of a clicked node. Additionally, nodes are no longer collapsed while unfiltering hierarchies making it much simpler to navigate.

One of my favorite widgets finally escaped the Incubator and now comes with the Task List. It’s a link that shows the active task on the window trim next to a button for two click activation of tasks from the history.

The widget is hidden by default and can be enabled on the Tasks preference page.

If you don’t want to wait for the release, checkout the 3.5 RC2 build from the new weekly update site location: http://download.eclipse.org/mylyn/snapshots/weekly/.

Due to the amazing developer community that is moving the Mylyn sub-projects forward, the 3.5 RC2 build also includes the first preview version of the Gerrit code review connector! More about that in a future post.