Cross-Account Package Management for NuGet in VSTS

In the following post, we will look at the difficulties in consuming packages from a different VSTS account, as part of a build process in our own account.

Background

Visual Studio Team Services (VSTS) offers package management services. As per the documentation, this is “an extension that makes it easy to discover, install and publish packages”. Team Services enables teams to collaborate with other teams through versioned, cohesive libraries and APIs, in the same way that many 3rd party vendors offer their libraries and frameworks today. In fact, VSTS Package Management exposes the most common package management services that developers use today – NuGet for .NET developers, NPM for NodeJS developers, and now in public preview, Java packages can be served as well.

Access to Packages

Unlike the public open feeds, your teams’ package managers are designed to enable access only to those accounts who should have access to the packages. When you create a new feed, VSTS asks you to decide who has permissions to contribute (i.e. publish to the feed), and who has access to read (i.e. download and consume packages from the feed):

image

You have two choices regarding the contribution of packages to the feed. You can specify that any of the team members (i.e. those who are members of the hosting team project) can add packages, and/or the build service account (i.e. the service account used to run builds in VSTS). Note that you will want to specify the latter if you use the build service to publish the packages (you should). You normally wouldn’t need to add permissions for the team members as well.

You also have two choices for controlling access to consumption of the feed as well. You can either limit access so that only members of the project can consume the feed’s packages, or you can allow everyone in the account.

This works fine for most organizations. Not for all, however.

Multi-Account Organizations

Some organizations own multiple VSTS accounts. For some the decision simply grew organically, with various teams trying out the services, and chose to stay this way because there is no easy migration path for a project from one account to another. Others intentionally decided to keep multiple accounts, one for each internal organization (different divisions, for example). Regardless of the reasons, the situation may arise, where components developed in and published by one account need to be consumed by components that are being developed in another account.

Unfortunately, this option does not exist. VSTS does not allow you to specify that you want to give access to members of another account.

How Can We Consume Packages from Another Account’s Feed?

The problem that we are trying to solve is the consumption of packages from Account A, in a build definition that builds an application under Account B. What we will need to do is to overcome the limitations that are set by VSTS’s package management services, and somehow access Account A, with a set of credentials that it will accept.

Solution

Step 1 – Access to the Publishing Account

The first thing you need to do, is to gain access to the account. You need credentials that allow you to view packages. This will be given to you by administrators of that account. You will be given a username, and a Personal Access Token, or PAT (see documentation about creating and using PATs). Your PAT will (ideally) be configured to grant you read access to Packaging for the required account.

Step 2 – Add a Custom NuGet Configuration File

Next, in order to customize how the build system restores NuGet packages, you are going to need to add a custom NuGet configuration file. You can set it up with the defaults, configuring whatever you might need for your own circumstances, like this:

image

Step 3 – Add Your Credentials to the Build Definition

Next, you will need to add the credentials for the package manager’s account to the build definition. You will store these as variables. Be sure to encrypt the PAT – you do not want any passwords to be saved in plaintext!

image

Step 4 – Add a Task to the Build Definition to Access the Package Feed

In the next step, you will need to create a custom source for accessing the other account’s feed. You will need a Command Line build task, and you will be running against the nuget.exe CLI tool. The simplest way to do this is to add nuget.exe to your source control. I prefer to put it under the solution, in a subfolder named Tools.

You will need to call the sources Add command, and specify the name of the feed, its URL, and the username and password/PAT. You will also specify which NuGet configuration file you are going to modify with this setting.

Note that you will want to set the name of the feed to be the same one that you specified in Visual Studio, when building your application.

Here is an example for the Arguments that you must set for the build service to recognize the other account’s feed:

sources Add -name OtherDivision -source https://assaf-account-b.pkgs.visualstudio.com/_packaging/PrimaryFeed/nuget/v3/index.json -user $(NugetUsername) -pass $(NugetPassword) -ConfigFile $(Build.SourcesDirectory)\Custom.nuget.config

Step 5 – Configure NuGet to Restore with Custom Configuration

Finally, you need to modify the NuGet restore command, so that it uses your custom configuration file:

image

Your build definition is now able to restore NuGet packages that are published by another Account!

Summary

In this blog post, you have learned how to configure NuGet in a VSTS build definition so that it can consume packages that are published by a different VSTS account.

I hope you find this useful.

Happy coding,
Assaf.

Posted in Package Management, VSTS | Tagged , , | Leave a comment

Scaling Scrum with Nexus in VSTS

In this post, I will cover what Scrum Nexus is, where, when, and why you would want to use it, and how to set up VSTS best to accommodate your Scrum practices. As VSTS’s tooling is not yet perfect for some of Nexus’s practices, I will discuss some viable fallbacks and workarounds.

Background – Why Scale?

Scrum.org’s official Scrum guide defines Scrum as “A process framework used to manage complex product development”. Scrum’s events, artifacts and rules revolve around a Scrum team, which consists of a Scrum Master, Product owner, and 3-9 development team members (a.k.a. developers).

This limit on team size is important. For Scrum to succeed, the team must consist of developers who can cover all the work required for the product to be delivered at the requisite quality level. If, however, there are fewer than 3 developers, the team is likely to miss some skills required to deliver the product. More than 9 team members, will require too great an effort to coordinate, and result in an unmanageable process.

Therefore, when delivering a large product, as is often the case in enterprise-level projects, the organization must scale beyond the single Scrum team. A framework is required to manage and coordinate the work of multiple Scrum teams.

Enter Nexus.

What is Nexus?

Scrum.org defines Nexus as “an exoskeleton that extends Scrum to guide multiple Scrum team on how they need to work together to deliver working software every Sprint”. While there are other systems out there for scaling Scrum, from a simple (and somewhat naïve) “Scrum of Scrums”, where Scrum Masters get together to coordinate the teams’ interdependencies, to full blown complex frameworks such as SAFe, I tend to prefer working with Nexus, as it is a simple extension of Scrum. It builds upon the knowledge that teams have working with Scrum, and applies the same processes, artifacts, and roles, to a larger scale, introducing minor tweaks, rather than new complex mechanics.

Nexus revolves around the notions that teams minimize risk by minimizing interdependencies. A special integration team (called the Nexus Integration Team, or NIT) is formed. The NIT is responsible to uncover and manage whatever dependencies exist between teams’ work items, by eliminating or carefully controlling their impact. The NIT is also responsible to guide the Scrum teams towards continuously integrating their work, to reduce the risk that comes from large integrations at the end of a sprint or a release.

The Nexus Integration Team members include cross-functional developers, such as the product’s DBA, build-master, architects, technical writer, and anyone who may be of greater use the organization as a coordinator of the integrated product, than a member of one Scrum team. The NIT has one Scrum Master, who depending on the teams, may be the one Scrum Master for the entire product, or just for the NIT. In any case the product must have one, and only one, product owner.

Setting up VSTS for Nexus

The Team Project

First things first – There should be one and only one team project for the entire product. You want to be able to view the entire backlog, measure, query, and track progress for the entire product, and you want to be able to view charts and reports that aggregate data for the entire product. If you separate the product so that each team has its own “Team Project”, you will not be able to do so. The Work tracking and dashboard capabilities of VSTS are limited by scope to a single Team Project. You cannot split a backlog, or visualize a Kanban board across multiple team projects.

This choice does have some limitations – the entire product, including all of its teams will have to follow the same process template, though if you have already decided to follow Scrum and Nexus, this should not be an issue.

Areas and Iterations

One Set of Iterations for All Teams

According to the Scrum guide, if you have multiple teams working on the same product, they should be delivering together on the same cadence. In VSTS, this means that there should be a single iteration cadence that all teams follow. All teams start together and end together because the integrated product increment is delivered at the end of every sprint.

The Nexus Integration Team is the Default Team

In VSTS the default project team has the project root as its area. Each of the other teams get an area under the project root:

clip_image001

The NIT Includes all Sub-Areas

In Nexus, there is a unique event that is added to all other events that normally occur in Scrum. This event is called the Nexus Sprint Planning session. In this meeting, the product owner and the NIT review the upcoming work and coordinate what PBI will be addressed in which sprint, by which team. In addition, the Nexus Integration Team is responsible for coordinating of the integration of all teams’ work, on a daily basis.

In order to accommodate these needs, the NIT’s backlogs and boards will be configured to view not only the work items (Epics, features, and/or PBIs) that are in the NIT’s area (the root area for the project), but also the work in each of the areas beneath it, i.e. work that has been associated with the Scrum teams.

This way, the integration team can visualize all of the PBIs on their board, and be ready whenever a PBI is ready for integration (e.g. moved to a column entitled “Integrate” – you will want to create this column).

Setting Up the Nexus Sprint Backlog

As previously mentioned, one of the NIT’s primary responsibilities is to coordinate work among the Scrum teams in order to mitigate interdependency risks. A commonly used tool is called the Nexus Sprint Backlog. The Nexus backlog is views the PBIs in a two-dimensional grid, where they look at each team’s sprint backlog for the upcoming and next few sprints. They then identify and mark each dependency, and note how risky it is:

· When interdependent PBIs are handled by different teams (e.g. 1 & 5, below), the risk is greater than when both PBIs are handled by the same team (e.g. 1 & 4) because coordination is more complex.

· When PBIs are not only handled by different teams, they also are expected to be completed in the same sprint (e.g. 4 & 5), the urgency requires greater coordination, and the risk is even higher.

· When there is a dependency on a PBI by someone outside the project, the risk is greater because there is likely to be an important commitment (e.g. 8). This risk goes up even further, if the PBI is needed by the end of this sprint (e.g. 9)!

image

While there is no Nexus Sprint Backlog in VSTS, as of the writing of this post, Microsoft’s Delivery Plans extension goes a long way towards giving us just this visualization. It is available in the marketplace so you can install it and use it to visualize your work across teams and sprints:

image

Visualizing Dependencies in VSTS

The delivery plans extension does not have the ability to visualize and track dependencies between backlog items. Yet. The product group has mentioned that this capability is on the roadmap, but I cannot confirm when this will be delivered.

In the meantime, we need to come up with a way to mitigate this drawback.

Work Item Relationship Links

One thing that you can do is mark a work item to be the predecessor of another. To do this, simply open the PBI, and under Related Work, click Add link to an existing work item. Select the work item that this work item depends upon:

image

While this will set the work items’ interdependencies, this will not show up in the plan. At least not as of today.

Tags

While this requires some manual work, I would suggest adding tags marking an item as “cross team dependent” or “external dependency”. Tags do show up in the plan and can augment it to give the NIT an at-a-glance idea of the risk involved with this project. Note the tags marked by the arrows in the screenshot below:

image

Summary

We have seen how teams wishing to use Scrum and Nexus to drive their development efforts, can set up VSTS to visualize and track their work. The following is a checklist for everything that you need to do:

1. All teams including the integration team work in the same Team Project

2. The Integration Team is the default team

3. The integration team’s area is the project root (e.g. \MyProject)

4. The Scrum teams’ areas are set under the integration team’s (e.g. under \MyProject\ScrumTeam1)

5. The integration team’s area should include all sub-areas

6. Add a column board entitled “Integrate” to mark PBIs that are ready to be integrated with the entire product

7. Install the Delivery Plan extension if it isn’t already installed for your account

8. Set up a delivery plan to include the Scrum teams and their sprints

9. Use the Predecessor link type to denote a dependency

10. Mark the kind of dependency with a tag, so that it can be seen in the plan.

I hope you find this useful. If you have found other tip and ideas to help drive Nexus, please share them with us!

Thanks,
Assaf

Posted in Agile, Scrum, Team, User Story, VSTS | Tagged , , , , , , | 1 Comment