Visual Studio Code using Azure Pipelines
September 12, 2018 João Moreno, @joaomoreno
One of my responsibilities as a developer on the Visual Studio Code team is to maintain and improve our build and continuous integration (CI) infrastructure. Given the latest feature announcements from Azure Pipelines, the Visual Studio Code team has dramatically changed how we leverage Microsoft's technologies to provide a better collaboration platform for both our developers as well as our users. In this blog post, I'll guide you through a bit of Visual Studio Code's history, focusing on our CI processes and tools and how they have changed over time.
Visual Studio Code Engineering
Like any other open-source project, we need to have the right tooling and capabilities to receive, triage, and address as many code contributions as possible. This rings especially true in the developer tools universe, in which users are developers themselves: they are a passionate, hard-working, and very effective group. As of this blog post, we have 148 open PRs as well as 3,482 closed ones, which sets an average of ~3 PRs/day given the 3-year project lifespan so far. It is important that we are well equipped for handling this scale of contributions, not only to keep the project development healthy but also to help give other open-source projects an example of how to operate at this scale. Part of how we do this is by streamlining our workflow by bringing the PR experience into the editor, but CI is the other important part of handling contributions at scale.
Up until very recently, we relied on the OSS community's default choices for public continuous integration: Travis CI for our Linux and macOS builds and AppVeyor for Windows. Additionally, we used Coveralls to provide detailed test coverage reports. These services provide quality reports for PRs and code branches on our public repository, as they automate compilation, run code hygiene checks and execute several test suites, all of which is essential in order to maintain quality in a distributed team with lots of incoming contributions. This combination of services requires the understanding and maintenance of at least 3 different systems, each with its own special file formats, syntax, quirks, limitations, etc.
Adopting Azure Pipelines
Earlier this year, we were reached out by the Azure Pipelines (then Visual Studio Team Services) team to try out something new. This announcement marks our move to a more streamlined continuous integration solution. Our builds now run simultaneously across all platforms, check it out:
There's a lot of cool stuff which needed to happen in order for us to make the move. Let's break it down:
- Azure Pipelines support for public projects enable us to run a public-facing Visual Studio Code project in which all our continuous integration builds run;
- Build Agents in Azure Pipelines have long supported the Windows, macOS, and Linux platform matrix;
- Microsoft-hosted agents in Azure Pipelines running macOS, Linux, and Windows provide a great stack of software to build projects without worrying about build machine maintenance;
- YAML CI allows creating YAML definitions which are kept close to the project's sources (for which Visual Studio Code provides great extensions).
Putting all of this together, we're finally able to focus on a single CI solution. The Visual Studio Code build on Azure Pipelines runs our compilation, hygiene checks and test suites in a single build, automatically distributing the build across different platforms. Since we're using Microsoft-hosted build agents, we don't have to worry about maintaining those machines.
Third Party Integrations
Azure Pipelines also provides GitHub integration which gives us build result indicators across our GitHub project page, namely in pull requests.
We've also built a chat bot which hooks up to Azure Pipeline's REST API and provides notifications to our internal chat when builds break.
Going forward
My next task will be to take advantage of code coverage reports in order to get a better end-to-end CI flow than we had with the previous tool mix.
Having made the jump to Azure Pipelines has proved to be a big success for us. Overall code quality is easier to reason about now since builds aren't scattered all around. We have also consolidated the number and format of our build definition files. We're very happy with the change and excited about what the future holds for Azure Pipelines.
If you'd like to learn more about public projects and Azure Pipelines, take a look at their blog post.
Would you like to give Visual Studio Code a try? Download it now for your platform of choice. And if you're like us, and always want to run the latest and greatest, then get our daily-built Insider release. Do you simply want to reach out or keep in touch? Follow us @code on Twitter.
On behalf of the VS Code team: Happy Coding!
João Moreno, @joaomoreno