Extension integration tests can be run on CI services. The
vscode-test library helps you setup extension tests on CI providers and contains a sample extension setup on Azure Pipelines. You can check out the build pipeline or jump directly to the
Azure Pipelines is great for running VS Code extension tests as it supports running the tests on Windows, macOS and Linux. For Open Source projects, you get unlimited minutes and 10 free parallel jobs. This section explains how to setup an Azure Pipelines for running your extension tests.
Then, add the following
azure-pipelines.yml file to the root of your extension's repository. Other than the
xvfb setup script for Linux that is necessary to run VS Code in headless Linux CI machines, the definition is straight-forward:
trigger: - master strategy: matrix: linux: imageName: 'ubuntu-16.04' mac: imageName: 'macos-10.13' windows: imageName: 'vs2017-win2016' pool: vmImage: $(imageName) steps: - task: NodeTool@0 inputs: versionSpec: '8.x' displayName: 'Install Node.js' - bash: | /usr/bin/Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 & echo ">>> Started xvfb" displayName: Start xvfb condition: and(succeeded(), eq(variables['Agent.OS'], 'Linux')) - bash: | echo ">>> Compile vscode-test" yarn && yarn compile echo ">>> Compiled vscode-test" cd sample echo ">>> Run sample integration test" yarn && yarn compile && yarn test displayName: Run Tests env: DISPLAY: ':99.0'
Finally, create a new pipeline in your DevOps project and point it to the
azure-pipelines.yml file. Trigger a build and voilà:
You can enable the build to run continuously when pushing to a branch and even on pull requests. See Build pipeline triggers to learn more.
language: node_js os: - osx - linux node_js: 8 install: - | if [ $TRAVIS_OS_NAME == "linux" ]; then export DISPLAY=':99.0' /usr/bin/Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 & fi script: - | echo ">>> Compile vscode-test" yarn && yarn compile echo ">>> Compiled vscode-test" cd sample echo ">>> Run sample integration test" yarn && yarn compile && yarn test cache: yarn