-
Notifications
You must be signed in to change notification settings - Fork 330
PR Pipeline - Code Coverage and Triggers #4360
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
benrr101
wants to merge
17
commits into
main
Choose a base branch
from
dev/russellben/pr-code-coverage
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
963bccc
First crack at adding test result publishing to non-manual test jobs.
benrr101 1ce8fca
If statement syntax?
benrr101 3e28bcd
Ok forget the if statement.
benrr101 824f0d2
Version
benrr101 7bd1466
Unique job names... could be better, but let's start here.
benrr101 7bc3a20
Tree in linux?
benrr101 3388570
Ok forget it, we don't need do emit the tree....
benrr101 412f2e0
Wire up test result uploading to sqlclient manual test jobs
benrr101 9818712
Give a test display name to sql manual tests
benrr101 800a688
First attempt at code coverage report publishing
benrr101 7056130
:robot: Code coverage takes a dynamic dependency on the platforms to …
benrr101 5acf9e3
We only need *one* set of parameters
benrr101 ba57adc
We only need *one* pool name
benrr101 f6f4726
Add triggers and fix code coverage verbosity thingy
benrr101 2f28f24
:robot: fixes for codecov CLI
benrr101 0172b62
:curly_haired_man: and :robot: comments
benrr101 aed4424
:robot: Use dotnet-tools.json
benrr101 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,6 +33,10 @@ parameters: | |
| - name: stageNameSecrets | ||
| type: string | ||
|
|
||
| # Name of the artifact to upload the test results (including code coverage) to. | ||
| - name: testResultsArtifactName | ||
| type: string | ||
|
|
||
| # Platform Parameters ==================================================== | ||
|
|
||
| # Display name of the platform. This will be formatted into the job's official name as well as | ||
|
|
@@ -128,6 +132,12 @@ jobs: | |
| - name: saPassword | ||
| value: $[stageDependencies.${{ parameters.stageNameSecrets }}.secrets_job.outputs['SaPassword.Value']] | ||
|
|
||
| - name: testDisplayName | ||
| value: "sqlclient_manual_${{ parameters.configDisplayName}}_${{ parameters.testSet }}" | ||
|
|
||
| - name: testResultsPath | ||
| value: "$(REPO_ROOT)/test_results/${{ parameters.platformDisplayName }}_$(testDisplayName)" | ||
|
|
||
| steps: | ||
| # Install dotnet and the runtime that will run the tests (if it is not netframework) | ||
| - template: /eng/pipelines/common/steps/install-dotnet.yml@self | ||
|
|
@@ -201,4 +211,13 @@ jobs: | |
| -p:BuildSuffix='${{ parameters.buildSuffix }}' | ||
| -p:TestFramework=${{ parameters.platformDotnet }} | ||
| -p:TestSet=${{ parameters.testSet }} | ||
| -p:TestResultsFolderPath=${{ variables.testResultsPath }} | ||
|
|
||
| # Publish test results | ||
| - template: /eng/pipelines/pr/steps/publish-test-results-step.yml@self | ||
| parameters: | ||
| buildConfiguration: ${{ parameters.buildConfiguration }} | ||
| platformDisplayName: ${{ parameters.platformDisplayName }} | ||
| testDisplayName: ${{ variables.testDisplayName }} | ||
| testResultsArtifactName: ${{ parameters.testResultsArtifactName }} | ||
| testResultsPath: ${{ variables.testResultsPath }} | ||
|
Comment on lines
212
to
+223
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That and the test result folder path isn't being passed into the build.proj step |
||
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| ################################################################################# | ||
| # Licensed to the .NET Foundation under one or more agreements. # | ||
| # The .NET Foundation licenses this file to you under the MIT license. # | ||
| # See the LICENSE file in the project root for more information. # | ||
| ################################################################################# | ||
|
|
||
| # This pipeline builds and tests all products that we release. It is intended to run as validation | ||
| # for every PR. The set of tests it executes is not exhaustive, as many tests require special | ||
| # environments to provide complete coverage. Instead, this PR strikes a balance between complete | ||
| # confidence and speed of PR validation. | ||
| # | ||
| # It is triggered by pushes to PRs that target the main, dev/*, feat/*, and release/* branches in | ||
| # GitHub. The dev/* pattern includes dev/automation/* branches created by AI agents. | ||
| # | ||
| # It maps to the "sqlclient-pr" pipeline in the Public ADO project: | ||
| # https://sqlclientdrivers.visualstudio.com/public/_build?definitionId=2281 | ||
| # And the "sqlclient-pr" pipeline in the internal ADO project: | ||
| # https://dev.azure.com/SqlClientDrivers/ADO.Net/_build?definitionId=2271 | ||
|
|
||
| # Set the pipeline run name to the day-of-year and the daily run counter. | ||
| name: $(DayOfYear)$(Rev:rr) | ||
|
|
||
| # Trigger PR validation runs for all pushes to PRs that target the specified branches. | ||
| # https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/github?view=azure-devops&tabs=yaml#pr-triggers | ||
| pr: | ||
| branches: | ||
| include: | ||
| # GitHub repo branch targets that will trigger PR validation builds. | ||
| - dev/* | ||
| - feat/* | ||
| - main | ||
| - release/* | ||
|
|
||
| paths: | ||
| include: | ||
| - .azuredevops/* | ||
| - .config/* | ||
| - doc/* | ||
| - eng/pipelines/common/* | ||
| - eng/pipelines/pr/* | ||
| - src/* | ||
| - tools/* | ||
| - azurepipelines-coverage.yml | ||
| - build.proj | ||
| - Directory.Packages.props | ||
| - dotnet-tools.json | ||
| - global.json | ||
| - NuGet.config | ||
|
|
||
| # Do not trigger commit or schedule runs for this pipeline. | ||
| trigger: none | ||
|
|
||
| parameters: | ||
| # General Parameters ===================================================== | ||
|
|
||
| # Dotnet CLI verbosity level. | ||
| - name: dotnetVerbosity | ||
| displayName: dotnet CLI Verbosity | ||
| type: string | ||
| default: normal | ||
| values: | ||
| - quiet | ||
| - minimal | ||
| - normal | ||
| - detailed | ||
| - diagnostic | ||
|
|
||
| - name: platforms | ||
| type: object | ||
| default: | ||
| - displayName: "windows_net462" | ||
| dotnet: "net462" | ||
| image: "ADO-MMS22-SQL22" | ||
| operatingSystem: "Windows" | ||
| - displayName: "windows_net8" | ||
| dotnet: "net8.0" | ||
| image: "ADO-MMS22-SQL22" | ||
| operatingSystem: "Windows" | ||
| - displayName: "windows_net9" | ||
| dotnet: "net9.0" | ||
| image: "ADO-MMS22-SQL22" | ||
| operatingSystem: "Windows" | ||
| - displayName: "windows_net10" | ||
| dotnet: "net10.0" | ||
| image: "ADO-MMS22-SQL22" | ||
| operatingSystem: "Windows" | ||
|
|
||
| - displayName: "linux_net8" | ||
| dotnet: "net8.0" | ||
| image: "ADO-UB22-SQL22" | ||
| operatingSystem: "Linux" | ||
| - displayName: "linux_net9" | ||
| dotnet: "net9.0" | ||
| image: "ADO-UB22-SQL22" | ||
| operatingSystem: "Linux" | ||
| - displayName: "linux_net10" | ||
| dotnet: "net10.0" | ||
| image: "ADO-UB22-SQL22" | ||
| operatingSystem: "Linux" | ||
|
|
||
| variables: | ||
| - template: /eng/pipelines/common/variables/common-variables.yml@self | ||
| - template: /eng/pipelines/pr/variables/pr-variables.yml@self | ||
|
|
||
| stages: | ||
| # Stage 1a: Build and pack all projects in the repository | ||
| - template: /eng/pipelines/pr/stages/pack-stage.yml@self | ||
| parameters: | ||
| buildConfiguration: Debug | ||
| buildSuffix: pr | ||
| stageName: ${{ variables.stageNamePack }} | ||
|
|
||
| # Stage 1b: Generate secrets | ||
| - template: /eng/pipelines/pr/stages/generate-secrets-stage.yml@self | ||
| parameters: | ||
| stageName: ${{ variables.stageNameSecrets }} | ||
|
|
||
| # Stage 2: Execute tests and collect code coverage | ||
| - template: /eng/pipelines/pr/stages/test-stages.yml@self | ||
| parameters: | ||
| buildConfiguration: Debug | ||
| buildSuffix: pr | ||
| dotnetVerbosity: ${{ parameters.dotnetVerbosity }} | ||
| poolName: $(PoolNameDefault) | ||
| stageNamePack: ${{ variables.stageNamePack }} | ||
| stageNameSecrets: ${{ variables.stageNameSecrets }} | ||
| testResultsArtifactName: ${{ variables.testResultsArtifactName }} | ||
|
|
||
| platforms: ${{ parameters.platforms }} | ||
|
|
||
| manualTestAzureKeyVaultUrl: $(AzureKeyVaultUrl) | ||
| manualTestAzureKeyVaultTenantId: $(AzureKeyVaultTenantId) | ||
| manualTestConnectionStringNpLocalhost: $(ConnectionStringNp_LocalhostDefault_UsernamePassword) | ||
| manualTestConnectionStringTcpLocalhost: $(ConnectionStringTcp_LocalhostDefault_UsernamePassword) | ||
| manualTestConnectionStringNpAzure: $(ConnectionStringNp_Azure_ManagedIdentity) | ||
| manualTestConnectionStringTcpAzure: $(ConnectionStringTcp_Azure_ManagedIdentity) | ||
| manualTestFileStreamDirectory: "$(Pipeline.Workspace)/filestream" | ||
| manualTestLocalDbAppName: $(LocalDbAppName) | ||
| manualTestLocalDbSharedInstanceName: $(LocalDbSharedInstanceName) | ||
| manualTestUserManagedIdentityClientId: $(UserManagedIdentityClientId) | ||
|
|
||
| # Stage 3: Collect code coverage | ||
| - template: /eng/pipelines/pr/stages/collect-coverage-stage.yml@self | ||
| parameters: | ||
| codeCovApiToken: $(CodeCovApiToken) | ||
| dependsOn: | ||
| - ${{ each platform in parameters.platforms }}: | ||
| - "test_${{ platform.displayName }}" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This actually isn't an issue, if you look at the test logs https://dev.azure.com/SqlClientDrivers/ADO.Net/_build/results?buildId=155942&view=logs&j=9aaebe84-0bb8-5049-010a-f5d7cc3b7829&t=6f7d4681-2ac1-5fe1-83e3-59d9d33c8750
I'm not sure there's a concept of "runtime variables" as is being asserted here. There are values that are only available at runtime, but as long as those are referenced by
$()they will remain$()until they are used at runtime, where they are further expanded to their value.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's worth noting that if this wasn't the case, we'd never be able to have $(Password) in our connection strings and expect it to be expanded when we write it to the config file.