[Feature] Changes Cosmos to use MI connection #4026

brendankowitz · 2024-08-14T16:30:57Z

Description

This pull request introduces several changes across multiple files to enhance Azure resource management, improve Cosmos DB configuration, and add new features to the FHIR server configuration. The most important changes include adding new package versions, modifying Azure deployment templates, and updating Cosmos DB configurations to support managed identity.

Azure Deployment Template Updates:

samples/templates/default-azuredeploy-docker.json: Added Cosmos DB role assignments, database, and configuration settings for FHIR server deployments. [1] [2] [3] [4] [5] [6]

Cosmos DB Configuration Improvements:

src/Microsoft.Health.Fhir.CosmosDb.Core/Configs/CosmosDataStoreConfiguration.cs: Added properties to support managed identity and refined database creation settings. [1] [2]
src/Microsoft.Health.Fhir.CosmosDb.Core/Features/Storage/ICollectionSetup.cs: Updated interface to include a method for installing stored procedures and versioned collection settings updates.
src/Microsoft.Health.Fhir.CosmosDb.Core/Features/Storage/ICosmosClientTestProvider.cs: Simplified the PerformTestAsync method signature.

FHIR Server Configuration Updates:

src/Microsoft.Health.Fhir.Api/Configs/FhirServerConfiguration.cs: Added ResourceManagerConfig to the FHIR server configuration.
src/Microsoft.Health.Fhir.Core/Configs/ResourceManagerConfig.cs: Introduced a new configuration class for resource management.

Azure Resource Management Enhancements:

Directory.Packages.props: Added Azure.ResourceManager.CosmosDB package version 1.3.2.
build/jobs/add-aad-test-environment.yml: Fixed resource group name handling in the Azure Key Vault restoration script.
build/jobs/provision-deploy.yml: Introduced variables for deployment name and resource group name to streamline the script.

These changes collectively enhance the Azure resource management capabilities, improve the deployment process, and add new configuration options for Cosmos DB and FHIR server setups.

Related issues

Addresses AB#95002.

Supersedes/includes relevant changes from the following PRs:

Testing

Describe how this change was tested.

FHIR Team Checklist

Update the title of the PR to be succinct and less than 65 characters
Add a milestone to the PR for the sprint that it is merged (i.e. add S47)
Tag the PR with the type of update: Bug, Build, Dependencies, Enhancement, New-Feature or Documentation
Tag the PR with Open source, Azure API for FHIR (CosmosDB or common code) or Azure Healthcare APIs (SQL or common code) to specify where this change is intended to be released.
Tag the PR with Schema Version backward compatible or Schema Version backward incompatible or Schema Version unchanged if this adds or updates Sql script which is/is not backward compatible with the code.
CI is green before merge
Review squash-merge requirements

Semver Change (docs)

Feature (new capability to connect to Cosmos with MI)

src/Microsoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ArmSdkCollectionSetup.cs

src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/FhirCosmosClientInitializer.cs

src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/CosmosDbCollectionPhysicalPartitionInfo.cs

src/Microsoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ArmSdkCollectionSetup.cs

build/jobs/add-aad-test-environment.yml

src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/CosmosDbCollectionPhysicalPartitionInfo.cs

+                if (_dataStoreConfiguration.UseManagedIdentity)
+                {
+                    resourceKey = await GenerateAuthTokenAad(
+                        host,
+                        cancellationToken);
+                }
+                else
+                {
+                    resourceKey = await GenerateAuthToken(
                        "get",
                        "pkranges",
                        $"dbs/{_dataStoreConfiguration.DatabaseId}/colls/{_collectionConfiguration.CollectionId}",
                        date,
-                        key));
+                        key);
+                }


src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/FhirCosmosClientInitializer.cs

brendankowitz · 2024-08-15T19:14:08Z

src/Microsoft.Health.Fhir.Shared.Web/Startup.cs

@@ -355,6 +356,7 @@ private void AddTelemetryProvider(IServiceCollection services)
        {
            var configuration = new TelemetryConfiguration();
            Configuration.GetSection("Telemetry").Bind(configuration);
+            services.AddTransient<IMeterFactory, DummyMeterFactory>();


@fhibf this failed to resolve for me when running locally against cosmos, any ideas?

src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/CosmosContainerProvider.cs

brendankowitz · 2024-08-15T21:19:58Z

src/Microsoft.Health.Fhir.CosmosDb.Core/Configs/CosmosDataStoreConfiguration.cs

+        public bool UseManagedIdentity { get; set; }
+
+        public bool AllowDatabaseCreation { get; set; } = true;
+
+        public bool AllowCollectionSetup { get; set; } = true;


@apurvabhaleMS - We'll need these new settings in managed service :)

UseManagedService = true AllowDatabaseCreation = false AllowCollectionSetup = false

Though in thinking about it, if we add the Cosmos DB Operator permission too it would setup the db automatically, like it used to...

UseManagedService = true AllowDatabaseCreation = false AllowCollectionSetup = true

src/Microsoft.Health.Fhir.Shared.Web/appsettings.json

...osoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ResourceManagerCollectionSetup.cs

apurvabhaleMS · 2024-09-03T18:06:09Z

...crosoft.Health.Fhir.CosmosDb/Registration/FhirServerBuilderCosmosDbRegistrationExtensions.cs

@@ -181,7 +181,21 @@ private static IFhirServerBuilder AddCosmosDbPersistence(this IFhirServerBuilder
                .Transient()
                .AsService<ICollectionDataUpdater>();

+            services.Add<ResourceManagerCollectionSetup>()


Would ResourceManagerCollectionSetup always be invoked?
I think for OSS it should, thinking more in terms of SF and AKS?

It is registered as self, but not always resolved. If none of the setup functionality is required then you can register the default:
services.RemoveAll<ICollectionSetup>()
services.Add<ICollectionSetup, DefaultCollectionSetup>()

Updated to below. When AllowDatabaseCreation and AllowCollectionSetup are false then it will return the noop provider

...osoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ResourceManagerCollectionSetup.cs

brendankowitz · 2024-09-05T22:08:59Z

/azp run

azure-pipelines · 2024-09-05T22:09:19Z

Azure Pipelines successfully started running 1 pipeline(s).

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch from de060ba to 975ffd6 Compare August 14, 2024 16:35

github-advanced-security bot found potential problems Aug 14, 2024

View reviewed changes

src/Microsoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ArmSdkCollectionSetup.cs Fixed Show fixed Hide fixed

src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/FhirCosmosClientInitializer.cs Fixed Show fixed Hide fixed

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch 2 times, most recently from e34fe66 to 48202d5 Compare August 14, 2024 19:25

github-advanced-security bot found potential problems Aug 14, 2024

View reviewed changes

src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/CosmosDbCollectionPhysicalPartitionInfo.cs Fixed Show fixed Hide fixed

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch 3 times, most recently from 793e7c8 to 2519ee2 Compare August 15, 2024 00:10

github-advanced-security bot found potential problems Aug 15, 2024

View reviewed changes

src/Microsoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ArmSdkCollectionSetup.cs Fixed Show fixed Hide fixed

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch from 2519ee2 to 45952ea Compare August 15, 2024 00:54

brendankowitz commented Aug 15, 2024

View reviewed changes

build/jobs/add-aad-test-environment.yml Outdated Show resolved Hide resolved

github-advanced-security bot found potential problems Aug 15, 2024

View reviewed changes

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch 4 times, most recently from 4e0eb3a to ec96065 Compare August 15, 2024 05:56

brendankowitz commented Aug 15, 2024

View reviewed changes

brendankowitz added this to the S147 milestone Aug 15, 2024

brendankowitz added Enhancement Enhancement on existing functionality. Azure API for FHIR Label denotes that the issue or PR is relevant to the Azure API for FHIR Open source This change is only relevant to the OSS code or release. labels Aug 15, 2024

brendankowitz marked this pull request as ready for review August 15, 2024 19:19

brendankowitz requested a review from a team as a code owner August 15, 2024 19:19

brendankowitz changed the title ~~[Draft] Changes Cosmos to use MI connection~~ [Feature] Changes Cosmos to use MI connection Aug 15, 2024

github-advanced-security bot found potential problems Aug 15, 2024

View reviewed changes

src/Microsoft.Health.Fhir.CosmosDb/Features/Storage/CosmosContainerProvider.cs Fixed Show resolved Hide resolved

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch 3 times, most recently from cb2cd50 to a1d5b8f Compare August 15, 2024 21:15

brendankowitz commented Aug 15, 2024

View reviewed changes

apurvabhaleMS reviewed Aug 19, 2024

View reviewed changes

src/Microsoft.Health.Fhir.Shared.Web/appsettings.json Outdated Show resolved Hide resolved

This was referenced Aug 19, 2024

Cosmos ARM changes for Managed Identity Authentication #3969

Closed

Cosmos managed Identity Authentication Changes #3950

Closed

brendankowitz modified the milestones: S147, S148 Aug 20, 2024

apurvabhaleMS reviewed Aug 28, 2024

View reviewed changes

...osoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ResourceManagerCollectionSetup.cs Show resolved Hide resolved

apurvabhaleMS reviewed Aug 28, 2024

View reviewed changes

...osoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ResourceManagerCollectionSetup.cs Show resolved Hide resolved

apurvabhaleMS previously approved these changes Sep 2, 2024

View reviewed changes

apurvabhaleMS reviewed Sep 3, 2024

View reviewed changes

...osoft.Health.Fhir.CosmosDb.Initialization/Features/Storage/ResourceManagerCollectionSetup.cs Show resolved Hide resolved

brendankowitz added 4 commits September 3, 2024 13:28

Cosmos OSS MI

89c4331

Update to install procs with ARM

28e8338

Fixes

8aced35

Refactored cosmos initialization

92e84ec

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch from a0ee0b6 to dec47bc Compare September 3, 2024 20:30

Feedback

db42bec

brendankowitz dismissed apurvabhaleMS’s stale review via db42bec September 5, 2024 16:49

brendankowitz force-pushed the personal/bkowitz/cosmos-mi-arm branch from dec47bc to db42bec Compare September 5, 2024 16:49

apurvabhaleMS approved these changes Sep 5, 2024

View reviewed changes

brendankowitz merged commit 66ba9a5 into main Sep 6, 2024
44 of 46 checks passed

brendankowitz deleted the personal/bkowitz/cosmos-mi-arm branch September 6, 2024 04:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Changes Cosmos to use MI connection #4026

[Feature] Changes Cosmos to use MI connection #4026

brendankowitz commented Aug 14, 2024 •

edited

Loading

brendankowitz Aug 15, 2024

brendankowitz Aug 15, 2024 •

edited

Loading

apurvabhaleMS Sep 3, 2024

brendankowitz Sep 4, 2024 •

edited

Loading

brendankowitz Sep 5, 2024

brendankowitz commented Sep 5, 2024

azure-pipelines bot commented Sep 5, 2024

[Feature] Changes Cosmos to use MI connection #4026

[Feature] Changes Cosmos to use MI connection #4026

Conversation

brendankowitz commented Aug 14, 2024 • edited Loading

Description

Azure Deployment Template Updates:

Cosmos DB Configuration Improvements:

FHIR Server Configuration Updates:

Azure Resource Management Enhancements:

Related issues

Supersedes/includes relevant changes from the following PRs:

Testing

FHIR Team Checklist

Semver Change (docs)

brendankowitz Aug 15, 2024

Choose a reason for hiding this comment

brendankowitz Aug 15, 2024 • edited Loading

Choose a reason for hiding this comment

apurvabhaleMS Sep 3, 2024

Choose a reason for hiding this comment

brendankowitz Sep 4, 2024 • edited Loading

Choose a reason for hiding this comment

brendankowitz Sep 5, 2024

Choose a reason for hiding this comment

brendankowitz commented Sep 5, 2024

azure-pipelines bot commented Sep 5, 2024

brendankowitz commented Aug 14, 2024 •

edited

Loading

brendankowitz Aug 15, 2024 •

edited

Loading

brendankowitz Sep 4, 2024 •

edited

Loading