Description
I want to use notebook tasks to integrate ArcGIS Enterprise with other non-GIS systems, but I’m encountering challenges when they use security to determine access. A notebook task runs as it’s Portal owner and is governed by sharing group permissions within Portal, which is ok, but this security context has no meaning when accessing external systems using other authentication methods like usernames and passwords, shared secrets, multi-factor authentication, token-based authentication and so on.
The reason I want to use a notebook is the flexibility of using python for ETL processing. On the occasions where I’m willing to trade off data freshness for performance gains with read-heavy workloads, I like copying data into hosted feature layers and tables. Offloading computation to a one-time or periodic refresh, or pre-aggregating data can eliminate the need to scan massive tables repeatedly.
If an ArcGIS Enterprise Portal is configured to use Security Assertion Markup Language (SAML) only, with built-in accounts disabled, and MFA is required, the implication is notebooks can only belong to and be run by actual Portal users. Real user accounts designed for humans have broader permissions that might not align with the limited scope needed for service-to-service interaction. Having notebook tasks assigned to portal users may not be the best.
Microsoft Azure provides several solutions to access resources within their ecosystem using Microsoft Entra. A Managed identity is a feature that allows Azure services to authenticate and access other Azure resources securely without needing to manage credentials like secrets or keys. If you do need store and manage sensitive information like secrets, keys and certificates there are Azure Key Vaults.
The only example I could find in the sample notebooks shows a technique of keeping secrets in a comma separated values (csv) file stored as a portal item. I would like to have something more sophisticated and following the principle of least privilege I want this permission to be assigned to a specific notebook. I don’t know if it would be better to have something that integrates with Microsoft Azure and runs on the cloud (ArcGIS Enterprise) or be the cloud (ArcGIS Online) with something more agnostic.
What is a way I can access secure non-GIS resources from Notebook Server running on Linux within Microsoft Azure?