Merge branch 'master' into DRIVERS-2782

Author: papafe

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,6 +2,12 @@
 
 Please complete the following before merging:
 
+- [ ] Is the relevant DRIVERS ticket in the PR title?
+
+<!--  All repo changes beyond typos now require a DRIVERS ticket; if you do not check the above box supply your reasoning below REASON BEGIN -->
+
+<!--  REASON END -->
+
 - [ ] Update changelog.
 - [ ] Test changes in at least one language driver.
 - [ ] Test these changes against all server versions and topologies (including standalone, replica set, and sharded

source/auth/auth.md

@@ -955,10 +955,6 @@ Examples are provided below.
 
 - mechanism_properties
 
-    - AWS_SESSION_TOKEN
-
-        Drivers MUST allow the user to specify an AWS session token for authentication with temporary credentials.
-
     - AWS_CREDENTIAL_PROVIDER
 
         An AWS [Custom Credential Provider](#custom-credential-providers) that returns AWS credentials. Drivers MAY allow
@@ -970,11 +966,10 @@ Examples are provided below.
 Drivers will need AWS IAM credentials (an access key, a secret access key and optionally a session token) to complete
 the steps in the
 [Signature Version 4 Signing Process](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html?shortFooter=true).
-If a username and password are provided drivers MUST use these for the AWS IAM access key and AWS IAM secret key,
-respectively. If, additionally, a session token is provided Drivers MUST use it as well. If a username is provided
-without a password (or vice-versa) or if *only* a session token is provided Drivers MUST raise an error. In other words,
-regardless of how Drivers obtain credentials the only valid combination of credentials is an access key ID and a secret
-access key or an access key ID, a secret access key and a session token.
+Regardless of how Drivers obtain credentials the only valid combination of credentials is an access key ID and a secret
+access key or an access key ID, a secret access key and a session token. These values MUST be present in the environment
+or be retrieved via the optional AWS SDK. If credentials are provided in the URI or client options, the driver MUST
+raise an error.
 
 AWS recommends using an SDK to "take care of some of the heavy lifting necessary in successfully making API calls,
 including authentication, retry behavior, and more".
@@ -1011,11 +1006,10 @@ Drivers MAY expose API for default providers for the following scenarios when ap
 
 The order in which Drivers MUST search for credentials is:
 
-1. The URI
-2. A custom AWS credential provider if the driver supports it.
-3. Environment variables
-4. Using `AssumeRoleWithWebIdentity` if `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_ARN` are set.
-5. The ECS endpoint if `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` is set. Otherwise, the EC2 endpoint.
+1. A custom AWS credential provider if the driver supports it.
+2. Environment variables
+3. Using `AssumeRoleWithWebIdentity` if `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_ARN` are set.
+4. The ECS endpoint if `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` is set. Otherwise, the EC2 endpoint.
 
 > [!NOTE]
 > See *Should drivers support accessing Amazon EC2 instance metadata in Amazon ECS* in [Q & A](#q-and-a)
@@ -1025,32 +1019,16 @@ The order in which Drivers MUST search for credentials is:
 > description of `AssumeRole` below, which is distinct from `AssumeRoleWithWebIdentity` requests that are meant to be
 > handled directly by the driver.
 
-##### URI
-
-An example URI for authentication with MONGODB-AWS using AWS IAM credentials passed through the URI is as follows:
-
-```javascript
-"mongodb://<access_key>:<secret_key>@mongodb.example.com/?authMechanism=MONGODB-AWS"
-```
-
-Users MAY have obtained temporary credentials through an
-[AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) request. If so, then in addition
-to a username and password, users MAY also provide an `AWS_SESSION_TOKEN` as a `mechanism_property`.
-
-```javascript
-"mongodb://<access_key>:<secret_key>@mongodb.example.com/?authMechanism=MONGODB-AWS&authMechanismProperties=AWS_SESSION_TOKEN:<security_token>"
-```
-
 ##### Environment variables
 
 AWS Lambda runtimes set several
 [environment variables](https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html#configuration-envvars-runtime)
 during initialization. To support AWS Lambda runtimes Drivers MUST check a subset of these variables, i.e.,
 `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN`, for the access key ID, secret access key and
-session token, respectively if AWS credentials are not explicitly provided in the URI. The `AWS_SESSION_TOKEN` may or
-may not be set. However, if `AWS_SESSION_TOKEN` is set Drivers MUST use its value as the session token. Drivers
-implemented in programming languages that support altering environment variables MUST always read environment variables
-dynamically during authorization, to handle the case where another part the application has refreshed the credentials.
+session token, respectively. The `AWS_SESSION_TOKEN` may or may not be set. However, if `AWS_SESSION_TOKEN` is set
+Drivers MUST use its value as the session token. Drivers implemented in programming languages that support altering
+environment variables MUST always read environment variables dynamically during authorization, to handle the case where
+another part the application has refreshed the credentials.
 
 However, if environment variables are not present during initial authorization, credentials may be fetched from another
 source and cached. Even if the environment variables are present in subsequent authorization attempts, the driver MUST
@@ -2165,6 +2143,8 @@ practice to avoid this. (See
 
 ## Changelog
 
+- 2025-09-30: Remove support for explicitly specifying MONGODDB-AWS authentication properties.
+
 - 2025-09-10: Update precedence of MONGODB-AWS credential fetching behaviour.
 
 - 2025-01-29: Add support for custom AWS credential providers.

source/auth/tests/legacy/connection-string.json

@@ -320,6 +320,15 @@ tests:
     mechanism: MONGODB-AWS
     mechanism_properties:
       AWS_SESSION_TOKEN: token!@#$%^&*()_+
+- description: should throw an exception if username provided (MONGODB-AWS)
+  uri: mongodb://[email protected]/?authMechanism=MONGODB-AWS
+  valid: false
+- description: should throw an exception if username and password provided (MONGODB-AWS)
+  uri: mongodb://user:[email protected]/?authMechanism=MONGODB-AWS
+  valid: false
+- description: should throw an exception if AWS_SESSION_TOKEN provided (MONGODB-AWS)
+  uri: mongodb://localhost/?authMechanism=MONGODB-AWS&authMechanismProperties=AWS_SESSION_TOKEN:token
+  valid: false
 - description: should recognise the mechanism with test environment (MONGODB-OIDC)
   uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=ENVIRONMENT:test
   valid: true

source/auth/tests/legacy/connection-string.yml

@@ -12,8 +12,7 @@ Drivers MUST test the following scenarios:
 7. Caching of AWS credentials fetched by the driver.
 
 For brevity, this section gives the values `<AccessKeyId>`, `<SecretAccessKey>` and `<Token>` in place of a valid access
-key ID, secret access key and session token (also known as a security token). Note that if these values are passed into
-the URI they MUST be URL encoded. Sample values are below.
+key ID, secret access key and session token (also known as a security token). Sample values are below.
 
 ```text
 AccessKeyId=AKIAI44QH8DHBEXAMPLE
@@ -37,14 +36,7 @@ directly from the custom provider instead of using the AWS SDK default provider.
 
 ### 2. Custom Credential Provider Authentication Precedence
 
-#### Case 1: Credentials in URI Take Precedence
-
-Create a `MongoClient` configured with AWS auth and credentials in the URI. Example:
-`mongodb://<AccessKeyId>:<SecretAccessKey>@localhost:27017/?authMechanism=MONGODB-AWS`
-
-Configure a custom credential provider to pass valid AWS credentials. The provider must track if it was called.
-
-Expect authentication to succeed and the custom credential provider was *not* called.
+#### Case 1: Credentials in URI Take Precedence *Removed*
 
 #### Case 2: Custom Provider Takes Precedence Over Environment Variables
 
@@ -59,11 +51,11 @@ Expect authentication to succeed and the custom credential provider was called.
 
 ## Regular credentials
 
-Drivers MUST be able to authenticate by providing a valid access key id and secret access key pair as the username and
-password, respectively, in the MongoDB URI. An example of a valid URI would be:
+Drivers MUST be able to authenticate when a valid access key id and secret access key pair are present in the
+environment. Drivers MUST provide the --nouri option to aws_tester.py in drivers-evergreen-tools for this test.
 
 ```text
-mongodb://<AccessKeyId>:<SecretAccessKey>@localhost/?authMechanism=MONGODB-AWS
+mongodb://localhost/?authMechanism=MONGODB-AWS
 ```
 
 ## EC2 Credentials
@@ -95,11 +87,11 @@ mongodb://localhost/?authMechanism=MONGODB-AWS
 ## AssumeRole
 
 Drivers MUST be able to authenticate using temporary credentials returned from an assume role request. These temporary
-credentials consist of an access key ID, a secret access key, and a security token passed into the URI. A sample URI
-would be:
+credentials consist of an access key ID, a secret access key, and a security token present in the environment. Drivers
+MUST provide the --nouri option to aws_tester.py in drivers-evergreen-tools for this test. A sample URI would be:
 
 ```text
-mongodb://<AccessKeyId>:<SecretAccessKey>@localhost/?authMechanism=MONGODB-AWS&authMechanismProperties=AWS_SESSION_TOKEN:<Token>
+mongodb://localhost/?authMechanism=MONGODB-AWS
 ```
 
 ## Assume Role with Web Identity

source/auth/tests/mongodb-aws.md

@@ -2519,6 +2519,9 @@ explicit session parameter as described in the [Drivers Sessions Specification](
 
 ## Changelog
 
+- 2025-09-30: Update `$lookup` prose test to reflect changes in
+    [MONGOCRYPT-793](https://jira.mongodb.org/browse/MONGOCRYPT-793).
+
 - 2025-08-06: Add `TextPreview` algorithm.
 
 - 2024-02-19: Add custom options AWS credential provider.

source/client-side-encryption/client-side-encryption.md

@@ -0,0 +1,3 @@
+{
+   "bsonType": "object"
+}

source/client-side-encryption/etc/data/lookup/schema-non-csfle.json

@@ -3457,6 +3457,7 @@ Use `encryptedClient` to drop and create the following collections:
 - `db.qe2` with options: `{ "encryptedFields": "<schema-qe2.json>"}`.
 - `db.no_schema` with no options.
 - `db.no_schema2` with no options.
+- `db.non_csfle_schema` with options: `{ "validator": { "$jsonSchema": "<schema-non-csfle.json>"}}`
 
 Create an unencrypted MongoClient named `unencryptedClient`.
 
@@ -3472,6 +3473,7 @@ Insert documents with `encryptedClient`:
     - Use `unencryptedClient` to retrieve it. Assert the `qe2` field is BSON binary.
 - `{"no_schema": "no_schema"}` into `db.no_schema`
 - `{"no_schema2": "no_schema2"}` into `db.no_schema2`
+- `{"non_csfle_schema": "non_csfle_schema"}` into `db.non_csfle_schema`
 
 #### Case 1: `db.csfle` joins `db.no_schema`
 
@@ -3652,7 +3654,7 @@ Expect one document to be returned matching:
 
 #### Case 8: `db.csfle` joins `db.qe`
 
-Test requires server 8.1+ and mongocryptd/crypt_shared 8.1+.
+Test requires server 8.1+ and mongocryptd/crypt_shared 8.1+
 
 Recreate `encryptedClient` with the same `AutoEncryptionOpts` as the setup. (Recreating prevents schema caching from
 impacting the test).
@@ -3673,7 +3675,12 @@ Run an aggregate operation on `db.csfle` with the following pipeline:
 ]
 ```
 
-Expect an exception to be thrown with a message containing the substring `not supported`.
+Expect an exception to be thrown with a message containing one of the following substrings depending on the
+mongocryptd/crypt_shared and libmongocrypt versions:
+
+- mongocryptd/crypt_shared \<8.2 **or** libmongocrypt \<1.17.0: `not supported`.
+- mongocryptd/crypt_shared 8.2+ **and** libmongocrypt 1.17.0+:
+    `Cannot specify both encryptionInformation and csfleEncryptionSchemas unless csfleEncryptionSchemas only contains non-encryption JSON schema validators`.
 
 #### Case 9: test error with \<8.1
 
@@ -3700,6 +3707,31 @@ Run an aggregate operation on `db.csfle` with the following pipeline:
 
 Expect an exception to be thrown with a message containing the substring `Upgrade`.
 
+#### Case 10: `db.qe` joins `db.non_csfle_schema`
+
+Test requires server 8.2+, mongocryptd/crypt_shared 8.2+, and libmongocrypt 1.17.0+.
+
+Recreate `encryptedClient` with the same `AutoEncryptionOpts` as the setup. (Recreating prevents schema caching from
+impacting the test).
+
+Run an aggregate operation on `db.qe` with the following pipeline:
+
+```json
+[
+    {"$match" : {"qe" : "qe"}},
+    {
+        "$lookup" : {
+            "from" : "non_csfle_schema",
+            "as" : "matched",
+            "pipeline" : [ {"$match" : {"non_csfle_schema" : "non_csfle_schema"}}, {"$project" : {"_id" : 0, "__safeContent__" : 0}} ]
+        }
+    },
+    {"$project" : {"_id" : 0, "__safeContent__" : 0}}
+]
+```
+
+Expect one document to be returned matching: `{"qe" : "qe", "matched" : [ {"non_csfle_schema" : "non_csfle_schema"} ]}`.
+
 ### 26. Custom AWS Credentials
 
 These tests require valid AWS credentials for the remote KMS provider via the secrets manager (FLE_AWS_KEY and

source/client-side-encryption/tests/README.md

@@ -324,7 +324,7 @@ tests:
           databaseName: *database_name
           collectionName: *encrypted_collection_name
       - name: assertIndexNotExists
-        object: "db"
+        object: testRunner
         arguments:
           databaseName: *database_name
           collectionName: *encrypted_collection_name