AnyUrl adds trailing slash

Initial Checks

• I confirm that I'm using Pydantic V2

Description

AnyURL now adds a trailing slash in validation of URLs that dont have a trailing slash. This caused CORS issues for us, because CORS doesnt allow trailing slash (I think, seems to be written here: https://www.w3.org/TR/2008/WD-access-control-20080214/)

In my opinion i see this as a bug because I dont see the option toggle this addition of trailing slash. Maybe its expected behaviour, please let me know then how I can turn it off.

Example Code

from pydantic import AnyHttpUrl as AnyUrlv2
from pydantic.v1 import AnyHttpUrl as AnyUrlv1

print(AnyUrlv2("http://localhost:3000"))
print(AnyUrlv1("http://localhost:3000", scheme="http"))


prints:

http://localhost:3000/
http://localhost:3000

Python, Pydantic & OS Version

pydantic version: 2.2.1
        pydantic-core version: 2.6.1
          pydantic-core build: profile=release pgo=false
                 install path: /Users/charelfeltenrq/.pyenv/versions/3.10.8/lib/python3.10/site-packages/pydantic
               python version: 3.10.8 (main, Dec  2 2022, 11:54:08) [Clang 14.0.0 (clang-1400.0.29.202)]
                     platform: macOS-13.3.1-arm64-arm-64bit
     optional deps. installed: ['email-validator', 'typing-extensions']

Selected Assignee: @dmontagu

somewhat related: pydantic/pydantic-settings#104

At this point, to my understanding, it seems its intended to add the slash. Is it? In that case, I am interested in whats the optimal way to remove this trailing slash, without forgoing the other validation (as seems to be the proposed solution in the related issue)

This problem also becomes apparent in the url builder:

from pydantic import AnyHttpUrl as AnyHttpUrl

AnyHttpUrl.build(
    scheme="https",
    host="google.com",
    path="/search",
)

produces: Url('https://google.com//search')

Is that the desired outcome? If yes, then does it mean that all Url builds we have in our codebase should now have a path that starts without a slash?

[samuelcolvin]

The second case is unrelated.

This is done by the URL rust crate, so it probably mandated by a rfc somewhere.

[samuelcolvin]

Sorry, the second case is related.

Another weird situation

from pydantic import AnyHttpUrl as AnyHttpUrl

AnyHttpUrl.build(
    scheme="https",
    host="google.com/",
    path="search",
)

Url('https://google.com//search')

[adriangb]

Another weird situation

That's technically valid and works, not sure we need to do anything special there.

I think we might come up with a better solution for this long term but for now if you don't need a Url instance you could do something like:

from typing import Annotated, Any, List

from pydantic_core import CoreSchema, core_schema

from pydantic import AfterValidator, BeforeValidator, GetCoreSchemaHandler, TypeAdapter
from pydantic.networks import AnyUrl


class Chain:
    def __init__(self, validations: List[Any]) -> None:
        self.validations = validations
    
    def __get_pydantic_core_schema__(self, source_type: Any, handler: GetCoreSchemaHandler) -> CoreSchema:
        return core_schema.chain_schema(
            [
                *(handler.generate_schema(v) for v in self.validations),
                handler(source_type)
            ]
        )


def strip_slash(value: str) -> str:
    return value.rstrip('/')


MyAnyUrl = Annotated[str, AfterValidator(strip_slash), BeforeValidator(str), Chain([AnyUrl])]

ta = TypeAdapter(MyAnyUrl)

print(ta.validate_python('https://www.google.com'))
# https://www.google.com

Thank you for posting this workaround, I am not 100% sure whats going on there, but to my understanding it seems that it creates a type MyAnyUrl which is a str, but includes the validation from AnyUrl. That could be useful, maybe we will use it or a similar idea.

However, I am still a bit baffled that it seems to be seen as not a problem that AnyUrl adds trailing / to all Urls, to me this is not validation anymore as it actually changes the Url. And also .build creating Urls with // in them is problematic for our situation.

For now, we have decided to roll back to pydantic v1. Will there be any way to keep using the pydantic v1 Url behaviour (i.e. not adding /) in v2? We tried via from pydantic.v1 import AnyUrl but this caused errors inside BaseSettings. Or is this really the desired behaviour and you are not planning on changing it back?

Lastly, just wanted to mention, I am a very happpy user of pydantic, and I love the work you all do, just this change of behaviour of Url is giving us a bit of a headache.

[tworedz]

Also finding this new behavior very annoying. In pydantic v1 urls inherited from str, so it can be easily passed to library codes, where they usually expects string representation of urls. But now we need to str(model.url_field) or model.url_field.unicode_string() when passing urls, because now it is pydantic_core._pydantic_core.Url

I'm not sure which variant right (old or new url), but this little slash causes very strange and hours of uncatchable bug fixing.

FYI, I come up with this solution:

from typing import Annotated
from pydantic import AfterValidator, AnyUrl, BaseModel

URL = Annotated[AnyUrl, AfterValidator(lambda x: str(x).rstrip("/"))]

class Model(BaseModel):
    url: URL

print(Model(url="https://google.com").url)
# > https://google.com

But sadly we cannot use this solution everywhere, because sometimes we need the last slash.

Seems, that v2 url validator adding slashes only to naked urls, i.e. without path, which is acceptable:

from pydantic import AnyUrl, BaseModel

class Model(BaseModel):
    url: AnyUrl

print(Model(url="https://google.com").url)
# > https://google.com/

print(Model(url="https://google.com/").url)
# > https://google.com/

print(Model(url="https://google.com/api").url)
# > https://google.com/api

print(Model(url="https://google.com/api/").url)
# > https://google.com/api/

Would be nice, if old behavior preserves, or like author suggested, we can choose behavior type. Thanks in advance!