feat: Allow defining meaningful hostnames on multi-tesseract network #206

johnbcoughlin · 2025-06-05T17:51:25Z

Relevant issue or PR

Description of changes

Tesseracts are able to communicate with each other over the multi-tesseract network, but the host names by which they can be addressed are assigned by the SDK, are not human-readable, and are hard to discover. This change allows the user to pass a list of meaningful tesseract "service names" to tesseract serve or engine.serve. These service names function as meaningful identifiers by which tesseracts can refer to each other.

As an example, suppose we have started two tesseracts using the following tesseract serve command:

tesseract serve tesseract-a tesseract-b --service-names tesseract-a,tesseract-b

From within Tesseract A, we can run

with Tesseract.from_url("http://tesseract-b:8000") as tx:
    tx.apply(...)

Testing done

Added an end-to-end test verifying that tesseract-b is reachable at the expected hostname from tesseract-a.
Verified same manually using the example given above.

tests/endtoend_tests/test_endtoend.py

xalelax · 2025-06-06T09:47:24Z

Nice idea 👍 should we also validate whether service names are distinct?

zmheiko · 2025-06-06T12:50:44Z

I think we might also want to restrict service names such that the corresponding urls are safe.

johnbcoughlin · 2025-06-06T17:19:58Z

Nice idea 👍 should we also validate whether service names are distinct?

Yes, good point. Will add this.

PasteurBot · 2025-06-06T18:10:28Z

CLA signatures confirmed

All contributors have signed the Contributor License Agreement.
_{Posted by the CLA Assistant Lite bot.}

codecov · 2025-06-06T18:12:42Z

Codecov Report

Attention: Patch coverage is 26.08696% with 17 lines in your changes missing coverage. Please review.

Project coverage is 25.68%. Comparing base (8997bd0) to head (3f61ec2).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
tesseract_core/sdk/engine.py	26.08%	16 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (8997bd0) and HEAD (3f61ec2). Click for more details.

HEAD has 12 uploads less than BASE

Flag BASE (8997bd0) HEAD (3f61ec2)

24 12

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #206       +/-   ##
===========================================
- Coverage   76.21%   25.68%   -50.54%     
===========================================
  Files          28       24        -4     
  Lines        3078     3002       -76     
  Branches      480      487        +7     
===========================================
- Hits         2346      771     -1575     
- Misses        519     2143     +1624     
+ Partials      213       88      -125

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

johnbcoughlin · 2025-06-06T19:00:48Z

I think we might also want to restrict service names such that the corresponding urls are safe.

This is a surprisingly frustrating validation to perform. :(

We might want to validate that the string is a valid host name for a URL. The most correct python way to do that seems to be to use the validators package. I haven't explored this package in detail, but neither urllib nor requests seem to expose this API.
More specifically, we want to validate that the string is a valid docker-compose service name. Neither I nor Claude could find any official specification on docker-compose service names. The only thing was a PR from 2015 (!) which changed it to the following:

VALID_NAME_CHARS = '[a-zA-Z0-9\._\-]'

Unfortunately the docker compose implementation has changed completely since then, and I couldn't find the corresponding validation logic in the golang codebase.

What do people think about enforcing the regex

[a-zA-Z0-9][a-zA-Z0-9-]*

which I believe would be more restrictive than the docker-compose service name, and include only characters that can appear in valid host names.

zmheiko · 2025-06-07T16:33:33Z

What do people think about enforcing the regex
[a-zA-Z0-9][a-zA-Z0-9-]*
which I believe would be more restrictive than the docker-compose service name, and include only characters that can appear in valid host names.

I think that would probably work just fine. If we run into cases that require less restrictive names, we can always relax the constraints later without breaking anything.

johnbcoughlin · 2025-06-08T05:36:23Z

@PasteurBot I have read the CLA Document and I hereby sign the CLA

dionhaefner · 2025-06-09T13:11:53Z

Thanks @johnbcoughlin. That looks like a useful feature.

Is there a precedent for this feature in other applications / libraries (such as the Docker CLI, or docker-py)? I.e., is service-names a common name for this feature?

johnbcoughlin · 2025-06-09T16:56:46Z

Is there a precedent for this feature in other applications / libraries (such as the Docker CLI, or docker-py)? I.e., is service-names a common name for this feature?

Yes, I believe "service" is a very common name for this concept. Docker CLI or docker-py isn't the right level of abstraction though, we want to look at container orchestrators.

Kubernetes has services
Docker-compose obviously has services, that's what this is exposing.
Docker Swarm has services
Amazon ECS has Service Discovery

In e.g. Kubernetes or docker-compose you would be providing these strings as part of a yaml config, so there is no direct analogue for the CLI argument name "service-names".

dionhaefner · 2025-06-11T09:04:46Z

tesseract_core/sdk/cli.py

+        typer.Option(
+            "--service-names",
+            help=(
+                "Comma-separated list of service names by which each tesseract should be exposed "


Suggested change

"Comma-separated list of service names by which each tesseract should be exposed "

"Comma-separated list of service names by which each Tesseract should be exposed "

dionhaefner · 2025-06-11T09:05:15Z

tesseract_core/sdk/cli.py

+            help=(
+                "Comma-separated list of service names by which each tesseract should be exposed "
+                "in the shared network. "
+                "Tesseracts are reachable from one another at http://{service_name}:8000"


Suggested change

"Tesseracts are reachable from one another at http://{service_name}:8000"

"Tesseracts are reachable from one another at http://{service_name}:8000. "

"Not supported when using --no-compose."

dionhaefner · 2025-06-11T09:05:35Z

tesseract_core/sdk/cli.py

@@ -491,6 +502,11 @@ def serve(
    else:
        ports = None

+    if service_names is not None:


Let's check here if --no-compose is given and error out with a helpful message.

dionhaefner · 2025-06-11T09:08:59Z

tesseract_core/sdk/engine.py

    if no_compose:
        if len(images) > 1:
            raise ValueError(
                "Docker Compose is required to serve multiple Tesseracts. "
                f"Currently attempting to serve `{len(images)}` Tesseracts."
            )
+        if service_names is not None:
+            raise ValueError(
+                "Tesseract service names are only meaningful with Docker Compose."


Suggested change

"Tesseract service names are only meaningful with Docker Compose."

"Tesseract service names can only be set when using Docker Compose."

dionhaefner · 2025-06-11T09:09:40Z

tesseract_core/sdk/engine.py

+    if len(set(service_names)) != len(service_names):
+        raise ValueError("Service names must be unique")
+
+    print(service_names)


Suggested change

print(service_names)

dionhaefner · 2025-06-11T09:09:51Z

tesseract_core/sdk/engine.py

+    print(service_names)
+    invalid_names = []
+    for name in service_names:
+        print(name)


Suggested change

print(name)

dionhaefner · 2025-06-11T09:10:37Z

tesseract_core/sdk/engine.py

+    if len(invalid_names) != 0:
+        raise ValueError(
+            "Service names must contain only alphanumeric characters and hyphens, and must "
+            f"not begin with a hyphen. Found invalid names: f{invalid_names}."


Suggested change

f"not begin with a hyphen. Found invalid names: f{invalid_names}."

f"not begin with a hyphen. Found invalid names: {invalid_names}."

dionhaefner · 2025-06-11T09:10:49Z

tesseract_core/sdk/engine.py

+        print(name)
+        if not re.match(r'^[A-Za-z0-9][A-Za-z0-9-]*$', name):
+            invalid_names.append(name)
+    if len(invalid_names) != 0:


Suggested change

if len(invalid_names) != 0:

if invalid_names:

dionhaefner · 2025-06-11T09:11:23Z

tesseract_core/sdk/engine.py

+    invalid_names = []
+    for name in service_names:
+        print(name)
+        if not re.match(r'^[A-Za-z0-9][A-Za-z0-9-]*$', name):


Not even underscores?

dionhaefner · 2025-06-11T09:11:56Z

tests/endtoend_tests/test_endtoend.py

+            # I get a "tesseract_core has no attribute Tesseract" error from this??
+            #"import tesseract_core; tesseract_core.Tesseract.from_url(\"http://t-9000:8000\").health()"


This is expected, see comment thread

Suggested change

# I get a "tesseract_core has no attribute Tesseract" error from this??

#"import tesseract_core; tesseract_core.Tesseract.from_url(\"http://t-9000:8000\").health()"

dionhaefner · 2025-06-11T09:12:33Z

tests/endtoend_tests/test_endtoend.py

+    project_id = project_meta["project_id"]
+    project_containers = [project_meta["containers"][i]["name"] for i in range(2)]
+
+    T1 = docker_client.containers.get(project_containers[0])


Use lower case for non-class variable names

Suggested change

T1 = docker_client.containers.get(project_containers[0])

tess_1 = docker_client.containers.get(project_containers[0])

dionhaefner · 2025-06-11T09:13:19Z

tests/endtoend_tests/test_endtoend.py

+    except Exception as e:
+        assert False, e
+    finally:
+        return


This prevents cleanup from running!

dionhaefner · 2025-06-11T09:13:36Z

tests/endtoend_tests/test_endtoend.py

+    except Exception as e:
+        assert False, e


No need to do anything, just let the exception bubble up.

Suggested change

except Exception as e:

assert False, e

dionhaefner · 2025-06-11T09:14:06Z

tests/endtoend_tests/test_endtoend.py

+        if project_id:
+            run_res = cli_runner.invoke(
+                app,
+                [
+                    "teardown",
+                    project_id,
+                ],
+                catch_exceptions=False,
+            )
+            assert run_res.exit_code == 0, run_res.stderr


I suggest you use the docker_cleanup fixture instead

dionhaefner

Some minor comments, but overall this looks like a solid feature. Thanks @johnbcoughlin!

johnbcoughlin requested review from dionhaefner, xalelax and apaleyes as code owners June 5, 2025 17:51

johnbcoughlin force-pushed the jack-service-names branch 2 times, most recently from 04fb9a2 to 11a2173 Compare June 5, 2025 17:52

johnbcoughlin commented Jun 5, 2025

View reviewed changes

tests/endtoend_tests/test_endtoend.py Show resolved Hide resolved

johnbcoughlin added 2 commits June 6, 2025 11:10

Allow configuring service names on multi-tesseract network

b38cb69

Additional validation of service names

3f61ec2

johnbcoughlin force-pushed the jack-service-names branch from 11a2173 to 3f61ec2 Compare June 6, 2025 18:10

dionhaefner reviewed Jun 11, 2025

View reviewed changes

	"Comma-separated list of service names by which each tesseract should be exposed "
	"Comma-separated list of service names by which each Tesseract should be exposed "

	"Tesseracts are reachable from one another at http://{service_name}:8000"
	"Tesseracts are reachable from one another at http://{service_name}:8000. "
	"Not supported when using --no-compose."

	"Tesseract service names are only meaningful with Docker Compose."
	"Tesseract service names can only be set when using Docker Compose."

	f"not begin with a hyphen. Found invalid names: f{invalid_names}."
	f"not begin with a hyphen. Found invalid names: {invalid_names}."

		# I get a "tesseract_core has no attribute Tesseract" error from this??
		#"import tesseract_core; tesseract_core.Tesseract.from_url(\"http://t-9000:8000\").health()"

	T1 = docker_client.containers.get(project_containers[0])
	tess_1 = docker_client.containers.get(project_containers[0])

feat: Allow defining meaningful hostnames on multi-tesseract network #206

Are you sure you want to change the base?

feat: Allow defining meaningful hostnames on multi-tesseract network #206

Uh oh!

Conversation

johnbcoughlin commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issue or PR

Description of changes

Testing done

Uh oh!

Uh oh!

xalelax commented Jun 6, 2025

Uh oh!

zmheiko commented Jun 6, 2025

Uh oh!

johnbcoughlin commented Jun 6, 2025

Uh oh!

PasteurBot commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CLA signatures confirmed

Uh oh!

codecov bot commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

johnbcoughlin commented Jun 6, 2025

Uh oh!

zmheiko commented Jun 7, 2025

Uh oh!

johnbcoughlin commented Jun 8, 2025

Uh oh!

dionhaefner commented Jun 9, 2025

Uh oh!

johnbcoughlin commented Jun 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dionhaefner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

johnbcoughlin commented Jun 5, 2025 •

edited

Loading

PasteurBot commented Jun 6, 2025 •

edited

Loading

codecov bot commented Jun 6, 2025 •

edited

Loading