[FR]: tar rule should support runfiles with symlinks

[alexeagle]

What is the current behavior?

We had to do some stuff to make js_image_layer work with rules_pkg.

Describe the feature

Should have feature parity with those, and be able to make clean bazelbuild/examples directories showing container builds for JS, Python, Java, etc with no userland starlark code.

[alexeagle]

I think it's okay for this to slip to 2.1 as it's not a breaking change.

[alexeagle]

I'm on the fence about shipping 2.0 final so long as you can't replace rules_pkg in our js_image_layer example. @thesayyn WDYT?

[thesayyn]

we can land that after 2.0. it's non-breaking change.

[alexeagle]

IIUC, We'll probably want to wait for bazelbuild/bazel#19944 to be available in Bazel 7.0 and direct users to upgrade so that we can handle the cases correctly under bzlmod.

[liningpan]

Hi! Is there a plan for implementing this? In the absence of this feature, is the recommended way to use a custom binary like js_image_layer?

[thesayyn]

If you don't care about symlinks tar rule should work for you already, js_image_layer is special as symlinks needs to be preserved in the resulting layers.

[liningpan]

IIUC, the tar rule basically dereferences symlinks, which causes a lot of duplication in the resulting archive.

[manuelnaranjo]

It sounds like https://github.com/aspect-build/bazel-lib/blob/main/lib/private/tar.bzl#L227 is missing a bit for symlinks

[manuelnaranjo]

I quickly hacked this bookingcom/bazel-lib@HEAD, I'm going to test it in our use case, what you think? Does it look like something we could merge?

[alexeagle]

Ooh thanks! @thesayyn could you TAL?

[manuelnaranjo]

FYI I tested it for our case and made a few improvements:

• I made the symlinks relative so that if I inspect the .tar the links make sense
• Symlinks can't have path being absolute
• For type=link we need link=target
  BTW I love this approach, it makes way more sense to express our container images with rules_oci and mtree manipulation than what the inspect and filter layer with rules_docker. At least now I can easily explain my peers what it does

[thesayyn]

I quickly hacked this bookingcom@HEAD, I'm going to test it in our use case, what you think? Does it look like something we could merge?

Ah unfortunately, this does not work if something is reported as symlink but not added to the runfiles.symlinks. We need to figure out symlinks are runtime as a post process action, probably using awk to make some system calls to mutate mtree to reflect the symlinks which can't detected at analysis time.

This is why we need a js_image_layer rule that has its own tar generator. https://github.com/aspect-build/rules_js/blob/98081d0fb66b3619f5a191ea29765012849d13f7/js/private/image/index.ts#L84-L104