Skip to content

Conversation

@johnbeard
Copy link

This is just the minimal expected workflow for how you can build and test changes.


As someone unfamiliar with both Scala (i.e. how do you drive this SBT thing?) and the Kaitai struct repo structure, I found it quite hard to figure out what the expected "development loop" is (i.e. what you basically need to do to "write come code, see if it works, adjust the code, see if it works"). I patched them together based on what seemed to work, of course let me know if I am doing it wrong!

These instructions don't actually currently work without patching, as the tests are broken by the Julia stuff as mentioned in kaitai-io/kaitai_struct#1127.

This is just the minimal expected workflow for how you can
build and test changes.
developers.adoc Outdated
* Use your new compiler directly: `./jvm/target/universal/stage/bin/kaitai-struct-compiler <args>`
* Running tests:
* Run tests from the `tests` repo
* Build the compiler if needed: `./build-compiler <lang>` (`lang` may be something like `cpp_stl_11`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps you meant build-tests (build-compiler is unnecessary since it does the same as sbt compilerJVM/stage and KSC is always compiled as a whole, not for a specific language - personally, I only use sbt compilerJVM/stage and never build-compiler as I mentioned here). It regenerates the language-specific unit test specs in spec/<lang> directories from spec/ks/*.kst files (see https://doc.kaitai.io/kst.html). However, I'd recommend ignoring build-tests and using ./spec_kst_to_all directly - it gives you much more control, as I wrote in kaitai-io/kaitai_struct_tests#108 (comment). I also personally always specify the -f/--force option:

Also, BTW, the -f/--force option could be enabled by default in my opinion (I'm a bit tired of always adding -f explicitly, but so far it hasn't bothered me that much to do anything about it), given that the generated test specs are tracked by Git and you can always rollback the changes you don't want, so you can't really lose any work if you use Git properly. Therefore, I really don't see the need for the special spec/ks/out output folder that is used by default.

So from my side, this should be more like ./spec_kst_to_all -f ... instead. The ... part depends on which test specs you want to regenerate and for which targets (≈ target languages). In most cases, you only want to regenerate specific tests (typically after making changes to existing spec/ks/*.kst files or adding more spec/ks/*.kst files for new test formats) - then you would use ./spec_kst_to_all -f <test_name>... (or ./spec_kst_to_all -f -t all <test_name>..., which matches the default setting, but perhaps it doesn't hurt to specify it explicitly).

But I'd argue that ./spec_kst_to_all doesn't need to be included in the "everyday" commands, because there is no reason to touch it unless you're changing or adding test formats, which is not that often (and when the time comes for it, it results in a commit to https://github.com/kaitai-io/kaitai_struct_tests). Perhaps it's better covered in https://doc.kaitai.io/kst.html#_invoking_kst_translator already.

developers.adoc Outdated

* Making changes in the `compiler` repo:
* Perform a "stage build" with SBT: `sbt compilerJVM/stage`
* Use your new compiler directly: `./jvm/target/universal/stage/bin/kaitai-struct-compiler <args>`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip: I have the ./jvm/target/universal/stage/bin directory permanently in my PATH, which is super useful because I never have to type that long path myself and can invoke kaitai-struct-compiler from any working directory.

And when you use terminal autocompletion to "help you" with typing this long path, you can accidentally call the wrong script in jvm/target/universal/scripts/bin/kaitai-struct-compiler, which won't work and will give you Error: Could not find or load main class io.kaitai.struct.JavaMain, see kaitai-io/kaitai_struct#1110. So it's error prone, which is why I wouldn't recommend it.

And fix asciidoctor list indentation >_<
@johnbeard
Copy link
Author

johnbeard commented Mar 19, 2025

Hi @generalmimon thank you so much for the detailed reply - it's exactly the kind of thing that I was in need of when cding about the place, Googling "what is Scala" and generally trying to get it working! I made some changes, and linked to the KST translator section.

Copy link
Contributor

@Mingun Mingun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, that this is great addition. Also, kaitai-io/kaitai_struct_tests#112 contains similar ideas for lowering entry level for newcomers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants