Skip to content

Add ability to choose a different formatter #646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Aug 5, 2025
Merged

Conversation

DannyBen
Copy link
Member

@DannyBen DannyBen commented Aug 1, 2025

cc #644, #645

This PR comes to replace #645, and solve the consecutive heredoc newlines using a different approach.

  1. There is now a new setting: formatter: internal.
  2. Setting it to none will avoid compacting newlines completely.
  3. Setting it to external will format it using shfmt --case-indent --indent 2.
  4. Setting it to any other string (e.g. shfmt --minify) will assume this is the command to run, and run it as a formatter. The command will be sent the pre-formatted script through stdin, and is expected to print the formatted script to stdout.

The previous String#lint extension method was split into two:

  1. The new String#remove_private_comments which removes the ## comments from the script
  2. The newline compaction responsibility is now in a new Script::Formatter class

TODO

  • Proof of concept
  • Finalize implementation
  • Existing tests pass
  • New code has new specs
  • 100% spec coverage
  • Documentation - Source or Preview

@meleu
Copy link
Collaborator

meleu commented Aug 1, 2025

The shfmt option is not working for me.

Steps to reproduce:

$ # go to the minimal example
$ cd bashly/examples/minimal/

$ echo 'formatter: shfmt' > bashly-settings.yml
$ bashly generate
creating user files in src
skipped src/root_command.sh (exists)
 ArgumentError
wrong first argument

$ # 💥 error here 👆

Confirming that other options are working fine

$ echo 'formatter: none' > bashly-settings.yml
$ bashly generate
creating user files in src
skipped src/root_command.sh (exists)
created ./download
run ./download --help to test your bash script

$ # 👍 look the generated script and confirm the 'none' option
$ # worked as expected (there are indeed extra lines in the code)

$ echo 'formatter: internal' > bashly-settings.yml
$ bashly generate
creating user files in src
skipped src/root_command.sh (exists)
created ./download
run ./download --help to test your bash script

$ # 👍 I confirm the generated script looks like expected.

$ # 😈 trying to break it
$ echo 'formatter: invalid-option' > bashly-settings.yml

$ bashly generate
creating user files in src
skipped src/root_command.sh (exists)
created ./download
run ./download --help to test your bash script

$ # 😯 Oh! The `invalid-option` is ignored and the final script is
$ # formatted just like using `formatter: internal`.

@DannyBen
Copy link
Member Author

DannyBen commented Aug 1, 2025

Thanks for the test. I am pushing a version in a few that passes all specs in my Ruby 3.4, and we can then see if it passes in other Rubys. I can't tell anything from the ArgumentError you observed, other than the fact that this is an internal Ruby error, and not a bashly / shfmt error - so perhaps differences between our ruby versions - which one you use?

@meleu
Copy link
Collaborator

meleu commented Aug 1, 2025

$ ruby --version
ruby 3.4.4 (2025-05-14 revision a38531fd3f) +PRISM [x86_64-linux]

$ shfmt --version
3.12.0

@DannyBen
Copy link
Member Author

DannyBen commented Aug 1, 2025

Ok - so specs pass now (although there is no spec for the new formatter yet).

I am not sure why you get that ArgumentError, need your help in tracking it.

Can you run DEBUG=1 bashly generate with the shfmt formatter?
It should output the full backtrace.

Another debugging option, is to open the lib/bashly/script/formatter.rb file and put debugger before or after the call to Open3.capture3 to see if the error occurs before or after. (but before that, do export DEBUGGER=1 so that the debug gem will be required).

@DannyBen DannyBen changed the title Add ability to choose a different formatter (internal, none, shfmt) Add ability to choose a different formatter (internal, none, shfmt) Aug 1, 2025
@meleu
Copy link
Collaborator

meleu commented Aug 1, 2025

The error seems to be related to the fact that Open3#capture3 expects to receive a string as the first argument, and you're passing an array of strings

Open3.capture3 %w[shfmt --case-indent --indent 2], stdin_data: script

The documentation also mentions that you can use several strings in args, like this:

Open3.capture3('echo', 'C #')
# => ["C #\n", "", #<Process::Status: pid 2282368 exit 0>]
Open3.capture3('echo', 'hello', 'world')
# => ["hello world\n", "", #<Process::Status: pid 2282372 exit 0>]

But I think that as you're passing an array of strings followed by an implicit hash {stdin_data: script}, the whole array of strings is being considered as the first argument (I'm assuming you were expecting the array to be automatically "unpacked" and given as distinct arguments to the method).

DEBUG=1 output
$ DEBUG=1 bashly generate
creating user files in src
skipped src/root_command.sh (exists)
/home/meleu/.local/share/mise/installs/ruby/3.4.4/bin/bashly:25:in '<main>'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/bin/bashly:25:in 'Kernel#load'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/bin/bashly:9:in '<top (required)>'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/mister_bin-0.8.1/lib/mister_bin/runner.rb:41:in 'MisterBin::Runner#run'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/mister_bin-0.8.1/lib/mister_bin/runner.rb:56:in 'MisterBin::Runner#execute'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/mister_bin-0.8.1/lib/mister_bin/command.rb:26:in 'MisterBin::Command.execute'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/mister_bin-0.8.1/lib/mister_bin/command.rb:17:in 'MisterBin::Command#execute'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/commands/generate.rb:35:in 'Bashly::Commands::Generate#run'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/commands/generate.rb:55:in 'Bashly::Commands::Generate#generate'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/commands/base.rb:23:in 'Bashly::Commands::Base#with_valid_config'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/commands/generate.rb:57:in 'block in Bashly::Commands::Generate#generate'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/commands/generate.rb:76:in 'Bashly::Commands::Generate#generate_all_files'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/commands/generate.rb:159:in 'Bashly::Commands::Generate#create_master_script'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/script/wrapper.rb:14:in 'Bashly::Script::Wrapper#code'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/script/wrapper.rb:26:in 'Bashly::Script::Wrapper#base_code'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/script/wrapper.rb:32:in 'Bashly::Script::Wrapper#clean_code'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/script/formatter.rb:18:in 'Bashly::Script::Formatter#formatted_script'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/gems/3.4.0/gems/bashly-1.2.13/lib/bashly/script/formatter.rb:32:in 'Bashly::Script::Formatter#shfmt_result'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/3.4.0/open3.rb:658:in 'Open3.capture3'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/3.4.0/open3.rb:235:in 'Open3.popen3'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/3.4.0/open3.rb:534:in 'Open3.popen_run'
/home/meleu/.local/share/mise/installs/ruby/3.4.4/lib/ruby/3.4.0/open3.rb:534:in 'Kernel#spawn'
 ArgumentError
wrong first argument

By the way, I've tested with Open3.capture3 "shfmt", stdin_data: script and it works beautifully. 👌

@DannyBen
Copy link
Member Author

DannyBen commented Aug 1, 2025

Excellent. I am pushing a version in a few with support for custom command, stay tuned.

@DannyBen DannyBen added this to the 1.2.14 milestone Aug 1, 2025
@DannyBen
Copy link
Member Author

DannyBen commented Aug 1, 2025

I am ready to merge.

@meleu
Copy link
Collaborator

meleu commented Aug 1, 2025

I'll be available for careful testing over this weekend.

@DannyBen
Copy link
Member Author

DannyBen commented Aug 1, 2025

I'll be available for careful testing over this weekend.

Cool. I will also be available for fixes. I am marking this PR as pending your approval.

@DannyBen DannyBen requested a review from meleu August 1, 2025 19:04
@DannyBen
Copy link
Member Author

DannyBen commented Aug 2, 2025

@pcrockett since it was you who nudged us in this direction, if you have time to review, I am sure it will be beneficial.

@DannyBen DannyBen requested a review from pcrockett August 2, 2025 05:18
@DannyBen DannyBen changed the title Add ability to choose a different formatter (internal, none, shfmt) Add ability to choose a different formatter Aug 2, 2025
@pcrockett
Copy link
Collaborator

@DannyBen it might take me a few days to get around to thoroughly reviewing or testing. but i am interested in it. in any case i won't be offended if you merge this before i get to it.

@pcrockett
Copy link
Collaborator

in any case, the feature as described by the docs sounds great. 👍

@DannyBen
Copy link
Member Author

DannyBen commented Aug 2, 2025

it might take me a few days to get around to thoroughly reviewing

No pressure. I will wait for several days at least as I prefer merging after your input than having to change it later.
Since it is a bigger than usual feature, I also assigned the new version 1.3 instead of 1.2.x.

@DannyBen
Copy link
Member Author

DannyBen commented Aug 2, 2025

@EmilyGraceSeville7cf since I see you are around - if you have comments on my settings schema update, let me know.

@meleu
Copy link
Collaborator

meleu commented Aug 2, 2025

I've manually tested the scenarios specified in spec/bashly/script/formatter_spec.rb and, as expected, I confirm they work as specified.

Other scenarios that are not in the spec and I've tested:

  • formatter: external when shfmt is not available
    • ✔️ correctly fails with Bashly::Error: Command not found: shfmt
  • formatter: rm -rf src/bashly.yml
    • Well, it really deletes the file. But the users should be smart enough to not do stupid things like this. Then, it's a ✔️

Copy link
Collaborator

@meleu meleu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with the results of this PR.

Nice job! 👍

Copy link
Collaborator

@pcrockett pcrockett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I played with it. I like it. Looks good.

I briefly had some doubts about the "custom command" mode. The only purpose it would serve is to allow users to specify their formatting commands in a Bashly settings file rather than a Makefile or GH Actions workflow.

But I think that can be kind of nice -- you control your script output from one YAML file. So it's worth keeping, especially since it looks like it'll be fairly straightforward to maintain.

@DannyBen
Copy link
Member Author

DannyBen commented Aug 5, 2025

briefly had some doubts about the "custom command" mode

Me too, exactly.

The main reason I eventually accepted it, is the fact that it helps to cleanly solve the problem that the blind newline compaction of the built in "formatter" caused. So instead of just providing a toggle to disable it, meaning people would have to use outside formatter if they wanted a nice script, we now have a way to still provide a complete flow.

@DannyBen DannyBen merged commit 9726e44 into master Aug 5, 2025
7 checks passed
@DannyBen DannyBen deleted the add/lint-customization branch August 5, 2025 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants