Fixes issue #937: Show Offset and VirtAddr for language-specific strings (Go and Rust)

[divyansh360]

This PR enhances the verbose output to display raw file offsets and virtual addresses for language-specific strings (Go and Rust)
File Offsets tested to match using Hex Viewer.
Virtual Addresses tested to match using r2.
Fixes: #937

Go Strings
Sample: 28f18dd863198ca06ee20bdd6893296ffd487099c8c3868550724b0484bc6057

Rust Strings (inspired and enhanced from #1019)
Sample: acc31048e00d1a0f4cd5569d5d4db539da8f506cc7a6a171942d015ecc817d43

Considering the string "readme.txt"

Looking forward to your feedback and guidance towards more contribution opportunities.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[gemini-code-assist]

Hello @divyansh360, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request addresses issue #937 by enhancing the verbose output of FLOSS to display raw file offsets and virtual addresses for language-specific strings in Go and Rust binaries. It introduces new functions to calculate file offsets for both languages and integrates these into the verbose output rendering. The changes also include updates to .gitignore to exclude the floss-venv/ directory and *.code-workspace files.

Highlights

• Feature Enhancement: Adds file offset and virtual address display for Go and Rust strings in verbose output.
• Go String Offset Calculation: Introduces get_file_offset_in_blob in floss/language/go/extract.py to determine the file offset of the string blob.
• Rust String Offset Calculation: Introduces get_file_offset_in_rdata in floss/language/rust/extract.py to determine the file offset of the .rdata section.
• Output Rendering: Modifies floss/render/default.py to include file offset and virtual address in the verbose output table for language-specific strings.

Changelog

Click here to see the changelog

• .gitignore
  • Added floss-venv/ to exclude the virtual environment directory.
  • Added *.code-workspace to exclude VS Code workspace files.
• floss/language/go/extract.py
  • Introduced get_file_offset_in_blob function to calculate the file offset of the string blob.
  • This function calculates the file offset based on the image base, virtual address, and pointer to raw data.
• floss/language/rust/extract.py
  • Introduced get_file_offset_in_rdata function to calculate the file offset of the .rdata section.
  • This function calculates the file offset based on the image base, virtual address, and pointer to raw data of the .rdata section.
• floss/main.py
  • Added conditional execution of floss.language.go.extract.get_file_offset_in_blob(sample) when the language is Go and verbose mode is enabled (line 694).
  • Added conditional execution of floss.language.rust.extract.get_file_offset_in_rdata(sample) when the language is Rust and verbose mode is enabled (line 710).
• floss/render/default.py
  • Modified render_language_strings function to include file offset and virtual address in the verbose output table.
  • Added logic to display Offset and VirtAddr columns when verbose mode is not DEFAULT (lines 182-201).
  • The virtual address is calculated by adding the file offset to the string offset (line 197).
• floss/results.py
  • Added file_offset: Optional[int] = None to the Metadata dataclass to store the file offset (line 210).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Did you know?

Rust's ownership system and borrow checker prevent many common programming errors, such as null pointer dereferences, data races, and memory leaks, at compile time.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This PR enhances the floss tool to display raw file offsets and virtual addresses for language-specific strings (Go and Rust). The changes look good overall, and the included screenshots are helpful for understanding the feature. I have a few suggestions for improvements.

Summary of Findings

• Error Handling in get_file_offset_in_blob and get_file_offset_in_rdata: The functions get_file_offset_in_blob and get_file_offset_in_rdata return -1 on error. Consider raising an exception instead to provide more context about the failure.
• Duplication of Logic: The functions get_file_offset_in_blob in floss/language/go/extract.py and get_file_offset_in_rdata in floss/language/rust/extract.py contain similar logic. Consider refactoring this into a common utility function.
• Clarity of Variable Names: In get_file_offset_in_blob and get_file_offset_in_rdata, the variable name pointer_to_raw_data might be misleading as it stores the offset, not a pointer. Consider renaming it to raw_data_offset for clarity.

Merge Readiness

The code changes are well-structured and enhance the functionality of the floss tool. However, addressing the error handling and potential code duplication would improve the robustness and maintainability of the code. I am unable to approve this pull request, and recommend that another reviewer does so. I recommend that the pull request not be merged until the high severity issues are addressed.

[mr-tz]

Cool, thanks! Can you please add some tests showing the offsets are calculated as expected?

[mr-tz]

Additionally let's check first if the data can be added to the string classes/datatypes during extraction.

[divyansh360]

Cool, thanks! Can you please add some tests showing the offsets are calculated as expected?

I edited the description to include these.
Also, an interesting thing I noticed is that, for Go strings, the initial few strings (untill "runtime method:" string) can be correctly matched with their offsets in hex viewer, but for virtual address a difference of 0xe00 needs to be made to reach to correct virtual address. I observed this same pattern in multiple Go samples, and also all of them had the same initial few strings requiring the adjustment.
For instance consider the string "l512"

I feel this is because of the initial Go runtime metadata block. I tried to dynamically make the adjustment by running a while loop to check for and use the first non-runtime string as a reference point, to stop making the adjustment.

But is this a proper fix? Would appreciate your guidance about the possible cause and how to proceed. Thanks!

[divyansh360]

Additionally let's check first if the data can be added to the string classes/datatypes during extraction.

Sure, I'll check if I can add it

[williballenthin]

But is this a proper fix? Would appreciate your guidance about the possible cause and how to proceed.

I'm not sure why this is so, but I think its worth a bit more investigation. I don't really like the strategy of brute forcing the offset delta unless there's no other way. I wonder if some of the data is stored in a separate section that is aligned differently than the file?