[tmva][sofie] Softmax contiguous memory fast path #19743
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lmoneta
requested changes
Aug 26, 2025
| auto stride = UTILITY::ComputeStrideFromShape(fShape); | ||
| size_t size = fShape.size(); | ||
| auto length_str = ConvertDimShapeToLength(fShape); | ||
| size_t length = std::stoul(length_str); |
There was a problem hiding this comment.
you don't need to convert to an integer, since in some case it might fail : e.g length = " N * 32"
| // Check if this is the special case where memory is contiguous. | ||
| if (axis == static_cast<int>(size - 1)) { | ||
| size_t axis_size = std::stoul(fShape[axis].GetVal()); | ||
| size_t num_rows = length / axis_size; |
There was a problem hiding this comment.
In case of length is not an integer , this could be written as following:
std::string axis_size = fShape[axis].GetVal());
std::string num_rows;
if (IsInteger(length) && IsInteger(axis_size)) {
num_rows = std::to_string( std::stoul(length_str)/ std::stoul(fShape[axis].GetVal()));
} else {
num_rows = "(" + length_str + ") /" + fShape[axis].GetVal():
}
olia110
force-pushed
the
feature/contiguous-memory-fast-path
branch
from
c51c2d9 to
5960569
Compare
sanjibansg
approved these changes
Aug 26, 2025
lmoneta
approved these changes
Aug 26, 2025
| auto length = ConvertDimShapeToLength(fShape); | ||
| auto stride = UTILITY::ComputeStrideFromShape(fShape); | ||
| size_t size = fShape.size(); | ||
| auto length_str = ConvertDimShapeToLength(fShape); | ||
| int axis = fAttrAxis < 0 ? size + fAttrAxis : fAttrAxis; |
There was a problem hiding this comment.
Here we could have "size_t" for axis instead of int
| out << SP << SP << "tensor_" << fNY << "[i] /= sum;\n"; | ||
|
|
||
| // Check if this is the special case where memory is contiguous. | ||
| if (axis == static_cast<int>(size - 1)) { |
There was a problem hiding this comment.
if we define axis as size_t we don;t need the casting here
| } | ||
|
|
||
| out << "\n" << SP << "//------ SOFTMAX - " << size << " " << length_str << " " << axis << "\n"; | ||
| out << SP << "for (size_t i = 0; i < " << num_rows << "; ++i) {\n"; |
There was a problem hiding this comment.
we could use here int in the generated code to save memory, as it is done in the general case below
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This Pull request:
An optimized code path has been added for the common case where the Softmax axis is the last dimension (axis == size - 1).