-
Notifications
You must be signed in to change notification settings - Fork 2.1k
build system: use thread-safe stdio #21438
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
9a524a2
to
f058796
Compare
We should probably give this a full build without fast fail, otherwise you'll be chasing your tail for days 😅 |
No, actually the memory requirements will go down with this. Unless... both printf variants get linked in into the same app. See #21439 for a fix of that. |
f058796
to
31f945b
Compare
in one of your commit messages and in the description of this PR you have every misspelled.
|
473d6ca
to
933acf3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this!
Unfortunately, something seems to fail on the first board I've tested this with:
$ make -C tests/sys/snprintf BOARD=adafruit-feather-nrf52840-sense clean flash term
make: Entering directory '/home/mikolai/TUD/Code/RIOT/tests/sys/snprintf'
rm -rf /home/mikolai/TUD/Code/RIOT/tests/sys/snprintf/bin/adafruit-feather-nrf52840-sense/pkg-build/cmsis
rm -rf /home/mikolai/TUD/Code/RIOT/tests/sys/snprintf/bin/adafruit-feather-nrf52840-sense/pkg-build/mpaland-printf
Building application "tests_snprintf" for "adafruit-feather-nrf52840-sense" with CPU "nrf52".
"make" -C /home/mikolai/TUD/Code/RIOT/pkg/cmsis/
"make" -C /home/mikolai/TUD/Code/RIOT/pkg/mpaland-printf/
"make" -C /home/mikolai/.riot/pkg/mpaland-printf -f /home/mikolai/TUD/Code/RIOT/pkg/mpaland-printf/mpaland-printf.mk
"make" -C /home/mikolai/TUD/Code/RIOT/boards
"make" -C /home/mikolai/TUD/Code/RIOT/boards/common/adafruit-nrf52-bootloader
"make" -C /home/mikolai/TUD/Code/RIOT/boards/common/init
"make" -C /home/mikolai/TUD/Code/RIOT/boards/adafruit-feather-nrf52840-sense
"make" -C /home/mikolai/TUD/Code/RIOT/core
"make" -C /home/mikolai/TUD/Code/RIOT/core/lib
"make" -C /home/mikolai/TUD/Code/RIOT/cpu/nrf52
"make" -C /home/mikolai/TUD/Code/RIOT/cpu/cortexm_common
"make" -C /home/mikolai/TUD/Code/RIOT/cpu/cortexm_common/periph
"make" -C /home/mikolai/TUD/Code/RIOT/cpu/nrf52/periph
"make" -C /home/mikolai/TUD/Code/RIOT/cpu/nrf52/vectors
"make" -C /home/mikolai/TUD/Code/RIOT/cpu/nrf5x_common
"make" -C /home/mikolai/TUD/Code/RIOT/cpu/nrf5x_common/periph
"make" -C /home/mikolai/TUD/Code/RIOT/drivers
"make" -C /home/mikolai/TUD/Code/RIOT/drivers/periph_common
"make" -C /home/mikolai/TUD/Code/RIOT/sys
"make" -C /home/mikolai/TUD/Code/RIOT/sys/auto_init
"make" -C /home/mikolai/TUD/Code/RIOT/sys/auto_init/usb
"make" -C /home/mikolai/TUD/Code/RIOT/sys/div
"make" -C /home/mikolai/TUD/Code/RIOT/sys/event
"make" -C /home/mikolai/TUD/Code/RIOT/sys/fmt
"make" -C /home/mikolai/TUD/Code/RIOT/sys/isrpipe
"make" -C /home/mikolai/TUD/Code/RIOT/sys/libc
"make" -C /home/mikolai/TUD/Code/RIOT/sys/luid
"make" -C /home/mikolai/TUD/Code/RIOT/sys/malloc_thread_safe
"make" -C /home/mikolai/TUD/Code/RIOT/sys/newlib_syscalls_default
"make" -C /home/mikolai/TUD/Code/RIOT/sys/preprocessor
"make" -C /home/mikolai/TUD/Code/RIOT/sys/stdio
"make" -C /home/mikolai/TUD/Code/RIOT/sys/stdio_uart
"make" -C /home/mikolai/TUD/Code/RIOT/sys/test_utils/interactive_sync
"make" -C /home/mikolai/TUD/Code/RIOT/sys/test_utils/print_stack_usage
"make" -C /home/mikolai/TUD/Code/RIOT/sys/tsrb
"make" -C /home/mikolai/TUD/Code/RIOT/sys/usb/usbus
"make" -C /home/mikolai/TUD/Code/RIOT/sys/usb/usbus/cdc/acm
"make" -C /home/mikolai/TUD/Code/RIOT/sys/usb_board_reset
text data bss dec hex filename
21252 128 4524 25904 6530 /home/mikolai/TUD/Code/RIOT/tests/sys/snprintf/bin/adafruit-feather-nrf52840-sense/tests_snprintf.elf
stty -F /dev/ttyACM0 raw ispeed 1200 ospeed 1200 cs8 -cstopb ignpar eol 255 eof 255
sleep 10
[INFO] uf2conv.py not found - fetching it from GitHub now
CC= CFLAGS= make -C /home/mikolai/TUD/Code/RIOT/dist/tools/uf2
[INFO] uf2conv.py successfully fetched!
SoftDevice version S140 6.1.1 found.
/home/mikolai/TUD/Code/RIOT/dist/tools/uf2/uf2conv.py -f 0xADA52840 /home/mikolai/TUD/Code/RIOT/tests/sys/snprintf/bin/adafruit-feather-nrf52840-sense/tests_snprintf.hex --base 0x26000
Converted to uf2, output size: 43008, start address: 0x26000
Flashing /media/mikolai/FTHRSNSBOOT (nRF52840-Feather-Sense)
Wrote 43008 bytes to /media/mikolai/FTHRSNSBOOT/NEW.UF2
sleep 2
/home/mikolai/TUD/Code/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200" -ln "/tmp/pyterm-mikolai" -rn "2025-08-21_10.12.25-tests_snprintf-adafruit-feather-nrf52840-sense"
Twisted not available, please install it if you want to use pyterm's JSON capabilities
2025-08-21 10:12:25,059 # Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
2025-08-21 10:12:26,061 # Help: Press s to start test, r to print it is ready
r
2025-08-21 10:12:27,896 # READY
s
2025-08-21 10:12:28,776 # START
2025-08-21 10:12:28,782 # main(): This is RIOT! (Version: 2025.10-devel-176-g18dac1-HEAD)
2025-08-21 10:12:28,785 # Testing snprintf() implementation...
2025-08-21 10:12:28,789 # Expected: "16045690984050070327"
2025-08-21 10:12:28,790 # Got: ""
2025-08-21 10:12:28,793 # Expected: "deadbeefaffe1337"
2025-08-21 10:12:28,795 # Got: ""
2025-08-21 10:12:28,798 # Expected: "0xdeadbeefaffe1337"
2025-08-21 10:12:28,800 # Got: ""
2025-08-21 10:12:28,803 # Expected: "DEADBEEFAFFE1337"
2025-08-21 10:12:28,804 # Got: ""
2025-08-21 10:12:28,808 # Expected: "0XDEADBEEFAFFE1337"
2025-08-21 10:12:28,809 # Got: ""
2025-08-21 10:12:28,813 # Expected: "1572555756765777411467"
2025-08-21 10:12:28,814 # Got: ""
2025-08-21 10:12:28,818 # Expected: "01572555756765777411467"
2025-08-21 10:12:28,820 # Got: ""
2025-08-21 10:12:28,824 # Expected: "-9223372036854775807"
2025-08-21 10:12:28,825 # Got: ""
2025-08-21 10:12:28,829 # Expected: "-9223372036854775807"
2025-08-21 10:12:28,830 # Got: ""
2025-08-21 10:12:28,832 # TEST FAILED!
2025-08-21 10:12:28,839 # { "threads": [{ "name": "main", "stack_size": 1536, "stack_used": 440}]}
The test succeeds on current master
. I'll try to debug later today.
I've also noted that https://github.com/RIOT-OS/RIOT/blob/18dac11c668e13bf428ac694083efc8ff177c866/tests/sys/snprintf/Makefile.board.dep already pulls in mpaland_printf
iff newlib
is selected. Should that be dropped within this PR?
help: newlib's stdio is not inherently thread-safe, but depends on the | ||
reentrancy hooks to be provided for correct operation. When |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What exactly is the reason we cannot provide such reentrancy hooks for newlib in RIOT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We actually could provide reentrancy hooks (just nobody did so yet). There even is a stale PR somewhere for it.
But that will be a hard sell unless it is opt-in due to the large RAM requirements (per thread!) it has. (The stale PR adds an opt-in module for reentrant newlib, if I recall correctly.)
The main reason people enable reentrancy in newlib is to have malloc()
/ free()
and friends thread-safe, which we already have with a much leaner implementation. The second reason is to have stdio thread-safe.
Reentrancy would also give you thread safety for a number of other standard C lib functions that operate on shared resources, but those are rarely ever used in the context of RIOT. (E.g. our VFS system fully bypasses the standard C lib anyway.) Once we have thread-safe stdio, there will be few use cases that would actually benefit from reentrancy, and some might rather rework the code than to pay the per-thread RAM overhead of newlib's reentrancy mechanism.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, thanks for the explanation! Would be great to have this noted down somewhere in the documentation for future reference, but I can't even find pages on neither newlib
nor picolibc
o.O
Would you be up for writing out some bits in a follow-up PR?
Ah sorry for the noise, that's probably the removed Should we rather add |
No, its IMO not noise. A test failing with default configuration is IMO a genuine bug :)
I did both. There might be users with tiny boards that can benefit from easily being able to disable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I can confirm the test is passing now for both configurations on the hardware I tested with (nrf52840dk
).
Please squash :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry, two nits more
- `arch_64bit` depends on long long support (rather than just enabling it by default), as otherwise `%p` will not work correctly ==> add hard dependency - On 32 bit platforms support of printing long long is not almost free ==> do not enable long long on `arch_32bit` by default - The ESP SDK contains binary blobs that already link against newlib and mpaland-printf is not ABI compatible with newlib's stdio, it is only API compatible. ==> mark mpaland-printf as incompatible to ESP MCUs Co-authored-by: crasbe <[email protected]>
This introduces a new feature category to declare which bugs are present in a given build that we cannot fix in RIOT, but need to work around. These bugs may be silicon bugs, software bugs in ROM, software bugs in binary blobs needed for a platform, or bugs in the toolchain. These features behave the same way as `arch_%` features: Every provided bug is always used, so that we can inspect `FEATURES_USED` to check which bugs need to be worked around. Co-authored-by: crasbe <[email protected]>
This makes use of the new bug modeling to declare all platforms that can use newlib and have no reentrancy hooks as affected by the non-thread-safe stdio bug. (Which is every platform but ESP* and AVR, the former because the reentrancy hooks are provided, the latter because we do not and never will support newlib on them.) Building on that, the mpaland-printf package is used when newlib is used and the bug is present. This way we can rely on the stdio being thread-safe on every platform and not causing random crashes at run time. Co-authored-by: mguetschow <[email protected]>
This drops a workaround that initialized newlib's reentrancy structure on boot to reduce the chances of crashes when using the non-thread-safe (unless reentrancy hooks are provided) stdio implementation of newlib. Now that the newlib stdio implementation is only ever used if it is thread-safe, we no longer need a workaround that reduces the chance of crashes on concurrent use of stdio.
That way users can rely on elimation of dead code from the optimizer instead of having to use preprocessor conditionals when feature-gating assert implementations behind `!NDEBUG`. Co-authored-by: benpicco <[email protected]> Co-authored-by: crasbe <[email protected]> Co-authored-by: mguetschow <[email protected]>
This avoids inconsistent output when external code gets linked in that directly links against newlib's assert implementation (e.g. binary blobs or packages that do not add `core/lib/include` to the include paths). This also greatly benefits wrapping printf, as newlib's `__assert_func()` directly links to internal printf functions of newlib. Co-authored-by: crasbe <[email protected]>
This is required to support printing 64 bit numbers, as is done in the test. In addition that 64 bit formatting test is feature gated, so that users can simply comment out the `USEMODULE += printf_long_long` to fit tiny boards. Co-authored-by: mguetschow <[email protected]>
ecdd4ea
to
4a84f6f
Compare
Yay 🎉 Thanks everyone for bearing with me on this one ❤️ |
This resulted in some failing nightly tests, as mpaland-printf prints As the printing format is indeed implementation-defined, the respective tests should probably be adapted to accept output both with and without |
Another problem, at least on my machine:
RIOT/pkg/mpaland-printf/Makefile.include Lines 16 to 25 in c0cc617
suggests that this is failing on purpose, but I'm not sure what to make out of this. Apparently that also didn't happen on CI for some reason (maybe it was not properly rebuilding all modules for the test?). |
There we go: #21677 |
Contribution description
build system: introduce
bug_%
feature categoryThis introduces a new feature category to declare which bugs are present
in a given build that we cannot fix in RIOT, but need to work around.
These bugs may be silicon bugs, software bugs in ROM, software bugs in
binary blobs needed for a platform, or bugs in the toolchain.
These features behave the same way as
arch_%
features: Every providedbug is always used, so that we can inspect
FEATURES_USED
to checkwhich bugs need to be worked around.
build system: use thread-safe stdio
This makes use of the new bug modeling to declare all platforms that
can use newlib and have no reentrancy hooks as affected by the
non-thread-safe stdio bug. (Which is every platform but ESP* and AVR,
the former because the reentrancy hooks are provided, the latter because
we do not and never will support newlib on them.)
Building on that, the mpaland-printf package is used when newlib is used
and the bug is present. This way we can rely on the stdio being
thread-safe on every platform and not causing random crashes at
run time.
sys/newlib: drop workaround for stdio
This drops a workaround that initialized newlib's reentrancy structure
on boot to reduce the chances of crashes when using the non-thread-safe
(unless reentrancy hooks are provided) stdio implementation of newlib.
Now that the newlib stdio implementation is only ever used if it is
thread-safe, we no longer need a workaround that reduces the chance
of crashes on concurrent use of stdio.
Testing procedure
Now the package
mpaland-printf
should be pulled in for all non-ESP boards when newlib is used. E.g.With this, the test in
tests/sys/snprintf
now should also pass for each and every board, except for ESP32 boards. The reason is that, since ESP* provides the reentrancy hooks, newlib's stdio is thread-safe on those. So we do not pull in the alternative printf. For ESP8266 boards newlib is configured to use the non-nano stdio, which passes the test. For ESP32 newlib-nano is used, which does not implement all format specifiers and, therefore, fails the test.Issues/PRs references
Depends on and includes: