June-2020 version for NXP MCUXpresso (example for Kinetis K64F)
See 2020 version for STM's STM32CubeIDE toolchain with newlib 3.0

Background

For reliability, smaller embedded systems typically don't use dynamic memory, thus avoiding associated problems of leakage, fragmentation, and resulting out-of-memory crashes. In Nadler & Associates smaller embedded projects (and large avionics products), we've historically used no runtime dynamic memory.

Some tool chains (and especially some features of C++, the topic of a separate article) require dynamic memory internally, even if your application uses no free storage calls. The newlib C-runtime library (used in many embedded toolchains) internally uses it's own malloc-family routines. newlib maintains some internal buffers and requires some support for thread-safety.

This article explains how to use newlib safely in a FreeRTOS project with GNU toolchain.
Warning: This article discusses newlib version 2.5-3.1. Later versions may be different...

newlib requirements

GNU ARM Embedded Toolchain distributions include a non-polluting reduced-size runtime library called newlib (or newlib-nano for the smallest variant). Unfortunately, newlib internally uses free storage (malloc/free) in startling places within the C runtime library.
Thus, newlib free storage routines get dragged in and used unexpectedly.
Ooops: Not MISRA-compliant, if you believe in that stuff.

The most common functions that bite unsuspecting embedded developers are sprintf using %f, dtoa, ftoa, rand, and strtok. Here's a list of newlib functions with reentrancy support (and hence using malloc).

newlib requires platform-specific support to:

  • provide thread-safety,
  • switch thread contexts, and
  • obtain memory to be doled out and managed by malloc/free/etc (sbrk).

Here's the detailed list of support functions required by newlib.
If this support is properly implemented, newlib works well in a threaded environment like FreeRTOS. This support is not properly implemented for FreeRTOS projects in Freescale KDS and NXP MCUXpresso until spring 2020, and in some current MCUXpresso examples.
See: BUG: malloc overruns heap, returns invalid pointer, and corrupts memory, from a 1-line application printf'ing a float

Example newlib use of malloc/malloc_r:
Placing a breakpoint within malloc_r (innermost malloc routine) shows the following:

  • newlib 3.0 does not use malloc before main() is called (unlike earlier NXP/newlib 2.5 startup, which malloc'd 5k). Verified for all four newlib variants.
  • With simple decimal output, sprintf does not call malloc
  • For %f output, sprintf does 4 malloc totalling ~200 bytes (only the first time it is called per task/thread).
    %f also requires proper linker arguments for float support.
  • printf or similar function expected to do IO (as opposed to string operation) allocates an IO control structure of 428 bytes.

Tip: How can I see who is calling malloc ?
The basic steps are:

  • create a wrapper function malloc_r,
  • add the wrapper options to the linker, and
  • place a breakpoint in malloc_r.

A complete example with instructions is included in the code below.

newlib under the hood - concurrency
newlib maintains information it needs to support each separate context (thread/task/ISR) in a reentrancy structure. This includes things like a thread-specific errno, thread-specific pointers to allocated buffers, etc. The active reentrancy structure is pointed at by global pointer _impure_ptr, which initially points to a statically allocated structure instance. If you use no threading or tasks, and you don't use malloc/free/etc or any of the reentrancy-dependent functions in multiple execution contexts, nothing more is required to use newlib in your application (newlib will maintain its required info in the single static structure). If your application or any library you use requires malloc/free/etc or any of the reentrancy-dependent functions in multiple contexts, newlib requires:

  • concurrency protection for malloc/free/etc. The free storage pool will be corrupted if multiple threads call into these functions concurrently! To prevent reentrant execution of malloc/free/etc routines newlib requires hook procedures __malloc_lock/unlock If an RTOS-based application does not replace newlib's complete internal malloc family (FreeRTOS does not), _malloc_lock/unlock must be provided for thread safety.
  • multiple reentrancy structures (one per context), and a mechanism to create, initialize, and cleanup these structures, plus switching _impure_ptr to point at the correct reentrancy structure each time the context changes.

WTF? Why are newlib's dtoa, ftoa, and sprintf calling malloc???
To quote Steele and White, Isn't it a pain when you ask a computer to divide 1.0 by 10.0 and it prints 0.0999999? To summarize and over-simplify a very long story, in 1990 my friend Will Clinger published a seminal paper giving a solution to this problem, but Will's solution required arbitrary precision and hence storage. See Will's retrospective on his seminal 1990 paper and subsequent developments. David Gay published an improved implementation (based on Will's paper), which was extremely widely adopted. newlib uses David Gay's code for dtoa and ftoa (using malloc), and sprintf uses dtoa and ftoa. Keith Packard's picolibc implements the no-malloc precise Ryū algorithm. There has been some recent discussion on the newlib mailing list including details of Gay's algorithm and maybe incorporating no-malloc Ryū algorithm from picolibc into newlib.
Really now, aren't you sorry you asked me why?

FreeRTOS support for newlib

FreeRTOS provides support for newlib's context management. In FreeRTOSconfig.h, add:
#define configUSE_NEWLIB_REENTRANT 1 // Required for thread-safety of newlib sprintf, strtok, etc...
With this option FreeRTOS does the following (in task.c):

  • For each task, allocate and initialize a newlib reentrancy structure in the task control block (TCB).
    This adds 96* bytes overhead per task (* size hugely dependant on newlib build options), plus anything else newlib allocates as needed.
  • Each task switch, set _impure_ptr to point to the newly active task's reentrancy structure.
  • On task destruction, clean up the reentrancy structure (help newlib free any associated memory).

Careful: Depending on how your supplier built newlib, and whether you are using nano, configUSE_NEWLIB_REENTRANT can chew up considerable memory! Check carefully if this is OK for your application!

FreeRTOS memory management
FreeRTOS internally uses its own memory management scheme and API (implemented by your choice of heapxx.c routines). Some FreeRTOS applications only use FreeRTOS-provided memory management. Unfortunately many libraries use the standard "C" malloc-family routines internally, including newlib itself. For these cases I've implemented the FreeRTOS memory management API on top of the "C" standard (using newlib) in the module heap_useNewlib.c below.

Using newlib safely with FreeRTOS - Possible Approaches

If you are certain your application does not use any newlib functions or any library that internally use the malloc-family and/or depend on thread-specific reentrant context, you could do nothing. But, are you really sure??? What about the libraries you use? For some hints, see this list of newlib functions with reentrancy support. The only really safe way to preclude accidental use is to provide HCF stubs for every one of these functions. Alternatively you can provide a linker -wrap command for each forbidden function but don't implement the wrappers, so link fails if forbidden routines are ever referenced.
You really need to be sure!

To avoid using newlib's printf and dragging in newlib reentrancy components, there are many cut-down printf implementations available (that do not use malloc). Won't help if you're using dtoa or strtok or others! Example light-weight printf implementations:

You must ensure you don't accidentally use any newlib facilities requiring reentrancy support and/or malloc-family in multiple tasks.

Another option is wrap newlib's malloc-family to use FreeRTOS free storage (ie heap_4.c), and specify newlib support for FreeRTOS. Tell the linker to wrap all newlib's malloc-family functions (using -Xlinker --wrap=malloc etc.), and provide a wrapper function that calls the FreeRTOS functions. I tried that, but newlib's printf family uses realloc, which is not supported in FreeRTOS heap implementations.

In the end (thanks to Richard Damon for encouraging this approach), I implemented the FreeRTOS memory API on top of newlib's malloc family, and provided all the hooks newlib's malloc family requires.

Using newlib safely with FreeRTOS - Recommended Solution Details

If your application needs a complete malloc family implementation, or you are using any newlib functions that require malloc (for example printf family or strtok), do the following (I've provided an implementation below):

  • Implement the hooks required by newlib (sbrk, __malloc_lock/unlock).
    Make sure your linker file matches the sbrk implementation!
  • Provide a heap implementation that implements the FreeRTOS memory API using the malloc family of newlib.

To use the implementation I've provided in your project:

  • Exclude from all builds any current FreeRTOS heap implementation, typically something like:
    Middlewares\Third_Party\FreeRTOS\Source\portable\MemMang\heap_4.c
  • Exclude from all builds any current sbrk implementation.
  • Either add the module heap_useNewlib_NXP.c I've provided, or the version now distributed by NXP.
  • Make sure your LD files provide the symbols required (defining the heap boundaries and size). The example LD file below show how, and also includes some helpful space-used computations, accessible with a post-build-step grep DRN on the map file.
  • If in multiple tasks your application needs a complete snprintf implementation, strtok, dtoa>, or other newlib functions requiring reentrancy support, or you're not really sure...
    Configure FreeRTOS for newlib support. In FreeRTOSconfig.h, add the line:
    #define configUSE_NEWLIB_REENTRANT 1
  • If are sure you're only going to access newlib's reentrant routines from a single FreeRTOS task and want to skip configUSE_NEWLIB_REENTRANT (with its attendent overhead)? Ensure malloc is never called except during startup and from the designated single task. Verify with a check in malloc_lock (sorry, I did not provide example code).

Note: Latest code and further documentation is on Github

Summary

Unfortunately some vendors distribute toolchains with incorrect examples of FreeRTOS/newlib.
However, with a bit of care newlib works well and safely with FreeRTOS.
Enjoy!
Best Regards, Dave

PS: newlib provides facilities for wrapping stdio functions, not covered in this article. You will want to use these, for example, if your application uses posix IO functions to read and write to a USB stick using a local FAT implementation.

PS: Hope Richard Barry and the FreeRTOS team will add heap_useNewlib.c to FreeRTOS ;-)

Additional References

http://www.billgatliff.com/newlib.html
http://wiki.osdev.org/Porting_Newlib
http://www.embecosm.com/appnotes/ean9/ean9-howto-newlib-1.0.html