Malloc and it's use with MCUs

[24Volts]

Hello everyone,

I have a project which needs to work in real time and can't
afford to hang or undergo any access violation issues during
its execution. I typically use malloc/free extensively!!! Then, I
stumbled on the first few paragraphs of the following article:

(see around the middle of the article under
"Memory Fragmentation")


http://www.design-reuse.com/articles/25090/dynamic-memory-allocation-fragmentation-c.html

Now I am worried! If using malloc in MCUs such as pic32 can cause
memory fragmentation, which may cause our programs to hang, then
why should we ever use malloc for embedded systems...
more over, why do compilers for MCUs even offer the possibility
of memory allocation if using malloc can represent such a risk?

Kowing this, really discourages me as it leads me to wonder why did I
bother learning how to malloc arrays, structures, arrays of structures
and arrays of pointers for??

My question is, should we or not use malloc in MCUs ?

Also, if one allocates say 50k in ram, would it be safe
to malloc 3 or four mallocs of 5k each? I don't see how
any problems can arise with this sort of leeway!

thanks

 

[Raven Luni]

The problem with these functions is that theyve never really been scaled properly to accomodate 'micro memory architectures'.

Fragmentation occurs when smaller blocks of memory are freed, and there is not enough contiguous space to allocate a larger block that would otherwise fit. Example: say you have 100 bytes of memory and you allocate 4 blocks of 25 bytes - lets call them A B C and D. If you free blocks B and D, you should have a total of 50 bytes of free space, but lets say you want to allocate another block E which is 50 bytes, this would not be possible because there is no area of free space that is 50 bytes long, only the fragmentated space which is 2 blocks of 25 bytes at different locations left by B and D.

The only real way to manage these problems on a MCU is to do it yourself in such a way that you wont run into these problems.

Your learning is not wasted by the way. malloc is just a function the points things at free space. All your pointers and structs etc work the same regardless of how theyre pointed.

 

[KrisBlueNZ]

In embedded systems it's not a good idea to use malloc() and free() willy-nilly, the way you might in a Windows app for example. This is because of fragmentation, and the resource-constrained nature of the environment.

It really depends on what kind of processing you're doing, but I would recommend doing your own simple memory management. For example, if you have several sections of code that need a buffer area of similar size but don't (can't) run simultaneously, malloc() a memory block when your program starts up, and pass it around to the different pieces of code that use it.

If different parts of your code use buffer areas at the same time, you need to allocate more blocks at startup. If it's possible that different parts of the code might all need memory areas that add up to more than the available memory on the device, then that's a problem whether you use malloc()/free() all the time or not!

If your architecture makes use of lots of small memory areas, I would allocate a block for them all, divide that block up into smaller blocks equal to the size(s) that your code uses, and handle allocation/deallocation of these smaller blocks explicitly.

It all starts with analysing how your program uses memory, then figuring out the best way to deal with it, on the assumption that any memory you use will be either statically allocated, or allocated via malloc() at startup and never free()d.

 

[24Volts]

Fragmentation occurs when smaller blocks of memory are freed, and there is not enough contiguous space to allocate a larger block that would otherwise fit. Example: say you have 100 bytes of memory and you allocate 4 blocks of 25 bytes - lets call them A B C and D. If you free blocks B and D, you should have a total of 50 bytes of free space, but lets say you want to allocate another block E which is 50 bytes, this would not be possible because there is no area of free space that is 50 bytes long, only the fragmentated space which is 2 blocks of 25 bytes at different locations left by B and D.

yes but lets say I set my heap size to a much larger amount then
the sum of blocks A,B,C,D and E. For example, suppose that I allocate a max
heap size of 10,000 bytes. Now, in reference to your example
I would free block B and D. If I then malloc another block of
50 bytes called E as you say, you are telling me that malloc
would not be able to find an empty spot for E when there is 9950 bytes
that are free????

I am ready to make my heap size much greater then the sum of all
my allocations to avoid any problems.... Why wouldn't this work?

I thank you guys for your replies!!!

 

[KrisBlueNZ]

It depends on how much free()ing you do, and the sizes of the blocks you are using. If you're malloc()ing and free()ing two or more different sizes of blocks, it's always possible to get into a situation where the whole heap is used up because of fragmentation.

If you're very careful with how you deal with malloc()ed objects, you can write your program in such a way that blocks can be moved around by a garbage collector task. This is messy to deal with, and obviously, can slow things down substantially.

 

[24Volts]

Yes, but KrisBlueNZ, what if we are careful not to malloc more than
4 blocks at a time? Take the previous example, leaving lots of leeway,
4 blocks of 25 bytes each with a heap size of 10K!!!

I don't see how this scenario can cause fragmentation???

even if the 4 blocks were 1K each but never surpassing 4 blocks
at any given time is still very possible because there will always
be enough room to free 1 to 3 blocks and then subsequently
malloc 1 to 3 blocks !!!

I don't see a problem with this... in any case I have tried stuff
like this and the MCU ran for days without any problems
but I never went more than 4 blocks at a time while
always respecting the proportion between heap size to block size
for example (heap size/block size=10K/1k).... which means
ideally the heap can support 10- 1K blocks where as I am only
using 4-1K blocks!!!!

Unless I am missing something.... I think malloc can be used this
way and be very safe. I thinks it's when we malloc huge blocks of memory
per heap size that creates problems.... for example:

Heap size: 10K
Block size: 3K to 5K
# of Blocks used at the same time: 3

... then I believe we would have issues!!!!

PS. I would use it as for each block can *vary* from 1 byte to 10K bytes.


thanks for replies!

 

[KrisBlueNZ]

As I said in post #5, the problem can occur if you're malloc()ing and free()ing blocks of two or more different sizes.

If you're always malloc()ing and free()ing blocks of the same size, then I don't see any way for the heap to get fragmented. Provided that malloc() and free() work the way you expect.

If I was doing that, I might still be inclined to malloc() a larger block of memory at program startup, split it into pieces of the size I need, and have my own malloc() and free() that just operates with those blocks. It depends on the specifics.

I would have a look at my library's malloc() and free() just to make sure they didn't do anything funny. And preferably have some way to report a malloc() failure.

 

[24Volts]

If you're always malloc()ing and free()ing blocks of the same size, then I don't see any way for the heap to get fragmented. Provided that malloc() and free() work the way you expect.

yes, exactly and furthermore if our malloc blocks can be the same
size (ex: 1K) so we can make sure they all fit in the allocated heap size,
and thus for predictability purposes, and then malloc/free different
size blocks as many times as we want but never surpassing the 1K limit
per block and never more than 4 blocks at a time. There are other
sizes that can be used in this scheme as long as they make sense
in its leeway size.

I would have a look at my library's malloc() and free() just to make sure they didn't do anything funny. And preferably have some way to report a malloc() failure.

I don't know if I have such functions in my library.... or maybe
Microchip compiler might have them ... I would have to check....

In any case there needs to be a certain degree of due-diligence
when sizing the max usable memory malloc.... and the max number
of block per instatnce and the block sizes.

Thanks KrisBlueNZ!

 

[KrisBlueNZ]

... then malloc/free different size blocks as many times as we want but never surpassing the 1K limit per block and never more than 4 blocks at a time. There are other sizes that can be used in this scheme as long as they make sense in its leeway size.

I think it would be cleaner if you only allocated blocks of a single size - the largest size you need - then just use part of the block if you don't need all of it.

You're welcome

 

[24Volts]

Understood!

thanks KrisBlueNZ!