使用mallopt调整malloc/new行为

同事在项目中使用new/delete的时候发现一个奇怪的现象:
int32_t i;
std::queue<char *> qTest;
for (i = 0; i < 100000; i ++) {
    char *p = new char[100];
    qTest.push(p);
    char *p1 = qTest.front();
    delete[] p1;
    qTest.pop();
}

当在一个循环内,如果申请空间,在循环内并释放掉,内存不会引起增长,即使重复上面的单元也不会增长内存,但是当:
int32_t i;
std::queue<char *> qTest;
for (i = 0; i < 100000; i ++) {
    char *p = new char[100];
    qTest.push(p);
    //char *p1 = qTest.front();
    //delete[] p1;
    //qTest.pop();
}
while( !qTest.empty() ) {
    char *p1 = qTest.front();
    delete[] p1;
    qTest.pop();
}
而发现通过delete[]后,内存并没有减少,即使重复上面的单元,内存只会增加,不会减少,用valgrind的memcheck工具查看,也无内存泄漏。
其实,这个与malloc的实现有关,一般来说,系统都会有默认的malloc行为,一般来说(针对FreeBSD和Linux系列),对于大于1M的数 据,malloc行为会直接调用系统的接口,直接向操作系统申请一块比数据块更大内存,然后划分成若干过chunk单元(这里会有一些算法设计),而这些 chunk分配给数据后,肯定还会剩下很多的trunk单元,留给以后的分配用,但这里耗费系统资源,代价较大;而对于小于等于1M的数据,则 malloc行为会利用那些trunk单元,占用系统资源少,速度快。同理free的时候,数据占用的内存被释放,如果大于1M,则系统会回收掉内存,节 省资源,而小于等于1M的数据,则内存释放时,不会还给操作系统,以空trunk形式存在,并由当前进程空间维护着。说到底,这种策略就是一个内存池的概 念。
内存池灵活度更加方便,一般来说(针对FreeBSD和Linux系列),每个trunk最小容纳的字节数是16bytes,而自行设计的内存池,可以更 加优化,节省大量产生的内存碎片,像上面的代码就是因为内存虽然释放,但是这些内存都没有还给操作系统,导致内存碎片越来越多,最好的设计方式就是使用一 个内存池,针对经常使用的分配大小,可以多分配这样的trunk,而其他大小的trunk可以少产生,从而达到优化目的。
另外,也可以使用mallopt来直接调整malloc的行为:
int mallopt (int PARAM, int VALUE)
     When calling `mallopt', the PARAM argument specifies the parameter
     to be set, and VALUE the new value to be set.  Possible choices
     for PARAM, as defined in `malloc.h', are:

    `M_TRIM_THRESHOLD'
          This is the minimum size (in bytes) of the top-most,
          releasable chunk that will cause `sbrk' to be called with a
          negative argument in order to return memory to the system.

    `M_TOP_PAD'
          This parameter determines the amount of extra memory to
          obtain from the system when a call to `sbrk' is required.  It
          also specifies the number of bytes to retain when shrinking
          the heap by calling `sbrk' with a negative argument.  This
          provides the necessary hysteresis in heap size such that
          excessive amounts of system calls can be avoided.

    `M_MMAP_THRESHOLD'
          All chunks larger than this value are allocated outside the
          normal heap, using the `mmap' system call.  This way it is
          guaranteed that the memory for these chunks can be returned
          to the system on `free'.  Note that requests smaller than
          this threshold might still be allocated via `mmap'.

    `M_MMAP_MAX'
          The maximum number of chunks to allocate with `mmap'.
          Setting this to zero disables all use of `mmap'.
值得注意的是mallopt是malloc底层的函数,需要使用info mallopt来查看相关帮助信息。

Monthly Archives

Pages

Powered by Movable Type 7.7.2

About this Entry

This page contains a single entry by Cnangel published on May 14, 2010 10:26 AM.

五一琐事 was the previous entry in this blog.

琐事点滴 is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.