Slide: 1

Interra Software InfrastructurePart II : Components

Shyamal Sharma

7/29/05

Slide: 2

Agenda Part II : Components

• Interra Infrastructure Components

Memory Manager

• Benefits of Personal Memory Manager

• Architecture

Dump-Restore

• Leverage personal Memory Manager

Slide: 3

Benefits of personal Memory Manager

• Efficiency.

Using malloc/free directly for each and every allocation and de-allocation is very inefficient.

• ‘malloc’ has bookkeeping overhead (16 bytes) for each allocation.

• Allocations of small objects, say of 16 bytes, have therefore a 100% overhead.

• The personal memory manager will use system ‘malloc’ only for very large block (say 8Mb ) allocations.Therefore the ‘malloc’ bookkeeping overhead has minimum impact here.

• The personal memory manager will take over the task of allocating memory ,with minimum overhead, from this large block.

Slide: 4

Benefits of personal Memory Manager cont..

• Easy of debugging.

The personal manager can have its own custom made and efficient memory leak detection system which is tuned to the memory manager architecture.

• Persistence

Owning personal memory manager helps in implementing efficient object persistence (serialization)

No need for graph traversal through the Object model ,serializing every data member object and being concerned about duplication and cycles.

Instead we can index the whole address space in an address independent way and dump the independent index when serializing data member.

Slide: 5

Benefits of personal Memory Manager cont..

• Ability to Profile memory usage at no extra cost.

The stats on what size of memory is being used and how often helps fine-tune run-time efficiency of the program.

• Better locality of reference

By assigning memory to objects from a large block allocated by system ‘malloc’.

Since successive memory access are faster if they are to be in nearby memory locations.

Slide: 6

Interra’s Memory Manager

• Managed by a separate group - ‘Software Infrastructure Components’

• Provided as a library to all objects (Cheetah, Jaguar, NOM etc.). APIs provided in memoryPI.hxx:

void* IcAlloc(size_t size);

void IcFree(void *ptr, size_t size);

void useSystemAllocDealloc();

• All classes in the client objects to override their ‘new’. void* operator new(size_t size) {return IcAlloc(size);}

void operator delete(void *p, size_t size) {IcFree(p, size);}

Slide: 7

Use Model

• #define NEW_DELETE_OVERRIDE \ void* operator new(size_t size) {return IcAlloc(size);} \ void* operator new[](size_t size) {return IcAlloc(size);} \ void operator delete(void *p, size_t size) {IcFree(p, size);} \ void operator delete[](void *p, size_t size) {IcFree(p, size);}

• #include memoryPI.hxx

• class Foo { public : NEW_DELETE_OVERRIDE … }

Slide: 8

Memory Manager - Architecture

• 8Mb Blocks are allocated using malloc()• A Block is divided into smaller Chunks of size 64K.

• The Chunk address is aligned to 64K.ie lower 16 bits ‘0’.• Therefore the top 16bits of the Chunk address can be taken as a unique id for a Chunk. chunkId = chunkAddress >> 16;

• Each of these Chunks are added to a free-list ‘pFreeChunks’.

8Mb block divided into 127 64K-Chunks

64K64K

pFreeChunks

Slide: 9

Memory Manager - Architecture cont.

• We have pre-selected set of ‘popular’ sizes.• All memory allocation requests will be satisfied in terms of one of the ‘popular’ sizes.

• Selected Popular sizes are index’d 1 to 35 • {0, 4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,64 80,96,112,128, 160, 192, 224, 256, 320, 384,448, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536 }• Chunks are divided into equal-sized MemoryCells.

• Size of MemoryCell can only be one of the ‘popular’ sizes.• Since one Chunk can allocate MemoryCell of one ‘popular’ size only, different Chunks are designated to cater to request of different ‘popular’ sizes.

Slide: 10

#define K 1024 #define M (K * K)#define CHUNK_INDEX(chunk) (((unsigned int) chunk) >> 16)#define CHUNK_ADDRESS(index) ((ICChunk*) ((unsigned int)(index) << 16))#define NUM_POPULAR_SIZES 35// 4 GBytes of address space can be divided into 64K 64k-Chunks.#define MAX_CHUNK_COUNT (64 * K)// There are 4G/8M = 512 blocks that can be allocated in 32-bits.#define MAX_BLOCK_COUNT (4 * K)/(BLOCK_SIZE/M)

struct ICChunk { ICChunk* _next;};

struct ICMemoryCell{ ICMemoryCell* _next;};

class ICChunkContext{ public: ICMemoryCell* _pNextCell; ICMemoryCell* _pLastCell; size_t _nAllocatedCells; size_t _nReusableChunks; ICChunk* _pChunk;};

Important Macros and Data Types

Slide: 11

Memory Manager - Architecture (cont..)

• A ChunkContext structure manages allocation of one ‘popular’ size from Chunks.• Therefore there will be 35 different ChunkContexts to manage all the ‘popular’ sizes.• There will also be 35 different free-list of MemoryCells corresponding to each of the ‘popular’ sizes.

pChunkContext[1]

pFreeCells[1]

ChunkMemoryCells

For Popular size 4

pChunkContext[5]

pFreeCells[5]

MemoryCellsChunk

For Popular size 20

Slide: 12

class ICMemoryPool{ // Data members ICChunkContext _pChunkContexts[NUM_POPULAR_SIZES + 1]; ICMemoryCell* _pFreeCells[NUM_POPULAR_SIZES + 1]; ICMemoryCell* _pReusableCells[NUM_POPULAR_SIZES + 1]; char* _pBlocks[MAX_BLOCK_COUNT]; static unsigned short _nChunkAllocs[MAX_CHUNK_COUNT]; static unsigned short _nChunkPopularSizes[MAX_CHUNK_COUNT]; size_t _nCurrentByteCount; size_t _nMaxByteCount; int _nTotalReusableChunks; ICChunk* _pFreeChunks;};

Memory Manager - Memory Pool

• Managing the memory sub-system

Slide: 13

• If allocation is requested for a size which is not equal to any of the popular sizes then the next larger ‘popular’ size is assumed for it. For example. If the requested size is 500 bytes. Since we do not have Chunks allocating memory cells of size 500 bytes, we take the next larger ‘popular’ size. ie 512. So the request is satisfied by the Chunk allocating memory cells of size 512.512 - 500 = 12 bytes would be an acceptable wastage.

• When a Chunk managed by the ChunkContext runs out of MemoryCells, a new Chunk from the chunk free list, pFreeChunks, replaces it. • By masking the lower 16 bits of any MemoryCell we can get its owner Chunk. ownerChunk = (MemoryCell & 0xffff0000)

Pool Management

Slide: 14

Allocation algo

1.Get the ‘popular_size’ which will cover requested size.

2.Try returning MemoryCell from the free list for the popular_size. pFreeCells[PopularSizeIndex]

3.Try returning MemoryCell from the Chunk managed by pChunkContext[PopularSizeIndex]

4.Try getting a free 64K Chunk from pFreeChunks and assign it to pChunkContext[PopularSizeIndex]

go to step 3.

5. Try getting a new malloc’d 8M block and divide it into Chunks and put them on pFreeChunks.

Go to step 4.

6. Use system malloc directly for requested size > 64K.

Slide: 15

De-allocation Algo

1.Get the ChunkIndex of the MemoryCell.

2.Get the popular_size of the ChunkIndex.

3.If malloc() was directly use’d to alloc, use free() to de-alloc.

4.Insert MemoryCell into the memory cell free list corresponding to the popular_size.pFreeCells[PopularSizeIndex]

Slide: 16

Memory Manager - Other features• Memory Leak detection

For every allocation, the allocated memory address will be entered into a database.

• The stack frame of the allocation to be entered along with the memory address.

For every de-allocation, the memory address will have its entry removed from the database.

Any entry, if any, remaining in the database at the end of program run will indicate a memory leak.

• The stack frame associated with the entry will help in locating the cause of memory leak.

• Memory Usage profile without any overhead.

Slide: 17

Serialization of Interra objects• Dump-Restore

Avoids the standard graph traversal method.

Dumps out the whole of memory chunks owned by our personal memory manager.

While restoring we re-create similar chunk structure from the dumped file.

• But first, all the physical addresses ( references to other objects ) used in the chunks are converted to address independent offsets.

• Owning the memory manager enables the conversion of every physical address in our chunks to an independent-offset.

• After restoring the chunk structure, all the independent offset needs to be converted to physical addresses. These physical addresses are with respect to the newly allocated chunks.

Slide: 18

Dump/Restore - An exampleenum ObjectType { NET,INST,MOD }

Class VeBase { ObjectType _type;}Class VeNet : public VeBase { int _NetId;}class VeInstance : public VeBase{ int _InstId;}Class VeModule : public VeBase{ int _MId; VeNet* _pNet; VeInstance* _pInst; int _nCount;}VeModule* pTop;

0x45670000

0x77880000

_type

_pNet

_MId

_pInst

_type

_type

_InstId

_NetId

2

1

0

50022

34

0x77880000

0x77880002

_nCount 100

Slide: 19

Dump/Restore - An example (cont..)

0x45670000

0x77880000

_type

_pNet

_MId

_pInst

_type

_type

_InstId

_NetId

2

1

0

50022

34

0x00020000

0x00020002

• Index the used ChunksIDs

4567

..

..1

7788 2

0

• Convert the physical addresses to independent offset using the relativeChunkIDArray[chunkID]

relativeChunkIDArray

_nCount 100

Object Model DUMP process

Slide: 20

Dump/Restore - An example (cont..)

_type

_type

_InstId

_NetId

1

0

22

34

• Start dumping the memory pool to a buffer• write total number of chunks.• dump each chunk

• write the relative chunkID.• write the used size of the Chunk. • dump the used portion of the Chunk.

• write the buffer to a file.

2

1

16

20

String_buffer / file

_type

_pNet

_MId

_pInst

2

500

0x00020000

0x00020002

_nCount 100

2

Slide: 21

Dump/Restore - An example (cont..)

• Read the dumped file to a buffer• Allocate Required number of Chunks

• Read the total Chunk count given

by the 1st entry(2) in buffer.• Allocate as many Chunks. 0xABAB0000 and 0xCDCD0000

• Index the ChunkIDs chunkIDArray[] = { 0 , 0xABAB, 0xCDCD }

Object Model RESTORE process

0xABAB0000

0xCDCD0000

Slide: 22

Dump/Restore - An example (cont.)• Read the dumped portion of each Chunk from buffer into allocated Chunks.

• Get the Chunk to read into• Read relativeChunkID from

buffer • use chunkIDArray[]

• Copy ‘n’ bytes of buffer to the

Chunk. • ‘n’ is after relativeChunkID value in the buffer.• Copy the next ‘n’ bytes from buffer to Chunk denoted by chunkIDArray[relativeChunkID]

0xABAB0000

0xCDCD0000

_type

_pNet

_MId

_pInst

_type

_type

_InstId

_NetId

2

1

0

50022

34

0x00020000

0x00020002

_nCount 100

Slide: 23

Dump/Restore - An example (cont..)

• Convert the address-independent offsets in the Chunks to physical addresses.

• Replace relativeChunkID with actual chunkID using chunkIDArray[relativeChunkID]

• Let the top level object pointer point to the appropriate physical address.

• ie. chunkIDArray[1] 0xABAB0000

0xCDCD0000

_type

_pNet

_MId

_pInst

_type

_type

_InstId

_NetId

2

1

0

50022

34

0xCDCD0000

0xCDCD0002

_nCount 100

Slide: 24

Thank You