The following terminology is used when discussing hierarchical
- The memory closest to the processor is known as a
cache. Some systems have separate caches for
data, in which case it has a split cache. An instruction
buffer is a special cache for instructions that also performs
other functions that make fetching instructions more efficient.
- The main memory is known as the primary memory.
- The low end of the hierarchy is the secondary memory.
It is often implemented by a disk, which may or may not be
dedicated to this purpose.
- The unit of information transferred between items in the
hierarchy is a block. Blocks transferred to and from cache
are also known as cache lines, and units transferred between
primary and secondary memory are also known as pages.
- Eventually the top of the hierarchy will fill up with
blocks transferred from the lower levels. A replacement
strategy determines which block currently in a higher level will
be removed to make room for the new block. Common replacement
strategies are random replacement (throw out any current block at
random), first-in-first-out (FIFO; replace the block that has
been in memory the longest), and least recently used (LRU;
replace the block that was last referenced the furthest in the
- A request that is satisfied is known as a hit, and a
request that must be passed to a lower level of the hierarchy is
a miss. The percentage of requests that result in hits
determines the hit rate. The hit rate depends on the size
and organization of the memory and to some extent on the
replacement policy. It is not uncommon to have a hit rate near
99% for caches on workstations and mainframes.