|
Parralel Computing Turorial |
|
|
|
|
Written by Hemanshu Patel
|
|
Sunday, 23 December 2007 |
|
Page 7 of 18
Parallel Computer Memory ArchitecturesDistributed MemoryGeneral Characteristics:
* Like shared memory systems, distributed memory systems vary widely but share a common characteristic. Distributed memory systems require a communication network to connect inter-processor memory.
Distributed memory architecture
* Processors have their own local memory. Memory addresses in one processor do not map to another processor, so there is no concept of global address space across all processors.
* Because each processor has its own local memory, it operates independently. Changes it makes to its local memory have no effect on the memory of other processors. Hence, the concept of cache coherency does not apply.
* When a processor needs access to data in another processor, it is usually the task of the programmer to explicitly define how and when data is communicated. Synchronization between tasks is likewise the programmer's responsibility.
* The network "fabric" used for data transfer varies widely, though it can can be as simple as Ethernet.
Advantages:
* Memory is scalable with number of processors. Increase the number of processors and the size of memory increases proportionately.
* Each processor can rapidly access its own memory without interference and without the overhead incurred with trying to maintain cache coherency.
* Cost effectiveness: can use commodity, off-the-shelf processors and networking.
Disadvantages:
* The programmer is responsible for many of the details associated with data communication between processors.
* It may be difficult to map existing data structures, based on global memory, to this memory organization.
* Non-uniform memory access (NUMA) times
|
|
Last Updated ( Sunday, 23 December 2007 )
|
Technology 