|
Parralel Computing Turorial |
|
|
|
Page 1 of 18 This presentation covers the basics of parallel computing. Beginning with a brief overview and some concepts and terminology associated with parallel computing, the topics of parallel memory architectures and programming models are then explored. These topics are followed by a discussion on a number of issues related to designing parallel programs. The last portion of the presentation is spent examining how to parallelize several different types of serial programs.
Overview What is Parallel Computing?
* Traditionally, software has been written for serial computation:
o To be run on a single computer having a single Central Processing Unit (CPU);
o A problem is broken into a discrete series of instructions.
o Instructions are executed one after another.
o Only one instruction may execute at any moment in time.
Serial computing
* In the simplest sense, parallel computing is the simultaneous use of multiple compute resources to solve a computational problem.
o To be run using multiple CPUs
o A problem is broken into discrete parts that can be solved concurrently
o Each part is further broken down to a series of instructions
o Instructions from each part execute simultaneously on different CPUs
Parallel computing
* The compute resources can include:
o A single computer with multiple processors;
o An arbitrary number of computers connected by a network;
o A combination of both.
* The computational problem usually demonstrates characteristics such as the ability to be:
o Broken apart into discrete pieces of work that can be solved simultaneously;
o Execute multiple program instructions at any moment in time;
o Solved in less time with multiple compute resources than with a single compute resource.
* Parallel computing is an evolution of serial computing that attempts to emulate what has always been the state of affairs in the natural world: many complex, interrelated events happening at the same time, yet within a sequence. Some examples:
o Planetary and galactic orbits
o Weather and ocean patterns
o Tectonic plate drift
o Rush hour traffic in LA
o Automobile assembly line
o Daily operations within a business
o Building a shopping mall
o Ordering a hamburger at the drive through.
* Traditionally, parallel computing has been considered to be "the high end of computing" and has been motivated by numerical simulations of complex systems and "Grand Challenge Problems" such as:
o weather and climate
o chemical and nuclear reactions
o biological, human genome
o geological, seismic activity
o mechanical devices - from prosthetics to spacecraft
o electronic circuits
o manufacturing processes
* Today, commercial applications are providing an equal or greater driving force in the development of faster computers. These applications require the processing of large amounts of data in sophisticated ways. Example applications include:
o parallel databases, data mining
o oil exploration
o web search engines, web based business services
o computer-aided diagnosis in medicine
o management of national and multi-national corporations
o advanced graphics and virtual reality, particularly in the entertainment industry
o networked video and multi-media technologies
o collaborative work environments
* Ultimately, parallel computing is an attempt to maximize the infinite but seemingly scarce commodity called time.
|
Technology 