Parallel processing is a term used to denote a large class of techniques that are used to provide simultaneous data-processing tasks for the purpose of increasing the computational speed of a computer system.

The purpose of parallel processing is to speed up the computer processing capability and increase its throughput, that is, the amount of processing that can be accomplished during a given interval of time.

  • The amount of hardware increases with parallel processing, and with it, the cost of the system increases.
  • Parallel processing can be viewed from various levels of complexity.
    • At the lowest level, we distinguish between parallel and serial operations by the type of registers e.g. shift registers and registers with parallel load
    • At a higher level, it can be achieved by having a multiplicity of functional units that perform identical or different operations simultaneously.
  • Fig 4-5 shows one possible way of separating the execution unit into eight functional units operating in parallel.
    • A multifunctional organization is usually associated with a complex control unit to coordinate all the activities among the various components.

                   

                                                         Fig 4-5: Processor with multiple functional units

  • There are a variety of ways that parallel processing can be classified.
    • Internal organization of the processors
    • Interconnection structure between processors
    • The flow of information through the system
  • M.J. Flynn considers the organization of a computer system by the number of instructions and data items that are manipulated simultaneously.
    • Single instruction stream, single data stream (SISD)
    • Single instruction stream, multiple data stream (SIMD)
    • Multiple instruction stream, single data stream (MISD)
    • Multiple instruction stream, multiple data stream (MIMD)

SISD

  • Represents the organization of a single computer containing a control unit, a processor unit, and a memory unit.
  • Instructions are executed sequentially and the system may or may not have internal parallel processing capabilities.
  • Parallel processing may be achieved by means of multiple functional units or by pipeline processing.

 

SIMD

  • Represents an organization that includes many processing units under the supervision of a common control unit.
  • All processors receive the same instruction from the control unit but operate on different items of data.
  • The shared memory unit must contain multiple modules so that it can communicate with all the processors simultaneously.

MISD & MIMD

  • MISD structure is only of theoretical interest since no practical system has been constructed using this organization.
  • MIMD organization refers to a computer system capable of processing several programs at the same e.g. multiprocessor and multicomputer system
  • Flynn’s classification depends on the distinction between the performance of the control unit and the data-processing unit.
  • It emphasizes the behavioral characteristics of the computer system rather than its operational and structural interconnections.
  • One type of parallel processing that does not fit Flynn’s classification is pipelining.
  • We consider parallel processing under the following main topics:

Pipeline processing

  • Is an implementation technique where arithmetic suboperations or the phases of a computer instruction cycle overlap in execution.

Vector processing

  • Deals with computations involving large vectors and matrices.

Array processing

  • Perform computations on large arrays of data.