Data Flow Architecture

What is a data flow software arch? What are the components?

• presents a system as a series of transformations on successive sets of data

• system is decomposed into functional process modules (sub systesm), where data directs and control the order of the data computation processing

• each module component in the arch. transform its input to its output data

What are some of the connection between modules components

• Input-output streams

• input-output files

• buffers

• piped streams

• or other type connections

Properties of Data Flow Architecture, Focus, interactions, property

• the focus in the data flow is the data availability

• data can flow in a graph topology witch cycles (linear, cyclic, tree)

• there is not interaction between modules except the data connection between them

• the modifiability and reusability are the property attributes of the data flow architecture

  • arch. elements are independent of each other

  • each element does not need to know the identity of any other

Pipes

stateless and serve as conduits for moving stream of data between multiple filters

pipes are special cases of filters; the modification function is the identity

• binairy or character stream

Filters

are stream modifiers, which process incoming data in some specialized way and send that modified data stream out over a pipe to another filter

• independent data stream transformer

• reads data from its input data stream

• transform and processes it

• writes over a pipe to next filter

does not need to wait for batched data

Batch Sequential

• each transformation element cannot start its process until its previous element completes computation

• data flow carries a batch of data as a whole to move from one element to another

• connection links between elements are conducted through temporary files

Applicable Design Domain of Batch Sequential

• data is batched

• each sub-system reads related input files and writes output files

Benefits of Batch Sequential

• simple division between sub-systems

• each subsystem can be a stand-alone program working on input data and producing output data

Limitations of Batch Sequential

• external control to (scheduler, control program)

• low throughput (waiting for batch, reduces efficiency)

• no interactive interface

Pipe and Filter Arch. Overview

• flow is directed by data

• composed of

  • data source

  • filters

  • pipes

  • data sink

• connection between components are data stream

  • data stream is a FIFO buffer type data structure

Serialization

process of saving an object onto a storage medium or to transmit it across a network connection link in binary form

Deflating or Marshaling

• the process of serializing an object

Deserialization or unmarshaling

extracting a data structure from a series of bytes

Push Only

• write only

• a data source may push data in a downstream

• a filter may push data in a downstream

Pull only

• read only

• a data sink may pull data from upstream

• filter may pull data

Push/Pull

• read/write

• filter may pull data from upstream and push transformed data in a downstream

Active Filter

• pulls in data and push out the transformed data (push/pull)

• it works with a passive pipe which provides read/write mechanisms for pulling and pushing

Passive Filter

• lets connected pipe push data, and pull data out

• it works with active pipes that pull data out from a filter and push data into next filter

UML for Pipe and Filter

Active pipe

must have active filters

Applicable design domain of pipe and filter arch.

• wherever the system can be broken into a series of processing steps over data stream

• data format on the data stream is simple and stable, and easy to be adapted if it is necessary

• there are significant work which can be pipelined to gain performance

• suitable for producer/consumer model (one part of system creates data and the other uses ti)

Benefits of Pipe and Filter

• Concurrency

• reusability: Encapsulation of filters makes it easy to plug and play and to substitute

• Modifiability: Low coupling between filters

• Simplicity: Clear division between piped filters

• Flexibility: it support sequential + parallel execution

Limitation of Pipe and Filter

• not suitable for dynamic interactions, data flows, no decision logic, its fixed

• Low Common Denominator is required for data transmission in the ASCII formats

  • the bits/s is dictated by the slowest transformer

• overhead of data transformation among filters such as parsing overhead into two consecutive filters

  • filters must deserialize, them serialize to send out

• error handling issue, don’t know what system is responsible for error, integration is hard, but unit is easy

Process-Control Architecture

• suitable for embedded system software design

• process control architecture decomposes the whole system into two type sub-systems

  • executor processor unit for changing process control variables

  • controller unit for calculating the amounts of changes

• Must have the following

  • controlled variable: should be measured by sensors

    • has a set point which is the goal to reach

  • input variable

  • Manipulated variable

    • adjusted by the controller

Process Control Architecture UML

Applicable Domain Process Control Architecture

• embedded software system involving continuous signal

• system that needs to maintain an output data at a stable level

• system has a target point

Benefits + limitaitons of Process Control Architecture

Better solution to the control system where no precise formula can be used to decide the manipulate variable

software can be completely embedded in the devices

Limits: can be unstable and requires thorough mathematical analysis