On Data Flow and Error Handling

by moodyharsh 2014-10-06

On Errors and Data Flow

The equivalent of an infinite loop for Flow Based approaches seems to be an infinite cycle within the Data Flow.
This is also a problem with the Nodes being called repeatedly, even if it exits after processing, which would be equivalent to Stack Overflow.
Then there are Exceptions and Deadlocks.

I can see the following strategies,

  1. Do nothing

Here both the user and developer will see Errors at runtime.

  1. Detect cycles at the graph level.

Here the developer can see it at test time and the user at runtime.

  1. Detect Exceptions at graph level.

Here the developer can manage them.

In control systems this is managed by feedback loops, I think.
The thing with Industrial Systems is that people can literally die, if the system is faulty.
So errors need to be handled quite rigorously.

Of course I can’t really speak of Industrial Systems because I have no experience.

I can however talk about DSP and electronic music.

  • In DSP there is this notion of sample rate, typically 48_000 samples / sec.
  • Y’all have probably heard the loud “pop” sounds. That can happen when even 1 sample goes bad.
  • With a complex DSP you also get processing latency, which means the Drummer and Guitarist can go out of sync.
  • Bad wires can amplify signals.
  • Power source can be faulty

You will find musicians who loathe software because they are terrible at latency estimations, among others.
However that doesn’t mean you ask for a perfect DSP, but you manage its limitations, even if it’s software.
This is possible because there is some form of feedback, the ears of the musician in this case.

An idea to use temporary connections for exceptions / feedback.

Consider A -> B -> C -> D -> E[1]

At runtime D <-> B and E <-> B can form temporary duplex connections.

Basically E or D can notify B that there is something wrong with the data pathway.

  • B can give D or E some new data ( duplex connection ) for resuming
  • Stop all flow
  • Ignore and do something else

What if a component goes berserk, who raises the exception then ?
How can exceptions and the faulty data pathways be represented visually ?

For A <               /G    
            \D -> C <
                        \ E -> F

I’d like to see D -> C -> E ( data pathway ) highlighted if an error in E can affect C and D.
You can see D -> C -> E as a traffic jam of sorts.

Should FBP depend on host-provided exceptions ?