Making FBP big

by moodyharsh 2016-06-05
fbp-big

FBP is certainly the most complete realisation of Flow based approaches.
If my understanding is correct then FBP is the truing-complete version of Data Flow.
FBP can manage its Data Flows and that is why it has an explicit graph.

The number of Programmers ~ 20 million
The number of Programming Languages ~ 20

There are 10,000 people for every engineer in this world.

Number of Data Formats ~ 2000
Number of Algorithms ~ 10,000
Number of Protocols ~ 100
Number of APIs ~ 20,000
Number of Libraries / Language ~ 50

Number of lines of Code for an average application ~ 100,000
Legacy Code ~ 2 trillion

Number of Software Monopolies ~ 10
Number of Software Companies in the world ~ 50,000
Domains of Software ~ Embedded, Communication, Workflows ( Personal / Enterprise / Scientific ), Entertainment

Dominant Programming Paradigms ~ Imperative, Object Oriented followed by Functional.

Problems with Software

Real Estate Mindset with the Architects, Investors, Brokers, Brochure Makers and Masons.
Hollywood Mindset for Startups.

Lack of Engineering Discipline, Engineering Training and Original Methods. Borrowed methods from Civil Engineering / Mechanical Engineering views dominate.

Existing Methods are Good Enough for Profits.

FBP solves these problems.

Questions

  1. How do we deal with legacy code ?
  2. How do we train new programmers and existing programmers ?
  3. Do we have a framework for each programming language ?
  4. How do we complement existing paradigms ?
  5. What is the theory of FBP ?
  6. What do we do about the 50 Workflow Systems that look like FBP ? How do we stop people from re-inventing FBP ?

Observations

On a Personal Level, I am a practical idealist at heart. I am sick and tired for Software Development and Management.
I would like to see through that Software Development be treated like Engineering rather than a profitable Italian Dish.

Seeing how scripting languages have become successful,

  1. Books / Documentation
  2. Applications, Frameworks, Plugins
  3. Easy to start using and deploying

For Example - PHP and Wordpress.

I don’t think FBP is easy to install or use.

Does FBP need formal theory ? FBP is very close to KPN.

If a tree falls in a forest and makes a sound, no one cares.
I think Internet-Of-Things and Enterprise provide a nice space for exploring useful Applications.

Bluntly put there are no books good books for FBP, the documentation is poor.
I would like to see the following Book Titles for FBP.

  1. Dealing with Legacy Code
  2. Cookbook for FBP
  3. How to develop Games using FBP
  4. Enterprise Integration using FBP
  5. Control your Robot with xxxx
  6. Scaling FBP.
  7. FBP for Functional Programmers
  8. Develop custom flows for yyyy
  9. Stop writing your Workflow Automation solution, use FBP instead !
  10. Visual Patterns in Component Architectures
  11. FBP for the [Beginning, Intermediate, Expert]

As you can see a lot of work needs to be done !

Dealing with Legacy Code

The the rule of thumb is, wherever you have streams of Data you can stick FBP there.

  1. Batch processing

FBP is a great fit for batch processing tasks.
If your application is going to deal with message queues you can use FBP
to process stuff at the other end of the queue.

As FBP comes with a network protocol? this should be possible.

  1. Query Builders

FBP can help building a visual way to build queries.
This can help with in places like analytics where the possible queries can explode !

An example can be seen here – https://groups.google.com/forum/#!topic/flow-based-programming/-i6DGYudEbQ

  1. Visual Applications

Things like Music, Image processing, Electronics, Fluid analysis, Physics …
can certainly benefit from an FBP engine.

One important limitation I would like to point out here.
What could be the processing latency of FBP nodes ?
If this is not a hard constant, it can be quite a problem !

I can certainly attest about music DSP where even 1/48_000 secs delay can change the
perception of the audio rendered. It makes a “pop” sound, kinda like a bad pixel.

  1. Message Delegation

This is strictly not Data Flow but you still can get a pipeline like model.
It fits neatly with smalltalk model but you need design patterns for others.

  1. Interceptor Filter / Pipes and Filters Pattern

Now this I have experience with. In fact probably everyone has experience of this !
Java servlets / Rack / WSGI all use interceptor in various forms.

The way this pattern is used is something like chain of responsibility,
HTTP Request -> Form Parser -> Cookie Parser -> Session Manager -> Authentication -> Controller

If easy to miss this pattern because every framework talks about MVC and not this.
It is fairly easy to implement but comes to cost of slowing down method calls.

http://en.wikipedia.org/wiki/Interceptor_pattern
http://msdn.microsoft.com/en-us/library/dn178466(v=pandp.30).aspx

  1. Spreadsheets and Reactive Programming

Still the best you can get with these are linear or broadcast models.
I prefer the Interceptor to this, which is far simpler.

See
https://github.com/tailrecursion/javelin
https://github.com/Reactive-Extensions/RxJS