We had requested to optimize various microprocessors in terms of making them do distribute mathematical tasks such as calculating prime numbers, matrix multiplication, etc. The project was strictly oriented towards optimizing algorithms that would run under these microprocessors.
Technology & hardware in the project
The main architecture this product used was SOA because of the fact that hardware itself had to interact with the internet layer due to some tasks requiring it (such as submitting results etc). Interesting part about this project is not the fact that it solves complex mathematical projects, but rather how he does it and how efficient it is.
We introduced an interesting distributed programming model in the game called job-stealing where the system knew exactly how to efficiently steal a job from other slave node and to distribute it across the network.
Taking in consideration that tasks were from dynamic nature, meaning we did not know how long will it take for it to run and finish it, we introduced another master node next to task scheduling one, which used machine learning concepts, who’s job was to predict the duration of certain tasks and approximate the result time.
All of this was being executed and displayed on hardware monitors and web dashboards.
SOA – Service Oriented Architecture – unlike classic server-client architecture or other rigid architectures this allows for loose coupling of parts of systems which enables easy replacement of parts. there’s no single point of failure.
Job/Work stealing – paradigm in work/load balancing where idle nodes attempt to steal part of load from other nodes which are overwhelmed. elegant solution that enables good balance of load.