Concurrency

Each source - such as the program that obtains data from a sensor - runs in its own thread. The thread reads data from an external device and appends the data to a stream. Agent execution is managed by a compute engine which runs in its own thread. A process has one thread for each source and one thread for the compute engine, and additional threads to manage inter-process communication.

You start an application by starting the compute engine (compute_engine.start()). Source threads can be started at any time. Typically sources and the compute engine run forever. You can, however, also stop the compute engine and join source threads.

When an application is run across multiple processes, each process has its own compute engine. Starting (or stopping) the compute engine on the main process causes all compute engines to start (or stop).

You can specify a streaming application to execute in a single process, or multiple processes in a shared-memory machine, or on multiple processes in a message-passing distributed system. IoTPy manages the distribution of an assembly of modules across processes.