- Development: Data Processes, aka Processing Logic (IoT Applications)
- Capabilities of Proc.Logic / IoT App in Snap4City
- Examples: Processing Logic / IoT App: typical patterns
- Typical strange patterns that may be not efficient in most cases, or wrong
- Example of Trigger / Sync Pattern
- Test and Verify Data Ingestion
- Test and Deploy of Proc.Logic / IoT App Node-RED Libraries
The development of Data Processes focusses on the functional aspects and includes the implementation of data flow procedures for data: ingestion, transformation, production, publication, load/retrieve, etc. These activities are the classical ETL/ELT of data warehouses and data lakes.
So that: Processing Logic (IoT App) high flexibility, rapid development any kind of applications, while dedicated Python processes for high performance data ingestion, for example when large amount of data communing from historical repositories has to be loaded in the platform for setup.
Processing Logic (IoT App) Development allows exploiting a large collection of macros/nodes (Node-RED Nodes) and most of them refer to MicroServices which are: the standard nodes provided into the Node-RED editor also called Processing Logic Editor (IoT App) in the context of Snap4City. Those provided by Snap4City (more than 180 MicroServices, which can be installed directly from the Node-RED library, repository of Palette); any other MicroServices/Node from other Libraries of Node-RED nodes and from the web since Node-RED is very diffuse.
Processing Logic (IoT App) of Snap4City can be used for creating data flow integrating in the same flow multiple activities of Data Ingestion, Data Transformation, Data Analytics and Server-Side Business Logic.
- Processing Logic = Node-RED + Snap4City Libraries
- Former: IoT App = Node-RED + Snap4City Libraries
- and can be executed on cloud as well as on IoT Edge: Linux, Windows, Arm, AXIS, Raspberry Pi, etc.
- Where these processes would be executed? It has to be executed on Cloud Snap4City platform or on some Edge computing of any kind?
- How I have to schedule this process if it not Event Driven? Periodic event?
- Which kind of Processing Logic (IoT App) I need to develop? Which kind of MicroServices I need to have on the Processing Logic node-red among the several possible?
- I have to process historical data or real time data, offline or in real time?
- We need to take them (PULL) or they will arrive (PUSH)?
- Which is the amount of data to be taken? They need to be decoded and how?
- Which are the gateways to be interfaces if any? Which kind of authentication?
- Which are the brokers to be interfaces if any? Which kind of authentication?