TCP connection-oriented synchronous data write

Question

thinking about the design of an IoT project where the devices don't have a standard application layer but rather a thin custom application layer utilizing TCP sockets. What I mean by this is that the device wakes up every 15 minutes or so opens a socket on the server and sends data in, waits for any incoming data and then goes back to sleep/powers off.

On the software side Node.js is being used with a net.createServer to create a socket server and handle incoming data using a single-threaded event loop. When the socket is created the bytes are read and translated into numeric values which are then published to a MQTT topic for downstream processing.

What I'm mulling over is the other part of this data exchange in which bytes are written upon a connection. I.e. sending a command to the device. Since it's a TCP connection-oriented device, the software has to queue any asynchronous data to the device and hold it while waiting for the device to connect. Previously I would hold this data in an array (used C++ previously) and then write to the device when it made a connection. This proved to be a poor design since sofware crashes from time to time and loses any outgoing data to the device.

Designing something new, is it a good practice to handle this in a database? Meaning the asynchronous data is written to the database and then the software simply reads the database for the data at connection time. If there are thousands of devices reporting in that's a lot of reads to the database as well as writes to it for confirmation purposes. Was thinking instead to possibly use a subscribe via MQTT topic (retained messages) when device reports in to read outgoing data. However I'm not sure how well this can scale and the speed at which the software can subscribe to a topic which seems like it would be a bottleneck. Any ideas on which one to go with or alternatives to these two designs?

Answer 1

This really depends upon how many devices you're trying to support and how often a transaction occurs with a given device and how much data is involved for a given device. Based on that, reasonable solutions range from:

1. Using a file per device to persist the data waiting to be sent to the device
2. Using a clustered regular database holding the data for each device
3. Using high transaction, high scale queuing software.

It also matters whether you will need to cluster your nodejs server to handle the load or not.

If you don't know right now that you have to go really high scale or how much load that scale leads to, then I'd be inclined to build an abstract class for the store (a class that could have any of the types of storage mechanisms mentioned above behind it) and initially put a "file per device implementation" behind the abstract store API. That "file per device" implementation can probably be built pretty quickly. You would just need a persistent ID to identify each device that you could then use in a filename.

Then, see how well that works and how well it scales and performs. Then, you only go to one of the more expensive solutions (to acquire, code, deploy and maintain) when you know you really need it. This builds the opportunity to scale more in the future into your architecture, but doesn't over invest in it now.