1

I have a ESP8266 Nodemcu device running a local HTTP server. I followed the quick-start instructions here.

My goal is to have a large number of these devices running in sync. To do that, I wrote this script:

#!/usr/bin/env python
import time
import sys
import socket
import requests

def myFunction():
    #This is what I have right now...
    ipAddresses = ["192.168.1.43", "192.168.1.44"]

    #
    #Instead, I need to search the local network for all the arduinos [ESP8266s] and append their address to the array.
    #

    for address in ipAddresses:
        TurnOnLED(address)

def TurnOnLED(address):
    try:
        r = requests.post('http://' + address + '/LedON')
        r.close()
    except requests.exceptions.ConnectionError:
        pass

#Main
def Main():
    try:
        print("Press CTRL-C to stop.")
        while ON:
            myFunction()
            time.sleep(60)
    except KeyboardInterrupt:
        sys.exit(0)

if __name__ == "__main__":
    Main()

This works, and allows me to control all of my devices from my desktop PC. My difficulty is with finding each IP address dynamically. I considered assigning a static IP address, but I would like them to be retail-optimized, so I cannot guarantee any particular IP address being unused out-of-the-box.

For the same reason, I want to be able to install the same arduino code on all of them. This is what I mean by naive. I want my python script to find the IP address of each device over the local network without any [unique] 'help' from the devices themselves. I want to install the same code on each device, and have my script find them all without any additional set up.

My first attempt was using the socket python module and looking for every hostname that began with "ESP-" since those are the first four characters of the factory hostname on all the boards. My query consistently turned up nothing, though.

This answer here has some good information, but my solution runs on a Macintosh, and I will not have the full host name, only "ESP-".

JeremiahDuane
  • 374
  • 4
  • 15
  • I don't get where is the arduino in all this? Or are you, improperly, calling "arduino" the ESP8266? (Not asking for pun, but just to try to understand. But, fyi, arduino and ESP8266 are two very distinct things. Even tho, arduino IDE has some capacity to interact with ESP8266). – chrslg Nov 11 '22 at 17:04
  • As for the "finding all IP" problem, there are several methods. But the cleanest one has to be the one where your devices register themselves somehow. Is that possible in your settings? I mean, that your PC also acts as a server, and the devices as clients. Not for the whole process (I suppose you've some good reason to prefer that devices has servers, and PC client), but just to register their existence. Like, every minute, each device says "hi, I am alive at IP..." to an UDP port of a "supervisor" machine (that may be your PC, or some other machine, like another ESP). – chrslg Nov 11 '22 at 17:09
  • It it is not possible to have one "supervisor" whose static IP is known by everyone, then you could scan the local network, like nmap does. But it needs to be done with superuser privileges. Is it possible? – chrslg Nov 11 '22 at 17:10
  • Or, you could just try to connect to all servers scanning all the possible IP. But that has to be done only once from times to times, not at every iterations. – chrslg Nov 11 '22 at 17:13
  • My bad, yes, I was referring to the ESP8266 Node MCUs. I understand the differences, it was just an oversight. This is the exact device: https://www.amazon.com/dp/B081CSJV2V?psc=1&ref=ppx_yo2ov_dt_b_product_details – JeremiahDuane Nov 11 '22 at 17:26
  • My only real reasoning for having the PC be the client is the tutorial I listed at the beginning of my post. I am somewhat of a noob, so if you have a better strategy I am all ears. That registration bit sounds like it would work! Do you have any suggested documentation for learning how to send a message to a UDP port? – JeremiahDuane Nov 11 '22 at 17:29
  • That would probably work too, super-user privileges are a given. My only concern is I would like for the solution to be to be run by anyone, with or without technical know-how. If that process can be done autonomously in a python script, that would work... Scanning all possible IPs is a little hackish, IMO, but if that can also be done autonomously, that would also work. When the script first starts I could pipe the IP addresses to a file and use the same ones from then on out. I'll probably do that regardless. – JeremiahDuane Nov 11 '22 at 17:31
  • Indeed. And even amazon's description keep the confusion alive, by mentionning "arduino". "Arduino" once upon a time was a specific product. Then became a common word used to describe clones of that very product. Then became a new alias of 5V TTL (aliexpress and amazon selling zillions of just 5V TTL sensors labeld "for arduino", when they have nothing specific to arduino). And now, even ESP8266, that are not even 5V TTL and would burn if connected to an arduino, are named "arduino".... well, let's try to not be too grumpy :D – chrslg Nov 11 '22 at 17:32

1 Answers1

1

So, now that I know what are the constraints, and what is just what is in the tutorial, here are my 2 cents (keep in mind that I too, am just a hobbyist about everything that has "voltage". And not even a good one).

1st strategy : PC is server

So, if I assume that your devices are, for example, temperature sensors, and you want to frequently grab all the temperatures, then, one strategy could be that those devices all connect, every minute, to the server, report the temperature, and disconnect. For example, using a HTTPS request to do so.

So host the server on your PC, and have your devices behave as clients

# Example request from micropython
import urequests
rep = urequests.get(URL)
print(rep.text)
response.close()

2nd strategy : a third party is server

A variant of that strategy is to have a sort of central data repository, that acts as a server. So, I mean, your PC is not the server, nor any ESP, but another machine (which could be one ESP, a rasppi, or even a droplet or EC2 instance in the cloud, doesn't matter. As long as you can host a backend on it). So, ESP8266 are clients of that server. And so is your PC. So that server has to be able to answer to request "set value from client", and "get value for the PC".

The drawback of those first 2 strategies, is that they are ok when the devices are the one sending data. It fits less if they are (as in your led example, but I surmise that was just an example) the ones receiving mostly the data. Because then, you would have to wait for the ESP to connect to have it get the message "you need to switch on the led".

3rd strategy : central registry

Or you can have it both way. That is keep your current architecture. When your PC wants something from the ESP8266, it has to connect to them, that are servers, and send it request. But, in parallel to that, ESP8266 also behave as clients to register themselves in a central directory, which could be on the PC, or on a 3rd party. The objective of that central directory is not to gather the data from the ESP8266. Just to gather a uptodate list of them. Each minute they, in parallel with their server activity, the ESP8266 send a message "I am alive" to this central dircetory. And then the PC (that could be, or not, hosting that central directory) just need to get all IP associated with a not too old "I am alive" message, to get a list of IP.

4th strategy : ARP

Once your PC is on, it could scan the network, using an ARP request, with scapy. Search for "scanning local network with scapy". For example here is a tutorial

From there, you get a list of IP. Associated with MAC address. And now, you need to know which ones are the ESP8266. You can here also apply several ideas. For example, you may use the MAC address to find guess which one are the ESP8266. Or you may simply try a dummy request on all found IP, to check which one are the ESP8266 (using a specific API of the server code of the ESP8266 you wrote)

Or you may decide to host the server on each ESP8266 on a specific port, like 8123, so that you can quickly rule out devices whose port 8123 is not listening.

5th strategy : don't reinvent the wheel

The best strategy is clearly a mix between my second and my third. Having a directory, and a 3rd party handling messages. But that is reinventing message brokers.

There is one well known middleware, fitted for ESP8266 (I mean, for IoT low profile devices), that is MQTT.

That needs some more tutorial reading and long trial and error on your behalf. You can start here for example. But that is just the 1st example I found on Google searching "MQTT ESP8266 micropython". There are zillions resources on that.

It may seem to be not the easiest way (compared to just copy and paste some code that list all the alive IP on a network). But in the long run, I you intend to have many ESP8266, so many of them that you can't afford to assign them static IP and simply list their IP, you probably really need a message broker like that, a preferably, not one that your reinvent

chrslg
  • 9,023
  • 5
  • 17
  • 31
  • Thank you so much! This is a more than enough to get me started... I think #4 might be my best option, but not because it is probably the easiest. One constraint that I didn't make clear is that I want to be able to run this script from ANY computer in the local network. That rules out #1 and #3, because I would still need to ascertain the list of IPs I need... – JeremiahDuane Nov 11 '22 at 18:27
  • ... #2 would work, but by "dozens" I was really considering having multiple "groups" in place. One group at the office, one group at the house, one group at my parent's. I'll probably only have 1-2 devices at each location, so I really don't want to set up a central server device just for 1-2... So not #3, and not #5 because it relies on #2/#3. – JeremiahDuane Nov 11 '22 at 18:27
  • That leaves #4, and I am honestly happy with that approach. I'll write a 'install' script that grabs all the IPs I need and pipes them to a file. – JeremiahDuane Nov 11 '22 at 18:31
  • @JeremiahDuane Oh, but then, be aware that strategy #4 works only on a local network. So at your office you'll see only the devices of your office, and at home, only those at home. You can ARP-scan the whole world :D. – chrslg Nov 11 '22 at 18:31
  • Haha, yes I am well aware. These are just synchronized lights, so I have no desire to turn them off/on everywhere, just wherever I am at :D – JeremiahDuane Nov 11 '22 at 18:32
  • Edit. Crossed messages. So you don't need to switch on leds at your parents from your office :D @JeremiahDuane whereas, a MQTT broker somewhere (anywhere you have the ability to host a server. So, a droplet, an EC2 instance, your NAS with embedded server, a Raspi whose server is DNATed, ...) would make this available everywhere: you could switch on a LED at your parent's from your office :D – chrslg Nov 11 '22 at 18:33
  • I wrote a service that reads my calendar and sends a command of red/green depending on if I am in a meeting or not, so that I am not disturbed. If I am not working from home at my parents, then I simply wouldn't need those lights to be on at all. I also wouldn't want my office lights to change if I was taking meetings at home in my bedroom :). If that were desirable, I would go with the EC2 instance. – JeremiahDuane Nov 11 '22 at 18:41