Mist-only Q-learning: the same agent with no tier to choose

What it shows

What is interesting here is what the learning loop looks like once you take the tier decision away. The MIST_ONLY architecture removes the cloud and the edge data centers from the action space, so the agent of examples/q_learning runs unchanged and its only remaining action is the mist. Open initialize in QLearningOrchestrator: it fills self.layers by keeping only the tiers the architecture allows, and with one layer left choose_action, node_in_tier and the reward bookkeeping still run but now learn which nearby device gives the lowest delay rather than which tier. The device that wins can be the one that made the request, since a mist peer includes the requester. Nothing in the script overrides the class, exactly as the Java Example 14 reuses the Example 11 class.

flowchart TD
    task["Task"] --> arch["Architecture MIST_ONLY"]
    arch --> layers["self.layers = mist only<br/>cloud, edge removed"]
    layers --> state["State<br/>load bucket, single tier"]
    state --> q["Q table, action 0 = mist"]
    q --> node["Least loaded eligible peer<br/>may be the requester"]
    node --> reward["Reward<br/>-delay, or -1000 if failed"]
    reward --> q
    classDef collapse fill:#cfe8ff,stroke:#333;
    class arch,layers collapse;

The scenario

settings/ keeps the Q-learning agent and swaps the fleet and the map:

The cloud and edge files are present but excluded by the architecture.

Run it

From the repository root, after pip install -e .:

python examples/mist_only_q_learning/mist_only_q_learning.py

No-install variant:

PYTHONPATH=src python examples/mist_only_q_learning/mist_only_q_learning.py

What to look for

mist only                tasks= 14500 ok= 11309 avg_delay=0.2539   cloud=    0 edge=    0 mist=14500 failed(delay)= 3191 failed(mobility)=   0 energy=48.5838

Widen or narrow edge_devices_range in settings/ to change how many peers the agent can learn between.

Files

Where to go next