Poll events execute in arbitrary event queues
There is the concept of a poll queue in gem5. This lets you register events that get called when there is file descriptor activity. These events are called similar to normal events that are scheduled in an event queue, but are triggered asynchronously by a SIGIO.
The implementation currently has a set of flags that signal whether there is a pending asynchronous event. The sim loop then checks if the flag is set and triggers an action (this code looks racy, but I haven't had a close look). This is done in all event loops in the case of a parallel simulator (e.g., multi-core KVM). In most cases, the first event queue will be used to service the event since that queue will be activated by the event source. However, that's not guaranteed and the thread is in fact nondeterministic.
This seems to have triggered bugs in KVM-based simulations:
The same async mechanism in the event queue is used for a few other events (see src/sim/async.hh for a full list) which may have similar issues.
Intuitively, it seems like asynchronous events requested by a SimObject should be delivered in that SimObject's event queue. Alternatively, we could define event queue 0 to be the main event queue and always use that one for asynchronous events. However, using a single event queue may lead to surprising behaviour (devices would live in different threads) when simulating multiple parallel systems. The best solution might be a hybrid where poll events are delivered using the same event queue as the device that requested the event and other events could be delivered using event queue 0. This could be achieved by handling all async events in event queue 0 and using a ScopedMigration to temporarily execute poll events in the target queue.