Parallelism - or something else?

#1

1) Give a description of the problem
Parallelism is the topic/problem, another thing I have been trying to wrap my head around, wish there was a single article detailing how this works. So I am used to Zipato, more than ST, and in Zipato there is a single command called “Join”. Essentially, it means that any time a “Rule” (equivalent of piston) starts running, if it finds other instances of that rule already running (and therefore older ones), it “joins” them to the newer rule. Essentially, what it means is any instances of that rule that it finds already running are killed off.

I can think of various uses for this. The most obvious is turning off the light after a No Motion event is registered. After a No Motionevent, a person may have walked into the room again (essentially some automation/piston fired but did not need to turn the light on as it was already on). That person is still in the room, yet if the initial piston is still running, it will switch off the light after say 3 minutes despite newer Motion events having happened since then. And there are numerous other examples that come to mind - I just wrote a piston to manage my 1st floor heating and I am certain that if 2 instances of the same piston run together, they might create a bit of a mess.

2) What is the expected behaviour?
Now one may either not allow parallel running of the same piston and have them lined up to run one after the other, but I believe the most desirable approach is to kill off older instances already running, overriding them with the latest one…

3) What is happening/not happening?
At the moment - nothing is happening, but I realised there was a conflict when multiple instances of the same piston are fired off at exactly the same time… It would be ideal for me to kill off the older instance immediately in the cases I can think of so far) and tun only the last one.

**4) Post a Green Snapshot of the piston![image|45x37]
Nothing to paste- just a general question

5) Attach logs after turning logging level to Full
Nothing to paste.

#2

I think the behaviour you describe is the default for webcore. Read up on task cancellation policy, there are some descriptions way better than I could write.

Basically though, if a piston is triggered again, any “instance” still running will be cancelled. The TCP can change this if required.

#3

Thanks for the reply - I will surely read that, appreciate it!

#4

The deal with webCoRE is that a new piston instance is started whenever a subscribed event arrives.

By default, webCoRE tries to serialize execution. If the new piston sees there is already an instance running it will wait at a ‘semaphore’ for up to ten seconds for the existing one to finish. More often than not it seems to be for the full ten seconds. It will then continue. As pistons can run for up to twenty seconds the previous one might still be running but that is a long time for a piston to run and I’ve never seen it.

You can ‘enable parallelism’ so this wait doesn’t happen. That can be useful in certain circumstances.

When you do a ‘wait’ in a piston it checks to see how much execution time it will have after the wait. If the wait is less than five seconds or there will be more than ten seconds of execution time left, the piston will go into a tight loop for the wait period. Otherwise it will schedule a timer for the end of the wait and exit. When this timer fires a new instance of the piston runs, fast forwards past the wait, and continues with the next task. This is called a ‘scheduled task’.

When pistons evaluate conditions they compare the result with what it was the last time they executed them. If the result has changed they assume, by default, that any scheduled tasks that were set up previously because of the previous result are no longer required and cancel then. The timers still run and the pistons fire, but they don’t do anything. This is the Task Cancellation Policy (TCP). That’s what handles the situation in the first post. Note that it works because the piston has set up a scheduled task. Effectively it is the wait that gets cancelled.

Sometimes you want the timed event to happen regardless so you can tell the piston that certain tasks should never be cancelled.

The TCP can also work with changes of piston state. I’ve never seen that done.

#5

Thanks @orangebucket, appreciate that.

I read your reply yesterday and have been looking at logs on one of my pistons (one trying to get a refresh from a 4-gang switch - we talked about this :slight_smile: and I put a refresh in the DTH but it seemed unable to handle multiple refreshes from the same switch). It seems that the DTH was missing some refreshes, or incapable of handling so many refreshes fractions of a second apart, so pressing all buttons on a 4-gang switch in rapid succession still showed only 2 or 3 buttons switched ON on the app. So my intention was to build a piston to get a final refresh at the very end, after all switch buttons are pressed…

So I thought that because I put in a wait of 2 seconds before the refresh, it would be enough - if a user presses all four buttons on a switch, presumably the piston will fire once on the first button press, wait for 2 seconds - then proceed to get an extra refresh (over and above that of the DTH)…

BUT - assuming the user continues pressing the other 3 buttons on a switch (and assuming the user will take less than 2 seconds in between each button press), then I assumed that the pistons fired the 1st, 2nd, and 3rd time round would all be cancelled, and only the last one would run, i.e. the one fired by the 4th button press.

I see from your explanation that I had understood the TCP all wrong. And I further confirmed that from my logs - I can see that when I press all four buttons on a switch:

  1. Button 1 pressed: The piston fires and runs in full, despite the others firing whilst the piston is still running.

  2. Button 2 pressed: The piston sees the first one running, then runs with “Piston waited at a semaphore for 9548ms”

  3. Button 3 pressed: Same as previous, “Piston waited at a semaphore for 10102ms”

  4. Button 4 pressed: Same as previous, “Piston waited at a semaphore for 9512ms”

That means all of them ran, in succession… A bit too many refreshes for my liking. I will have to look into ways of making the first piston abort upon an attempt to run the second instance of the same piston.

Also - I thought 10 seconds for the semaphore may be a very long time, in some situations (mostly lighting, not - for example - heating). Is there a way of reducing that 10 seconds for specific pistons?

EDIT

The piston is working fine, by the way… I mean, at least, about 10 seconds after the button presses, I get a final refresh for that switch reflected in the app. I am just looking to improve on that now.

#6

Hi @Paul1964

It doesn’t seem like it works in this manner. I have tested by firing the same piston (with a 2 second wait inside it) 4 times in a row, in rapid succession (by pressing 4 buttons on a switch within say a couple of seconds maximum, on the app)… It seems that the piston ran four times from my logs, though the 2nd, 3rd, and 4th times they waited at a semaphore for almost 10 seconds… My logs show this quite clearly, even though I would have preferred that it does what you say, ie. cancelled the first 3 and only ran the last instance of this piston…

So I guess I’ll have to find an alternative solution.

#7

@orangebucket is the guru on the inner workings of webcore. I read the detailed description posted. I just wish I could remember everything!

I think what I said is probably valid when the first instance has suspended for over 10 secs. e,g if you have a piston to turn on a light after motion, wait 2 mins then turn it off. If the motion happens again, when the first instance wakes again, it is cancelled.

I’m not sure if you can achieve what you want with other task cancelation and/or task execution policy settings, I’m afraid its beyond my current webcore knowledge.

#8

Not that I can think of.

I don’t know why it is usually the full ten seconds. The code seems to be frequently checking something to see if the previous instance is still running, and that something seems to suggest it is, even when it stopped several seconds earlier. It could be that the check only works for certain pistons or it could even be a bug. I find the webCoRE code difficult to interpret.

#9

Hi again, just saying, after doing a bit more research and stuff…

If I use “Cancel all pending tasks” upon entering a piston, would it not kill any other running instances of that piston?

And only of that specific piston, right? So it will not kill off any other instance of a different piston that happens to be running, right?

#10

I’ve never used it, but my understanding is it just marks all currently scheduled future executions of the piston for cancellation. I think they may only get removed from the list when any instance of the piston exits (or perhaps starts up, or both).

#11

Thanks @orangebucket… I think it’s better I open a separate Topic about “Cancel all pending tasks” to get a definitive answer from someone who has used it. From my research on the community I thought it meant it would cancel all already running instances that are executing a “wait” command (i.e. have scheduled a task to continue running the piston later - or just waiting).

I’ll see what answers I get there… But again, it’s a little frustrating having to ask the community and not having a central definitive reference that explains with clarity and without ambiguity what a command is expected to do :confused:

#12

That is the practical effect of it. However in terms of implementation, pistons only actually schedule the next execution and they never actually unschedule ones they no longer need (unscheduling is discouraged by ST as it is resource heavy). So it is just a lot more passive than it sounds - ‘cancel’ often just means ‘take that off the to do list’.

#13

So some updates to this thread in regards to how things work on HE.

HE does basically similar behavior as describe above, except avoiding the semaphore side effects seen in ST.

So by default, serialization is on.

  • if the piston is running and another event comes in, it is queued up

  • when the piston execution that is running is completing, it checks if anything is queued up and if it is it continues running the queued up things (in order).

  • events queued up take precedent over timer processing except if a timer event is the current event (ie events in order are processed first)

The result is on HE, you don’t see much for semaphore waits, but events still do wait for the previous event to complete due to serialization

You can enable parallelism (ie events don’t queue but they run in parallel). In general I don’t expect one to see meaningful benefits as when parallel events execution is enabled, you use atomicState which has a lot of overhead…

I’m seeing right now in HE, that simple event processing start to finish is about 10ms, so it is pretty quick.