Finding the SECOND lowest number in an array


#1

1) Give a description of the problem
I am dumping weather stats from WUnderground into an array, but sometimes WUnderground returns bad data.

2) What is the expected behavior?
I need to find the lowest LEGITIMATE number (ignoring the bogus numbers)

3) What is happening/not happening?
I am currently doing this:
min(a[0],a[1],a[2],a[3],a[4],a[5],a[6],a[7],a[8],a[9],a[10],a[11],a[12],a[13],a[14],a[15],a[16],a[17],a[18],a[19],a[20],a[21],a[22],a[23],a[24],a[25],a[26],a[27],a[28],a[29],a[30],a[31],a[32],a[33],a[34],a[35],a[36],a[37],a[38],a[39],a[40],a[41],a[42],a[43],a[44],a[45],a[46])
but that often returns the bad data, as seen below.

(Good data will ALWAYS be between 0 and 100)

Does anyone know of a way I can determine the SECOND lowest number in an array??
(or the lowest number in the range of 0-100 would work too)


#2

What about looping the array you get from Weather Underground and storing the lowest number to a variable if it is within the defined range?


#3

I like figuring things like this out.

Can you post the command you’re using to generate the variable array? I’ll use it to come up with a way to do what you’re trying to.


#4

This piston is grabbing the current barometer pressure, and storing it into a 47 numbered array. Each new data pull (every hour) bumps the oldest data off the list, so my variable {a} should always contain the previous 47 hours worth of data.

Here is my code:

  1. request the data
  2. math to make it relevant to me
  3. store final number into the last array slot

temp2

And it works flawlessly… until WUnderground sends back bad data (or maybe it’s null data) and then line 2 does the math on that.

You can read about the entire dilemma I am having at the following link if you wish, but finding the second lowest number in the array would ‘solve’ the problem.


#5

Can you post up some logs? Statements logging to console like I inserted in this piston will help you figure out what’s going on. In the samples I’ve tried, the condition evaluates as you’d expect it to:


#6

Your timing was perfect. Bad data just now came thru, and slipped past somehow:

I highlighted the bad data in the pic above


#7

I really like your idea @StevenJonSmith, but as far as I know, the barometric pressure in webCoRE is a forecast (in the future) instead of showing what has already happened. (I am only interested in the PREVIOUS pressure, and I cannot find a way to query that directly in webCoRE)

Since the pressure from the past few days (trend) helps us determine what weather is coming, I wanted to keep track of the current pressure, and store the data in an array… Keeping the previous 47 hours, before bumping the oldest point to make room for the latest data.

If someone knows of a way that webCoRE can query the hourly pressure from the past 2-3 days, I would love to hear about it!


#8

Do you have the logs from when it happened?


#9

Yes, shown 3 posts back… above the highlight


#10

I have since added more detailed logs to be written…

The 1st line is the only important one, showing if the latest data is accurate or not…
Line 2 & 3 simply let me know that there is bad data still lingering in the 47 point array
(my graph looks like crap until I have 47 hours in a row of good data)

temp


#11

I don’t see any logs in this thread, except the log to console stuff you added in post 10. To debug this, you need to turn logging level to full and then post up the logs from a time bad data slips through:

image


#12

OK, I will do that now… but one caveat though…
This piston uses 21 chunks, so my log for this piston is very limited in space.

In other words, if I don’t catch the issue within one hour of it happening, it will be lost.

Knowing this, do you really want it set to FULL ???


#13

I’m not sure if Medium will show the calculations being done. Try it, and if it does, then medium is enough. If not, it’d have to be full unfortunately.


#14

Here is a medium log for one single run of the piston. It looks like some data was lost at the bottom.
This was a good data test, but my question is, would it display enough information if the data was bad?

4/24/2018, 3:40:06 PM +948ms
+1ms 	╔Received event [Home].execute = :xxxxx: with a delay of 131ms
+245ms 	║Runtime (80988 bytes) successfully initialized in 58ms (v0.3.104.20180323) (244ms)
+246ms 	║╔Execution stage started
+574ms 	║║Executed virtual command setVariable (4ms)
+582ms 	║║Executed virtual command setVariable (4ms)
+586ms 	║║Executed virtual command wait (0ms)
+587ms 	║║Waiting for 500ms
+1105ms 	║║Executed virtual command setVariable (4ms)
+1109ms 	║║Executed virtual command wait (1ms)
+1110ms 	║║Waiting for 500ms
+1628ms 	║║Executed virtual command setVariable (4ms)
+1668ms 	║║Good = 48.26 is inside range 40 - 60 (Using 48.26)
+1669ms 	║║Executed virtual command log (1ms)
+1675ms 	║║Executed virtual command wait (1ms)
+1676ms 	║║Waiting for 500ms
+2198ms 	║║Executed virtual command setVariable (5ms)
+2207ms 	║║Executed virtual command setVariable (4ms)
+2359ms 	║║Executed virtual command setVariable (4ms)
+2495ms 	║║Executed virtual command setVariable (3ms)
+2498ms 	║║Executed virtual command wait (0ms)
+2499ms 	║║Waiting for 500ms
+3014ms 	║║Executed virtual command setVariable (5ms)
+3026ms 	║║Executed virtual command setVariable (4ms)
+3060ms 	║║Executed virtual command setVariable (5ms)
+3068ms 	║║BAD = pressureDiff > 60 (Bad data is present)
+3069ms 	║║Executed virtual command log (1ms)
+3095ms 	║║Executed virtual command setVariable (5ms)
+3109ms 	║║Executed virtual command setVariable (3ms)
+3116ms 	║║BAD = pressureMin < 0 (Bad data is present)
+3117ms 	║║Executed virtual command log (1ms)
+3124ms 	║║Executed virtual command wait (0ms)
+3124ms 	║║Waiting for 100ms
+3947ms 	║║Executed virtual command httpRequest (715ms)
+3951ms 	║║Executed virtual command wait (1ms)
+3952ms 	║║Waiting for 500ms
+4461ms 	║║Executed virtual command setVariable (5ms)
+4471ms 	║║Executed virtual command setVariable (4ms)
+4477ms 	║║Executed virtual command setVariable (3ms)
+4485ms 	║║Executed virtual command setVariable (4ms)
+4492ms 	║║Executed virtual command setVariable (5ms)
+4499ms 	║║Executed virtual command setVariable (4ms)
+4507ms 	║║Executed virtual command setVariable (5ms)
+4514ms 	║║Executed virtual command setVariable (4ms)
+4521ms 	║║Executed virtual command setVariable (4ms)
+4528ms 	║║Executed virtual command setVariable (4ms)
+4535ms 	║║Executed virtual command setVariable (4ms)
+4542ms 	║║Executed virtual command setVariable (4ms)
+4549ms 	║║Executed virtual command setVariable (4ms)
+4557ms 	║║Executed virtual command setVariable (5ms)
+4563ms 	║║Executed virtual command setVariable (3ms)

The real math takes place right above the line:
"+1668ms ║║Good = 48.26 is inside range 40 - 60 (Using 48.26)"
so I think we will be good if I am able to catch it immediately after it happens.


#15

Well, I just learned something new… Full logging on a 21 chunk piston returns NOTHING.
Yet Medium logging returns what you see in my previous post.

I guess Medium it is then.


#16

Maybe you can force the error by setting you {wuPressure} to a blank value or 0?


#17

After your post #5 above, I made a new testing piston, to force in 0 or leave it blank, and I could not replicate the problem. (webCoRE caught it every time)


#18

That’s interesting then. Gonna have to get lucky and catch it screwing up in the logs!


#19

Yea, I think you are right. I have an alert setup if the data is bad, so HOPEFULLY I am awake and home to catch it when it happens.

Currently, the past 47 hours have been flawless
stilepressure2

I will post bad logs the next time I see them


#20

I wonder if you could try catching it after-the-fact. Maybe by checking the values in your array and pausing the piston if any are below zero?