TWC - Sunrise/Set - Forecast & conditions slightly different

#21

The confusing thing is why it would change so often. Sometimes, it goes slower but I have seen it change every 5 minutes for half an hour. Do they really update the forecast that often? I wouldn’t think so.

#22

From what I have seen, all indications point towards a rounding math formula.
(likely because they are trying to “guesstimate” times for those living between stations)

So my hunch is, the data they are pulling from has not changed, but because each pull uses slightly different math (the current time is always changing), then the output will consistently be a smidgen off.

I see this kind of variance a lot when doing math with time.

Here is a quick visual… Two formulas, based on the same data, plus the ever changing time:
pic
One pair matches, one pair is slightly off, due to rounding 21 decimals.
(the variance would be much greater if I only used 3 decimals)

One other thought just popped into my head…

I wonder if $twc.forecast is based on our GPS location… Meaning, GPS jumps a bit throughout the day, and would account for the slight changes.

Wait a second, this can’t be…
My cell does not report GPS to webCoRE, so from webCoRE’s point of view, my GPS should never change…

#23

They might perhaps base sunrise/sunset predictions on the current conditions. Just as likely to be a different explanation though.

A document I was reading, that was produced by HM Nautical Almanac Office, suggested that the height relative to the horizon and weather conditions could make a difference of a minute or two in the observed sunrise and sunset times compared to the standard predictions.

It is still curious, though.

#24

This is 100% true. I have watched the sunset from sealevel, then immediately got into an elevator to the 15th floor… Where I watched the sunset a second time!

Note: This works best if both you and the elevator move swiftly.

I would be sincerely impressed with TWC if they implemented that… but for the benefit of the doubt, I am now considering logging these in my test piston to better understand.

$twcweather.conditions.cloudCeiling » null, 100, 2100, 8500
$twcweather.conditions.cloudCoverPhrase » "Clear", "Partly Cloudy", "Mostly Cloudy", "Cloudy"
$twcweather.conditions.visibility » 16.09

… but in all honesty, there are many other dataPoints that affect our perception as well… This one may also be tricky to ‘nail down’.

#25

I think it is a little unlikely too. However it seems temperature, humidity and pressure make a difference.

#26

So I recorded the forecast narrative along with the sunrise/set times. It often changed at the same time the times changed but not always. It does appear there are frequent updates but the frequency is not consistent. Here is a plot of the delta time for condition vs forecast over the last 24+ hours:

chart
chart.png1155×371 19.5 KB

Neglecting the midnight to 3 am, you can see a little clearer:

chart-2
chart-2.png1155×371 29.5 KB

EDIT: Should have mentioned the data is every 5 minutes.

#27

Thanks for this. I appreciate the visual.

I think I can better elucidate my thoughts now…

I suspect somewhere in the backend (the weather source? IBM? SmartThings?):

  • a time is being converted into a decimal
  • math is being done with that decimal
  • the results are converted back into time (a few secs off)

To clarify, I have no hard proof of this.
This hunch is based on my past experiences with time and calculations.
(depending on the complexity of math, it is rare those 3 steps return a perfect answer)

What baffles me though is, why would the sunrise math at 2pm be any different than the sunrise math done at 3pm? Mathematically speaking, the current time should not be in the equation when calculating the sunrise or sunset for the day.

Now, if I was defending TWC for a moment… Most of the data in forecast can benefit from math based on the current time. For example, if I was calculating the forecast wind between two points, doing math based on the current time makes sense.
► 2pm forecast of 4mph
► 3pm forecast of 8mph
… then we can use math to “guesstimate” 2:30 to be 6mph

This strategy of estimating the “gradient spectrum” seems to be a common thing with TWC, especially noticeable on fast changing data like wind speed. (IE: Nowhere near the accuracy levels of WUnderground)

If my ramblings above are accurate, then someone from above just needs to stop using a time in the formula when estimating the sunrise/sunset…

(or the lazy man’s way, and just let the forecast reference the condition)

However it is done, logically, the sunrise & sunset times should only change once per day. (per location)

However, I am grateful for this quirk, because it helps to showcase some of the convoluted logic in TWC’s weather data.

It’s not just forecast though. I have actually seen this multiple times using the condition:
$twcweather.conditions.sunriseTimeLocal
Here it switched 3 times in 2m 12s:
pic

#28

I believe the math can cause issues but I wonder if it is just a reporting issue. Say there are 10 weather reporting stations within 100 miles of me (maybe more if they include any crowdsourcing data?). If they take into account the position of all the stations reporting at a given update and do some sort of averaging, that could shift the “location” of the forecast around enough to make a difference depending on which stations reported or didn’t. Or not. Probably never know unless someone at TWC chimed in.

#29

Good point.

I think the “gradient spectrum” includes both time and distance, as well as averages from multiple sensors. (which would explain why all my numbers look “washed out”, with no real spikes ever occurring)

Edit: Added pic to show typical example ($twcweather.conditions.windSpeed)

pic

In real life, the speed varied much more than this “flat-line” number.
(IE: the more they try to ‘smarten’ up their algorithm, the ‘dumber’ the results)

*sighs*

I really miss the raw data from WUnderground…

Speaking of which… Is anybody willing to share a private API key with me?
(I promise to stay within agreed upon limits)

#30

… A few months later…

After analyzing this giant log, I have a few more observations:

$twcweather.conditions.sunriseTimeLocal
$twcweather.conditions.sunsetTimeLocal

  • These two stays consistent all day long…
  • Conditions update to the new day sometime between 12 am and 12:35 am.

$twcweather.forecast.sunriseTimeLocal[0]
$twcweather.forecast.sunsetTimeLocal[0]

  • These two change slightly all day long, but generally stays within a 7 sec window.
  • Forecasts update to the new day sometime between 3 am and 3:35 am.

Here is an 8 hour snippet showing minutes and seconds…
The consistent “Conditions” are in yellow…
The fickle “Forecasts” are in blue.

pic

Notice how often Forecasts jump around, and how it took over three hours to update to the new day.

TL;DR:

I would not recommend using either of these lines for current queries.

$twcweather.forecast.sunriseTimeLocal
$twcweather.forecast.sunsetTimeLocal

Although in all fairness, if you need to “predict” the sunset two weeks in the future, I guess 7 sec off is no big deal.:
$twcweather.forecast.sunsetTimeLocal[13]

Storing Seconds in a Variable (Precise Sunrise & Sunset)
#31

Agree. I only started looking at these because I wanted moon rise/set data. While sun rise/set is available in both conditions and forecast, moon data is only available in forecast for some reason. Would love to have more reliable ‘condition’ data for the moon rise/set. For now I am stuck using the forecast and make sure to update after 3:30 am.

#32

My apologies… That wasn’t meant to be directed at you specifically…
I was just tidying up, and did one more analysis before pausing that test piston.

I had forgot I had left it running… :man_facepalming: