TWC - Sunrise/Set - Forecast & conditions slightly different

guxdude · 2020-07-28 21:15:12 UTC

in the weather documentation there is:

$twcweather.conditions.sunriseTimeLocal "2018-12-19T07:28:58-0500"

and

$twcweather.forecast.sunriseTimeLocal	    [
                                              "2018-12-19T07:28:58-0500",
                                              "2018-12-20T07:29:31-0500",
                                              "2018-12-21T07:30:02-0500",
                                              "2018-12-22T07:30:32-0500"
                                            ]

$twcweather.forecast.sunriseTimeLocal[0] seems to correspond with today according to the date but the rise time is not the same. for today, I logged these values when testing:

7/28/2020, 11:16:00 AM +90ms
+0ms	╔Received event [Las Vegas].test = 1595960160089 with a delay of 0ms
+1052ms	║conditions=2020-07-28T05:45:05-0700 forecast=2020-07-28T05:45:15-0700

I was planning to observe for a couple days to see if it is just reporting the wrong day or it is actually reporting something different than the local sunrise. I also haven’t looked in detail at sunset yet.

EDIT: FYI, I discovered this because conditions don’t have moonrise/moonset but forecast does. When I saw the forecast for sunrise/sunset, I got curious.

WCmore · 2020-07-28 23:14:18 UTC

Interesting… I had not yet compared those two time points. I ran my location thru similar tests, and my difference was approx 10 seconds, as was yours… This is no where near big enough of a difference for me to assume it is the next days data.

Too soon to confirm really, but it is almost like one uses (lets say) zip code, and one uses county data. A few miles difference would easily explain those 10 seconds.

I will set mine to log for a few days, to see if my hypothesis pans out, LOL…

guxdude · 2020-07-28 23:46:07 UTC

I didn’t think 10 seconds made sense for a whole day difference at this point in time.

So far, I can get the moon phase icons to show up in the title or footer text but not for major tics

WCmore · 2020-07-29 00:06:14 UTC

I had a feeling the rotation would cause error.
You might be able to use alt codes, or that other trick you used here.

Edit:

I just ran my piston again, and now all times match exactly.
(both sunrises match, and both sunsets match)

Edit 2:

Just tested again, and now they differ by 3 seconds.
The strange part is, no matter how off one was, the other was off identically

guxdude · 2020-07-29 01:33:46 UTC

You know, I was doing things fast when I was first getting started and I thought maybe my eyes were playing tricks on me. I think I also saw them align once and also saw them off by as much as 20 seconds but usually it is around 10. I don’t think the conditions ever change, though. Always seems to be the forecast. Something suspicious there.

WCmore · 2020-07-29 01:59:36 UTC

I have a hunch, but want my logs to run a bit longer first, LOL

Are yours also perfectly mirrored between the rise and set times?
(equal gaps between forecast and conditions?)

guxdude · 2020-07-29 06:07:53 UTC

So, I created a simple piston to just log the data every hour at 2 minutes past. Here is the data for the last 4 hours.

data	conditions	time	forecast	time
7/28/2020, 7:01:59 PM +94ms
║SUNRISE	7/28/20	5:45:05	7/28/20	5:45:09
║SUNSET	7/28/20	19:48:15	7/28/20	19:48:16
║MOON	7/28/20	14:32:01	7/29/20	1:10:59
7/28/2020, 8:01:59 PM +93ms
║SUNRISE	7/28/20	5:45:05	7/28/20	5:45:08
║SUNSET	7/28/20	19:48:15	7/28/20	19:48:12
║MOON	7/28/20	14:31:57	7/29/20	1:10:58
7/28/2020, 9:03:29 PM +858ms
║SUNRISE	7/28/20	5:45:05	7/28/20	5:45:05
║SUNSET	7/28/20	19:48:15	7/28/20	19:48:15
║MOON	7/28/20	14:31:59	7/29/20	1:10:55
7/28/2020, 10:02:46 PM +851ms
║SUNRISE	7/28/20	5:45:05	7/28/20	5:45:06
║SUNSET	7/28/20	19:48:15	7/28/20	19:48:14
║MOON	7/28/20	14:31:58	7/29/20	1:10:56
7/28/2020, 11:02:04 PM +392ms
║SUNRISE	7/28/20	5:45:05	7/28/20	5:45:14
║SUNSET	7/28/20	19:48:15	7/28/20	19:48:16
║MOON	7/28/20	14:32:01	7/29/20	1:11:04

NOTE: moon data is only available in forecast. The two columns are actually moonrise & moonset for the moon data.

Observations:

The condition values never change
The forecast values are different every time
The offset between forecast and condition for sunrise and sunset are sometimes equal and opposite but not always (see 7:02)
Sometimes the data matches (see 9:04)

EDIT: Added one more data point and rearranged chronologically.

WCmore · 2020-07-29 17:04:42 UTC

I am logging as well, with no identifiable patterns on Forecast yet.
(the variances are only a few seconds, so I am only logging minutes & seconds)

Even two minutes apart, the Forecast can jump 5 seconds, or more.

guxdude · 2020-07-29 17:31:02 UTC

Looking at the overnight logs, the forecast doesn’t update to the next day until sometime after 3 AM just like with the weather. So for 3 hours, they are off by a day.

WCmore · 2020-07-29 19:22:44 UTC

… and the rest of the time, it is off by a few seconds, LOL

Usually, [0] returns null from approx 3pm to 3am…

For some reason,
$twcweather.forecast.sunriseTimeLocal[0]
retains (somewhat inaccurate) data 24 hours a day…

Edit:

Maybe the null from 3pm to 3am only applies to “daypart[0]”

Edit 2:

Confirmed. Most (all?) $twcweather.forecast.___[0] has old data from midnight to about 3:15am.

Here is quick data for $twcweather.forecast.dayOfWeek[0]

Fri at 11:03pm returned Friday 
Sat at 12:03am returned Friday (incorrect)
Sat at 01:03am returned Friday (incorrect)
Sat at 02:03am returned Friday (incorrect)
Sat at 03:03am returned Friday (incorrect)
Sat at 04:03am returned Saturday (Finally!)

Note: I updated the Wiki to (hopefully) better explain TWC’s convoluted logic…

guxdude · 2020-07-29 21:22:52 UTC

So, I did another test with logging every 5 minutes and the data appears to change every 30 minutes (20 and 50 past the hour in my tests but could be anywhere between 15-20 and 45-50). Perhaps whichever station provides the update also sets the sun/moon forecast to their value?

WCmore · 2020-07-29 22:02:14 UTC

Strange… In my earlier testing, I’m pretty sure I caught my forecast.sunrise changing at least 4-5 times in the same hour.

guxdude · 2020-07-29 23:33:59 UTC

Could just be the short period I tested (~2 hrs). I set up a better piston to log for a longer period of time. Will let you know what I find…

WCmore · 2020-07-29 23:36:20 UTC

Slightly off topic, but I just realized that I need to go back in and edit some pistons…
(now that we know $twcweather.forecast.___[0] is only accurate ~20.5 hours a day)

Which unfortunately, likely includes all of these:

$twcweather.forecast.dayOfWeek
$twcweather.forecast.expirationTimeUtc
$twcweather.forecast.moonPhase
$twcweather.forecast.moonPhaseCode
$twcweather.forecast.moonPhaseDay
$twcweather.forecast.moonriseTimeLocal
$twcweather.forecast.moonriseTimeUtc
$twcweather.forecast.moonsetTimeLocal
$twcweather.forecast.moonsetTimeUtc
$twcweather.forecast.narrative
$twcweather.forecast.qpf
$twcweather.forecast.qpfSnow
$twcweather.forecast.sunriseTimeLocal
$twcweather.forecast.sunriseTimeUtc
$twcweather.forecast.sunsetTimeLocal
$twcweather.forecast.sunsetTimeUtc
$twcweather.forecast.temperatureMax
$twcweather.forecast.temperatureMin
$twcweather.forecast.validTimeLocal
$twcweather.forecast.validTimeUtc

guxdude · 2020-07-29 23:49:56 UTC

many of those have ‘conditions’ values which are more reliable. we usually use those.

WCmore · 2020-07-29 23:50:46 UTC

Note to self:

Perhaps an easy way to check to see if the data is current is by this little snippet:

IF $dayOfWeekName is equal to $twcweather.forecast.dayOfWeek[0]
    THEN Grab [0] data & assume it is today
    ELSE Grab [1] data & assume it is today
END IF

Hmmm… Perhaps even storing the 0 or 1 as a global… Which can automatically be referenced by all other weather pistons trying to grab the most current data…

Edit after deep thought:

I cannot give myself a , but this idea is definitely worthy.
It could literally solve TWC’s “outdated data” issue 100% across the board.

IE: This same logic should also be able to be applied to forecast.daypart[0] (which I use a lot).

WCmore · 2020-07-29 23:52:00 UTC

Which is likely why this 3+ hour boo-boo was over looked for so long.

guxdude · 2020-07-30 01:42:05 UTC

OK, my new piston has been running for a couple hours now and over the 2 hours, the values have changed 13 times. Definitely not as benign as this morning.

orangebucket · 2020-07-30 15:38:56 UTC

Interesting discussion going on here. However could you perhaps be overlooking the fact that the actual sunrise time varies depending on weather conditions?

So the figure for ‘conditions’ might perhaps be calculated assuming 34’ of refraction, or whatever the standard number used for predictions is, and the ‘forecast’ could be adjusting for the latest observed or forecast temperature and humidity and anything else that affects refraction.

I’m not necessarily suggesting that is what is happening, only that it is perhaps worth considering.

WCmore · 2020-07-30 17:41:14 UTC

Interesting point of view…

I know, from the point of view of the observer, the sunrise/set may change a few seconds due to weather… but so far, I have never seen any math formula to adjust that.
(and knowing how consistently unreliable TWC data changes are, I suspect they have not implemented that either)

My hypothesis is the following:
conditions.sunX are somewhat “fixed” times. (either based on zip code, city or county)
forecast.sunX seems to be calculated internally by TWC, and then the results are rounded.

When it comes to estimated math based time formulas, it is easy to be a few seconds off every time it is calculated.