Incoming URLs json payload

#1

Hello all, I’ve been using incoming mailgun Webhooks to pistons with $args for a while and it’s been working fine, unfortunately mailgun changed their outbound webhooks to use json payload, for the life of me I couldn’t find an answer if it’s possible to parase those incoming hooks at this time? The json content does show up In the piston log. Thanks

Webcore Logs:

testing on $args.recipient

Comparison (dynamic) event-data:campaigns:[], delivery-status:[attempt-no:1, certificate-verified:true, code:250, description:, message:OK, mx-host:gmail-smtp-in.l.google.com, session-seconds:0.5049421787261963, tls:true, utf8:true], envelope:[sender:XXXXXX, sending-ip:198.61.254.13, targets:XXXXXXX, transport:smtp], event:delivered, flags:[is-authenticated:true, is-routed:false, is-system-test:false, is-test-mode:false], id:239MfwdoSYCSYqy240v-MQ, log-level:info, message:[attachments:[], headers:[from:“XXXXXX”, message-id:20181024020117.1.8BDAAD53DD7A7249@XXXXX, subject:Test e-mail, to:], size:565], recipient:[email protected], recipient-domain:gmail.com, storage:key:XXXXXXXXX, url:https://sw.api.mailgun.net/XXXXX

Comparison (dynamic) null is (string) [email protected] = false

the Json payload (via webhook.site)

{
“signature”: {
“timestamp”: “1540346479”,
“token”: “XXXXX”,
“signature”: “XXXXXX”
},
“event-data”: {
“tags”: [],
“timestamp”: 1540346478.534998,
“storage”: {
“url”: “https://sw.api.mailgun.net/XXXXX”,
“key”: “XXXXX”
},
“recipient-domain”: “gmail.com”,
“id”: “XXXXX”,
“campaigns”: [],
“user-variables”: {},
“flags”: {
“is-routed”: false,
“is-authenticated”: true,
“is-system-test”: false,
“is-test-mode”: false
},
“log-level”: “info”,
“envelope”: {
“sending-ip”: “198.61.254.13”,
“sender”: “XXXXX”,
“transport”: “smtp”,
“targets”: “XXXXXX”
},
“message”: {
“headers”: {
“to”: “”,
“message-id”: “20181024020117.1.8BDAAD53DD7A7249@XXXXX”,
“from”: “XXXXX”,
“subject”: “Test e-mail”
},
“attachments”: [],
“size”: 565
},
“recipient”: "[email protected]",
“event”: “delivered”,
“delivery-status”: {
“tls”: true,
“mx-host”: “gmail-smtp-in.l.google.com”,
“attempt-no”: 1,
“description”: “”,
“session-seconds”: 0.5049421787261963,
“utf8”: true,
“code”: 250,
“message”: “OK”,
“certificate-verified”: true
}
}
}

#2

Do some digging into $response, $json and parse as json. That may get you in the right direction. I have a feeling since it’s already in json format, you could use $response.storage.url to get https://sw.api.mailgun.net/XXXXX. It’s really tough to help, for me without it to pick at right in front of me.

I would try getting what is showing in the logs into a variable, and then try to drill down to the information you want to cull out.

#3

so $response.recipient test now shows: Comparison (dynamic) is (string) [email protected] = false

$args.recipient shows Comparison (dynamic) null is (string) [email protected] = false

log info $args shows the complete json as well:
event-data:campaigns:[], delivery-status:[attempt-no:1, certificate-verified:true, code:250, description:, message:OK, mx-host:gmail-smtp-in.l.google.com, session-seconds:0.4309380054473877, tls:true, utf8:true], envelope:[sender:XXXXXX, sending-ip:198.61.254.13, [email protected], transport:smtp], event:delivered, flags:[is-authenticated:true, is-routed:false, is-system-test:false, is-test-mode:false], id:PrhKsMP1QHKqGbOkOkl9og, log-level:info, message:[attachments:[], headers:[from:"xxxxx;, message-id:20181024035757.1.5E712F69CDFA0A45@XXXXX, subject:Test e-mail, to:[email protected];], size:565], recipient:[email protected], recipient-domain:gmail.com, storage:key:XXXXX, url:https://sw.api.mailgun.net/v3/XXXXXXXXX…[TRUNCATED]

#4

log info $response is null

#5

trying to parse json $args gets me Error parsing JSON data

#6

@ipaterson any ideas?

#7

$args is displaying as if it had already parsed the JSON into groovy collections. When you log $args it’s not logging JSON so my guess would be that it can be accessed with $args.event-data.recipient but I’m not sure if the hyphen character will work there. You might be able to do $args['event-data'].recipient if the former does not work.

#8

event-data.recipient was the answer thanks so much! for anybody else who might be searching

I am using the mailgun free account with webhooks for “delivered” emails to post to the piston, so basically any device that can send email and can have it’s own SMTP server specified (in my case reolink cameras)

thanks again

#9

That’s great to know, I had no idea that $args would parse JSON from a web request! Is Mailgun doing a POST to the piston URL with that data as the request body?

I suspect this would not work with a GET (i.e. adding ?{"some":"json","data":5} to the piston URL) in case anyone tries it out.

#10

Thanks @ipaterson. I knew if I called in a big gun, we could save some time helping them out

#11

IN THEORY it is possible to post the actual email from mailgun, I couldn’t get that to work right last time I tried (probably over a year ago) and this scenario is better for me because using the delivery hooks also gets me a notification email that goes to a folder on gmail/my phone with a jpg

I tried doing the whole thing of POSTing the email to a piston and trying to get the jpg out that might be too much at this point I think that has something to do with the whacky stuff smartthings did with s3 storage from what I recall

#12

What sorcery is this? Try as I might, I can’t get Webcore to include the hyphen as part of the tier name. I’m trying to scrape the “From” address for my own evil purposes though. I’ve tried $args.event-data.message.headers.from and $args.event-data.envelope.sender, and every permutation of event-data I can think of – $args['event-data'], $args.'event-data' and $args."event-data" (both of which the groovy manual says are valid for identifiers), $args.event\u002Ddata, but webcore denies them all. Even tried serializing str = replace(json($args), "event-data", "eventdata") but couldn’t find a way to deserialize the json string back into a hierarchical object.

The only workaround I’ve found is the evil kludge of string manipulation.

replace(json($args), '/^.+sender:([^@]+).+$/', '$1')

… which works, but it offends and shames me to take perfectly good hierarchical JSON and scrape it as flat text. What am I missing?

#13

I looked at my pistons, I had not used this setup in 2 years unfortunately but did confirm how I had it set up at the time, see below comparing to email address

which in the actual webcore code shows up as {$args.event-data.recipient}

image
image.png1710×960 63.5 KB

#14

I have since switch to nest cameras and am doing this through AWS Lambda which then triggers a much simpler piston

#15

Brilliant. Thanks! Yeah, I was manually entering $args as part of an Expression. Using Argument rather than Expression, webCoRE is more tolerant of the hyphen character in the object name. That’s exactly what I was missing.