Upgrade Sendgrid web hook code to new API

[adooylabs]

Last feb sendgrid announced that they are changing their event notification API. Any plans with regards to this?

[stereoscott]

Looks like the post describing the changes are here: http://sendgrid.com/blog/event-webhook-update-batched-vs-unbatched-events/

@adooylabs, want to take a stab and forking this and making a pull request to lee?

[mikemarsian]

+1 for an update that would support the new format.

I was sure 0.2 already supports the new format, since it says in the changelog "0.2 - Supports version 3 of the Sendgrid webhook released on September 6th, 2013, which provides the proper headers and JSON post body without hacks or middleware. If upgrading to this version, please make sure to update the Webhooks settings in your SendGrid App to use V3 of their API"

But I guess that's a different change from the one announced in February by SendGrid

[mikemarsian]

@leejarvis any plans to implement these changes? Unfortunately currently this excellent gem is unusable.

[stereoscott]

@mikemarsian pull requests are welcome! ;)

[leejarvis]

@mikemarsian Sorry but I don't currently have the time to invest in this. I don't use SendGrid or this gem anymore. I think it would be great for gridhook to support the new API functionality which is why I am calling on people like yourself who use and care about this, to help me maintain and add support for the changing API at SendGrid. As Scott suggests, we're open to pull requests for this functionality, and I'm happy to add commit bit for more contributors so this library remains useable and useful.

[stereoscott]

@mikemarsian @adooylabs I just dug into this a bit and as far as I can tell the API changes announced in February don't break this gem. It sounds like they just stopped sending unbatched events, but this gem supports the newer batched format.

I just tried a test event notification (https://sendgrid.com/app/appSettings/type/eventnotify/id/15) against the endpoint provided by the gem and sengrid sent over a batch of events which rails identifies as json and passes into the app in the "_json" parameter. This in turn gets handled by this line: https://github.com/leejarvis/gridhook/blob/master/lib/gridhook/event.rb#L14

Were there other changes made to the webhook API that I'm not seeing?

[mikemarsian]

Hi @stereoscott @adooylabs @leejarvis ,
I have tried sending webhook post events using SendGrid's test tool and receive them in grdihook, and it doesn't work. I also tried sending post events using Postman (with the same data as in Sendgrid's test) and I receive them in gridhook just fine.

Here are the RequestBin's of both posts:

HTTP post using SendGrid test tool (gridhook returns 200 ok, but doesn't fire up any events in the app):
http://requestb.in/12fdsy91?inspect

HTTP post using Postman with the same data (gridhook fires up events and everything works just fine):
http://requestb.in/tyorgkty?inspect

I have tried adding newlines to postman's data (to be similar to Sendgrid's format), and it still works.

Any idea what's going on?

[leejarvis]

At a quick glance it looks like the request content-type differs (application/json in the first vs text/plain in the latter). That's probably the issue (we should allow both, and have tests to support this)

[stereoscott]

@mikemarsian if you don't mind, give this branch a go: https://github.com/quikly/gridhook/tree/accept-plain-text

That version will attempt parse the post body as JSON even if the json content-type header is not set.

[stereoscott]

I'm wondering why when sendgrid notifies my app it uses the proper content type, but @mikemarsian is seeing something different. I'm still using this gem in production and the sendgrid notifications continue to work properly. And when sendgrid says to test out your local implementation using curl to run:

curl -X POST -H "Content-Type: application/json" -d '[{"email":"john.doe@sendgrid.com","timestamp":1337197600,"smtp-id":"<4FB4041F.6080505@sendgrid.com>","event":"processed"},{"email":"john.doe@sendgrid.com","timestamp":1337966815,"category":"newuser","event":"click","url":"http://sendgrid.com"},{"email":"john.doe@sendgrid.com","timestamp":1337969592,"smtp-id":"<20120525181309.C1A9B40405B3@Example-Mac.local>","event":"processed"}]'

...which seems to imply they send the correct content-type header (see: https://sendgrid.com/docs/API_Reference/Webhooks/event.html)

[mikemarsian]

Sending post with curl with SenGrid's provided example (that has application/json content type) works just fine with my gridhook. So it must be something else.

[ckozus]

@stereoscott I'm using the branch you suggested and it works perfectly with the current SendGrid API. Is there any chance accept those changes as a pull request?

Anything I could do to help this happen? I think it's best for all if we keep it that way.