Authentication Error With Twilio Webhook for SMS

Andrew_Cooper · 12-04-2019 01:30 PM

I keep getting the below error when trying to run a POST Webhook for Twilio’s messaging service. I have verified that my sid and auth token are correct multiple times.

What is the correct ‘language’ for the headers for the authorization? I have tried the following:

account_sid:‘123456’
auth_token:‘654321’
client:‘Client(account_sid, auth_token)’

Authorization:‘Basic <123456:654321>’

Authorization:’ Basic 123456:654321’

‘123456:654321’

I don’t know what else could be causing this. I am waiting to hear back from AppSheet support because the normal Custom Twilio service in SMS is not working either (saying I am using a Free account, when we are on a Business Account)

Here is the full set up of the webhook rule

LeventK

@Andrew_Cooper
I believe your HTTP content shall be Application/json when using POST with the Twilio

Andrew_Cooper

Tried it with that as well, and just tried again for safe measure. Still the same issues. I saw another post further back about ttrying URL_ENCODE which is why I switched to that. Never switched it back to JSON after

LeventK

@Andrew_Cooper
Are you trying to send an SMS or trying to record a reply that’s sent to your Twilio number? If you are trying to send, then you need to set your workflow as SMS not Webhook. When you select CUSTOM from the SMS list, than you will have fields to set your account sid and token

Andrew_Cooper

It’s a mix of both. We found a bug with AppSheet that wasn’t allowing business plans to do custom Twilio messages, so that will solve this, but we do want to be able to record/route the replies to the message as well. Will we need a hybrid of a GET and the custom Twilio sms? Or will we be able to do everything from Twilio after the first message is sent? (Still very new to Twilio)

LeventK

@Andrew_Cooper
You will need Google Apps Script for that. You need to create a standalone Google Apps Script, write a doPost(e) code to receive the reply, process it as needed and then deploy it as a webapp first. After deployment, record the webapp’s endpoint URL. Then you need to configure your Twilio messaging with using the below configuration and showing the webapp’s endpoint as a webhook endpoint.

When a SMS reply is received to your Twilio number, it will automatically trigger the designated custom webhook you have created and record the reply body to your gSheet.

LeventK

@Andrew_Cooper
This code will work for you provided you are unfamiliar with Google Apps Script. Make noted changes in the below code:

//=============================================================================
//                     CUSTOM TWILIO WEBHOOK FOR APPSHEET
//=============================================================================
// Created by: Levent KULACOGLU
// Created on: 18/MAY/2018
// Modified by: 
// Modified on: 
// Version: v1.000
//
// All Rights Reserved.
// (c)2018 ABLE3 VENTURES, LLC
//=============================================================================
var shtID = "Your gSheet ID goes here";
//=============================================================================

function doPost(e) {
  Logger.log("TwiML received a SMS message...Decoding message...");
  try {
    var keyValuePairs = e.postData.contents.split('&');
    var json = {};
    for(var i=0,len = keyValuePairs.length,tmp,key,value;i <len;i++) {
      tmp = keyValuePairs[i].split('=');
      key = decodeURIComponent(tmp[0]);
      value = decodeURIComponent(tmp[1]);
      if(key.search(/\[\]$/) != -1) {
        tmp = key.replace(/\[\]$/,'');
        json[tmp] = json[tmp] || [];
        json[tmp].push(value);
      }
      else {
        json[key] = value;
      }
    }
    Logger.log("TwiML Raw Data > "+JSON.stringify(json));
    
    var smsBody = json.Body
    smsBody = smsBody.replace(/(?!(\s|:|;|'|@|-|[.]|[,]|[/]|p{P}))\W+/g," ");
    
    SpreadsheetApp.openById(shtID).getSheetByName("YourDesignatedTabName").getRange(DesiredRowNr, DesiredColumnNr).setValue(smsBody);
    
    return XmlService.createDocument();
    
  } catch (e) {
    MailApp.sendEmail("YourEmailAdressToReceiveErrorMessage", "Twilio Webhook Error Occured!", 
      "\r\nMessage: " + e.message
      + "\r\nFile: " + e.fileName
      + "\r\nLine: " + e.lineNumber);
    e = (typeof e === 'string') ? new Error(e) : e;
    Logger.severe('%s: %s (line %s, file "%s"). Stack: "%s" . While processing %s.',e.name||'', 
               e.message||'', e.lineNumber||'', e.fileName||'', e.stack||'', processingMessage||'');
    throw e;
  }
}

Phil

You want to produce the following. I am not sure of the syntax, but this is essentially what we do internally.

"Basic " + Base64Encoded(TwilioAccountSID + “:” + TwilioAuthToken)

Andrew_Cooper

Thank you so much for all of this. I saw your first post last night and my immediate thought was ‘well this will take a while since I don’t know the first thing about Google Scripts’, so I really appreciate this.

On the subject of the authorization header though, what is the correct format as far as AppSheet goes, in the event we decide to use the webhook functionality for something else down the road?

LeventK

@Andrew_Cooper
You’re welcome, it’s my pleasure. That totally depends on to what kinda API/webhook endpoint you will make a call and what is the API POST structure of that API/webhook. In general, API endpoints accepts POST method but the header information might differ between the vendors. Some API endpoints require the authorization in the URL, some require the authorization in the header and some require the authorization inside the POST body. I humbly advise read the vendor’s API documentation thoroughly. Should you require any assistance, just post the community and we will be glad to help you out for sorting out the problem.

Andrew_Cooper

It is for Twilio’s webhook for all of their functions. Their documentation states that it follows Basic HTTP Authentication, but I tried that, along with all of the methods in the original post with no results. Their documentation only shows the actual ‘call’ set up for Curl, Ruby, Python, etc, but not HTTP.

So if anyone on here has done a successful webhook for anything using Appsheet, any help would be appreciated. No matter how I format the authorization part of the header, it gives me an authentication error.

LeventK

@Andrew_Cooper
Your HTTP Content Type shall be application/json or JSON, that’s for sure. Remove all your HTTP headers and construct a JSON Payload inside the Body

{
	"Body": "<<URLENCODE("Your body message here")>>",
	"From": "+1234567890",
	"To": "+1234445566"
}

Also add below as an HTTP Header

Authorization: "Basic " + Base64Encoded(TwilioAccountSID + “:” + TwilioAuthToken)

Andrew_Cooper

I will try that out in the morning and let you know. Thanks again for all of the help with this!

Andrew_Cooper

So the header portion is working! Now, the next set of issues (which this seems almost like an appsheet issue, but I am very new to webhooks and APIs so it is probably me doing something wrong)

Every time I try to send a test or execute the webhook, it tells me there is no body, or gives me another error, no matter how I format the body portion of it. Even if I have Body:“test” as a header. The Log shows those \s appearing out of nowhere (“Payload”:"{“Body”:…) so I don’t know if that has anything to do with it or not.

I’ve been trying at this isnce last night and for about 30 minutes this morning and I am at a loss now.

AppSheet Error: Workflow rule ‘SendText’ action ‘Test Text Response’ Body template. Expression ‘URLENCODE(“test”)’ is invalid due to: Unable to find function ‘URLENCODE’, did you mean ‘USERROLE’?.

No Body Error:

LeventK

@Andrew_Cooper
It’s totally my bad, do apologize…It should be ENCODEURL

Andrew_Cooper

That fixed that part, but it is still giving the same body error.

From the log, it looks like AppSheet isn’t reading the body correctly. It keeps showing those random 's in the audit log and in the tests.

LeventK

@Andrew_Cooper
Can you please contact me via levent@able3ventures.com? I would like to test the endpoint thru Postman and try to figure out what might be the issue. Than we will have an idea. Thnx.

Andrew_Cooper

I just saw this but I (finally!!!) figured it out

Twilio only accepts HTTP Post requests in form-urlencoded format.

So after testing it in Postman (for both calls and messages), the format needed to be:

And now I can breathe. Time to test this out with row level data

EugeneB

@Andrew_Cooper is that possible to send multiple message in the TO:? how do you do it with the row level data?

Phil

I have updated the webhook documentation to explain how you can specify a Basic authentication header in a webhook call.

See topic " HTTP Basic Authentication Header" in this article https://help.appsheet.com/en/articles/962008-invoking-webhooks-from-a-workflow-rule-or-scheduled-rep...

LeventK

@Phil you’re an awesome man!!

Phil

Hi Andrew,

Yes, I was in the process of replying to let you know that when I saw your latest post.
My internet service was down this morning so just getting back online now.