We Keep Coding

CakePHP 4.x Ajax request with CSV-file

I am developing a responsive user interface in CakePHP 4.x which occasionally uses Ajax requests.
My Ajax requests are performing just fine but I am having a lot of trouble incorporating a CSV-file in the request so my controller can handle the data. What I want to accomplish is that that I can choose a CSV-file, press submit and that the Ajax-request sends the file to the controller and uses the independent rows to update the database.
My code:
Javscript:
function importProducts() {
/* Getting form data */
let form = document.getElementById('importProductsForm');
let formData = new FormData();
let file = $(form.products_file).prop('files')[0];
formData.append("csv_file", file);
/* Moving product stock */
ajaxRequest('Products', 'importProducts', formData, processImportProducts);
}
function ajaxRequest(controller, action, data = null, callback = null) {
$.ajax({
url : "<?=$this->Url->build(['controller' => '']);?>" + "/" + controller + "/" + action,
type : 'POST',
data : {
'data': data
},
dataType :'json',
/*processData: false,*/
/*contentType: false,*/
success : function(dataArray) {
let response = dataArray.response;
if (typeof response.data !== 'undefined') {
data = response.data;
if (callback != null) {
callback(data);
}
} else if (response.success == 0) {
data = null;
giveError(response.errorTemplate);
} else {
data = null;
if (callback != null) {
callback(data);
}
}
},
error : function(request,error)
{
console.error(error);
}
});
}
At the moment the controller function does not do anything special but receiving and setting the data:
public function importProducts() {
$this->RequestHandler->renderAs($this, 'json');
$response = [];
if($this->request->is('post')) {
$data = $this->request->getData();
$response['test'] = $data;
} else {
$response['success'] = 0;
}
$this->set(compact('response'));
$this->viewBuilder()->setOption('serialize', true);
$this->RequestHandler->renderAs($this, 'json');
}
After some research I discovered I could use the FormData object to send the file. The error I then received was 'illegal invocation'. After some more research I discovered this had to with automatic string parsing by Ajax. According to some other StackOverflow posts I could resolve this by setting the processdata and contenttype properties to false. This fixed the problem but resulted in an Ajax request which always would be empty (that does not contain any data). I tested this without the CSV-file with a regular data object that contains a variable with a string but also resulted in a empty request (no data send to controller).
So my problem is that without the processdata property as false I get the 'illegal invocation' error, otherwise with processdata as false I literary do not receive any data in my controller. I am looking for solution to resolve this problem so I can send my CSV-file or at least the data within the file to my controller.
Other solutions than using the FormData are also welcome, for example I tried to read the CSV-file in Javascript and turn this into another object (with the jquery csv api) to send to the controller, sadly without success until now.

Node JS Express how can I send a 404 error if a bad request is made to third party API?

In my Node JS server I have this route handler that sends a request to a third party API to get a username:
app.get('/players/:player', apiLimiter, function(request, response) {
const player = request.params.player;
const api_url = `https://api.com/shards/steam/players?filter[playerNames]=${player}`;
var options = {
method: "GET",
observe: 'body',
};
let apiRequest = https.request(api_url, options, function (res) {
let data = "";
res.on("data", chunk => {
data += chunk;
})
res.on("end", () => {
let objectParsed = JSON.parse(JSON.stringify(data));
response.send(objectParsed);
})
if(!player) {
res.status(404).send("Not found.");
}
})
apiRequest.end();
})
This works fine to get a user that exists. However, if I put in a fake username to my /players page, that page still loads with a 200 status instead of getting a 404 response. The page loads and looks broken because it's not actually getting any data from the API.
I feel like this is a dumb question .. In my research I have found how to handle errors if it's just the route, and not if it's the route dependent on the path parameter as in /players/:player
I found a question that was similar to mine (How to throw a 404 error in express.js?) and I tried using an If statement: if (!player){res.status(404).send("Not found."); } but no dice. Am I using this if statement in the wrong place?
How can I get my Node JS server to respond with a 404 if the user from the database doesn't exist?

You have to check the result of the API call and see if you got valid data back and send the 404 there. I also added a check to make sure something was passed for the player name and send back a 400 (bad request) if there's no player specified at all:
app.get('/players/:player', apiLimiter, function(request, response) {
const player = request.params.player;
if (!player) {
res.status(400).send("No player specified.");
return;
}
const api_url = `https://api.com/shards/steam/players?filter[playerNames]=${player}`;
var options = {
method: "GET",
observe: 'body',
};
let apiRequest = https.request(api_url, options, function(res) {
let data = "";
res.on("data", chunk => {
data += chunk;
})
res.on("end", () => {
let objectParsed = JSON.parse(data);
// test objectParsed here
if (!some condition in objectParsed) {
res.status(404).send("No data for that player name.");
} else {
response.send(objectParsed);
}
});
});
apiRequest.end();
});
Also, you don't want JSON.parse(JSON.stringify(data)) here. Your data is already a string. Just do JSON.parse(data).
FYI, if you use a small http request library such as got(), this code gets a lot simpler as it accumulates the response and parses the JSON for you in one line of code as in:
let data = await got(options).json()

Get intents, entities, contexts and all data

In the case, the actually conversation-simple have one function with all the values, but the function update every time if flows conversation.
I want create one function or other form to be able to capture all that data that is currently on the data.
In the case have Intents, context, entities, etc.
conversation.message(payload, function(err, data) {
if (err) {
return res.status(err.code || 500).json(err);
}
return res.json(updateMessage(payload, data));
});
});
The data inside updateMessage parameter have all I need, but if I create other function and try get this values, does not work.
In the case I use the values and get with app.js for open some REST webservice.
I try it:
function login (req, res) {
numberOrigin = null;
sessionid = null;
var dataLogin = {
data: { "userName":"xxxxx","password":"xxxxx","platform":"MyPlatform" },
headers: { "Content-Type": "application/json" }
};
client.registerMethod("postMethod", "xxxxxxxxxxxxxxx/services/login", "POST");
client.methods.postMethod(dataLogin, function (data, response) {
if(Buffer.isBuffer(data)){
data = data.toString('utf8');
console.log(data);
var re = /(sessionID: )([^,}]*)/g;
var match = re.exec(data);
var sessionid = match[2]
console.log(sessionid);
}
});
}
function openRequest(data, sessionid, numberOrigin ){
//console.log(data); dont show the values.. show the data response of login
var dataRequest = {
data: {"sessionID": sessionid,
"synchronize":false,
"sourceRequest":{
"numberOrigin":numberOrigin,
"description": JSON.stringify(data.context.email) } },
headers: { "Content-Type": "application/json" }
};
numberOrigin +=1;
client.post("xxxxxxxxxxxxxxxxxx/services/request/create", dataRequest, function (data, response) {
if(Buffer.isBuffer(data)){
data = data.toString('utf8');
console.log(data);
}
});
}
function updateMessage(res, input, data, numberOrigin) {
var email = data.context.email; // this recognize but this function is responsible for other thing
if (email === 'xxxxxxxxxxxx#test.com') {
console.log(data);
login(data);
openRequest(data, sessionid, numberOrigin)
}
}
In case, I just want get the values with my app.js for use inside REST. I got it with ajax but everything on the client side (index.html), and that made me show my credentials, so I decided to do it in REST for security my code..
If have some form to solved this, please let me know.
If have other form to do it, I'll be happy to know.
Thanks advance.

The issue is likely that you need to write to the response object res.. In the updateMessage function the response is passed in. In order for data to be sent back to the browser you need to write to the response. I have a demo app which calls the weather channel to get the weather based on an intent, similar to what you are trying to do with your login function. Please take a look at this code
https://github.com/doconnor78/conversation-simple-weather/blob/master/app.js#L130
You will need to pass the original res (response) object into the appropriate function and then write data to the response (res) once you get it from the third party service.

Reading claims from another application using jquery

I have a static method in static class in MVC application that returns User Claims, when I access directly the url of this application,I am getting those values but when I access application url from another application using Javascript,it is not returning anything.I am not getting any error.It is returning empty result.I am also not getting CORS issue.i suspect it is something related to authentication & passing user credentials,both site is under same ADFS configuration
public static UserDetails GetUserDetails()
{
var userdetails = new UserDetails();
var objClaims = ((ClaimsIdentity)Thread.CurrentPrincipal.Identity).Claims;
foreach(var c in objClaims)
{
else if (c.Type == ConstantsHelper.emailAddress)
{
userdetails.Email = c.Value;
}
else if (c.Type == ConstantsHelper.userName)
{
userdetails.UserName = c.Value;
}
else if (c.Type == ConstantsHelper.shortName)
{
userdetails.ShortName = c.Value;
}
}
return userdetails;
}
Code to access it from another application.
function GetLoggedInUsermethod() {
var url = GetLoggedInUser;
$.ajax({
type: "GET",
url: url,
crossDomain: true,
success: function (json) {
},
error: function (e) {
}
});
}

If the calling application is hosted in different domain (different ports will qualify for different domains), then you may need to add the Access-Control-Allow-Origin header to the response with the value set to the calling application's domain for the call to succeed.
More details here.

Posting to Twitter through OAuthSimple.js

I've been stuck on this one for a while. I'm trying to use OAuthSimple.js to interact with Twitter in a Chrome extension I've written.
The signing process seems to work fine for requests to retrieve a user's statuses, but I can't seem to construct a request that will successfully authenticate when I try to retweet, reply, or mark a tweet as favorite.
I'm following the guides here. I have also tried numerous ways of structuring the request, and comparing the request contents against the output of the OAuth tool provided by Twitter ( which seems to check out ), but I'm still getting 401 errors and generic "We couldn't authenticate you" responses.
Here's how I'm trying to form the request:
var sendTwitterRequest = function(url, params, method, callback) {
var request = null;
if ( localStorage.twitterAuthToken ) {
OAuthSimple().reset();
request = OAuthSimple(TwitterConsumerKey,TwitterConsumerSecret).sign({
action:method,
method:"HMAC-SHA1",
dataType:"JSON",
path:url,
parameters:params,
signatures:{
oauth_version:'1.0',
oauth_token:localStorage.twitterAuthToken,
oauth_secret:localStorage.twitterAuthVerifier
}
});
console.log(request);
$j.ajax({
url:request.signed_url,
type:method,
data:request.parameters,
success:callback
});
}
};
Then, making calls into this method like this:
// this works, I get the data and can do stuff with it
sendTwitterRequest('http://api.twitter.com/1/statuses/user_timeline.json?user_id=',null,'GET',someMethod());
// this fails and throws a 401 error every time
sendTwitterRequest("https://api.twitter.com/1/statuses/retweet/"+tweetKey+".json",null,'POST',someOtherMethod());
Am I missing something? Thanks in advance!

It turns out the requests I am creating are fine, I just needed a final one to exchange request tokens for OAuth tokens. I thought this step was covered when the user was prompted for input, but turns out I was wrong.
I also ended up switching from OAuthSimple.js to just OAuth.js, on account of the fact that I could only get OAuth.js to process both the token requests and the timeline requests.
Some of this is pretty specific to what my application is doing, so you will probably need to modify it.
The new sendTwitterRequest method:
var sendTwitterRequest = function(options){
var accessor={
consumerSecret:TwitterConsumerSecret
};
var message={
action:options.url,
method:options.method||"GET",
parameters:[
["oauth_consumer_key",TwitterConsumerKey],
["oauth_signature_method","HMAC-SHA1"],
["oauth_version","1.0"]
]
};
if(options.token){
message.parameters.push(["oauth_token",options.token])
}
if(options.tokenSecret){
accessor.tokenSecret=options.tokenSecret
}
for(var a in options.parameters) {
message.parameters.push(options.parameters[a])
}
OAuth.setTimestampAndNonce(message);
OAuth.SignatureMethod.sign(message,accessor);
try {
$j.ajax({
url:message.action,
async:options.async||true,
type:message.method||'GET',
data:OAuth.getParameterMap(message.parameters),
dataType:options.format||'JSON',
success:function(data) {
if (options.success) {options.success(data);}
}
});
} catch ( e ) {
}
};
And the methods that depend on it:
// asks Twitter for an oauth request token. User authorizes and the request token is provided
requestTwitterToken = function() {
// this is semi-specific to what my extension is doing, your callback string may need
// to be slightly different.
var callbackString = window.top.location + "?t=" + Date.now();
var params = [
[ 'oauth_callback', callbackString ]
];
sendTwitterRequest({
url: "https://api.twitter.com/oauth/request_token",
method: 'POST',
parameters: params,
format: 'TEXT',
success: function(data) {
var returnedParams = getCallbackParams(data);
if ( returnedParams.oauth_token ) {
chrome.tabs.create({
url:"https://api.twitter.com/oauth/authorize?oauth_token=" + returnedParams.oauth_token
});
}
},error:function( e ) {
console.log( 'error' );
console.log( e );
}
});
};
// exchanges the Twitter request token for an actual access token.
signIntoTwitter = function(token, secret, callback) {
var auth_url = "https://api.twitter.com/oauth/access_token";
var authCallback = function(data) {
var tokens = getCallbackParams(data);
localStorage.twitterAuthToken = tokens.oauth_token || null;
localStorage.twitterAuthTokenSecret = tokens.oauth_token_secret || null;
callback();
};
try {
sendTwitterRequest({url:auth_url, method:'POST', async:true, format:'TEXT', token:token, tokenSecret:secret, success:authCallback});
} catch ( e ) {
console.log(e);
}
};
With this, the steps are as follows:
ask Twitter for a token ( requestTwitterToken() ) and provide a callback
in the callback, check to see if a token is provided. If so, it's an initial token
pass the token back to Twitter and open the Twitter auth page, which allows the user to grant access
in the callback to this call, see if an access token was provided
exchange the request token for an access token ( signIntoTwitter() )
After that, I simply use the sendTwitterRequest() method to hit Twitter's API to fetch the timeline and post Tweets.