We Keep Coding

CakePHP 4.x Ajax request with CSV-file

I am developing a responsive user interface in CakePHP 4.x which occasionally uses Ajax requests.
My Ajax requests are performing just fine but I am having a lot of trouble incorporating a CSV-file in the request so my controller can handle the data. What I want to accomplish is that that I can choose a CSV-file, press submit and that the Ajax-request sends the file to the controller and uses the independent rows to update the database.
My code:
Javscript:
function importProducts() {
/* Getting form data */
let form = document.getElementById('importProductsForm');
let formData = new FormData();
let file = $(form.products_file).prop('files')[0];
formData.append("csv_file", file);
/* Moving product stock */
ajaxRequest('Products', 'importProducts', formData, processImportProducts);
}
function ajaxRequest(controller, action, data = null, callback = null) {
$.ajax({
url : "<?=$this->Url->build(['controller' => '']);?>" + "/" + controller + "/" + action,
type : 'POST',
data : {
'data': data
},
dataType :'json',
/*processData: false,*/
/*contentType: false,*/
success : function(dataArray) {
let response = dataArray.response;
if (typeof response.data !== 'undefined') {
data = response.data;
if (callback != null) {
callback(data);
}
} else if (response.success == 0) {
data = null;
giveError(response.errorTemplate);
} else {
data = null;
if (callback != null) {
callback(data);
}
}
},
error : function(request,error)
{
console.error(error);
}
});
}
At the moment the controller function does not do anything special but receiving and setting the data:
public function importProducts() {
$this->RequestHandler->renderAs($this, 'json');
$response = [];
if($this->request->is('post')) {
$data = $this->request->getData();
$response['test'] = $data;
} else {
$response['success'] = 0;
}
$this->set(compact('response'));
$this->viewBuilder()->setOption('serialize', true);
$this->RequestHandler->renderAs($this, 'json');
}
After some research I discovered I could use the FormData object to send the file. The error I then received was 'illegal invocation'. After some more research I discovered this had to with automatic string parsing by Ajax. According to some other StackOverflow posts I could resolve this by setting the processdata and contenttype properties to false. This fixed the problem but resulted in an Ajax request which always would be empty (that does not contain any data). I tested this without the CSV-file with a regular data object that contains a variable with a string but also resulted in a empty request (no data send to controller).
So my problem is that without the processdata property as false I get the 'illegal invocation' error, otherwise with processdata as false I literary do not receive any data in my controller. I am looking for solution to resolve this problem so I can send my CSV-file or at least the data within the file to my controller.
Other solutions than using the FormData are also welcome, for example I tried to read the CSV-file in Javascript and turn this into another object (with the jquery csv api) to send to the controller, sadly without success until now.

AWS Lambda Node.js HTTP GET Request never executed

I have the following intent. When intent is entered I want to perform a GET request to an external API. The intent is entered, however my http.get requested is not. If you look below I added a log statement within the request and it is never executed. Any ideas what the problem may be?
'BlogEntrySlugIntent': function () {
var url = 'http://jsonplaceholder.typicode.com/posts/1';
http.get( url, function( response ) {
console.log('In get request');
response.on( 'data', function( data ) {
var text = 'Data returned is: ' + data;
this.emit(':tell', 'hellloooo');
});
});
},

DropZone.js determine when successful upload

I am using Dropzone.js to upload an Excel file, of which it's contents is then imported to a table in my database.
I currently have methods in my c# which check the file being uploaded, to make sure it is valid (checks header row) and can be imported to the database.
The validation works fine, as does DropZone.js in theory. However, no matter if the file passes validation and is imported, or not, DropZone will always show the 'tick/check' mark - to notify the user that the action has completed successfully.
Here is my Dropzone:
Dropzone.options.forecastDropzone = {
init: function () {
thisDropzone = this;
this.on("success", function (file, Message) {
console.log(Message.Message)
toastr.info(Message.Message, {
timeOut: 0, tapToDismiss: true, preventDuplicates: true
});
});
},
};
HTML:
<form action="/Power/Upload" class="dropzone" id="forecastDropzone"></form>
And the 'Upload' method which is being called:
[HttpPost]
public ActionResult Upload()
{
string filename = "";
string path = "";
try
{
foreach (string fileName in Request.Files)
{
HttpPostedFileBase file = Request.Files[fileName];
path = AppDomain.CurrentDomain.BaseDirectory + "Uploads/";
filename = Path.GetFileName(Request.Files[fileName].FileName);
Request.Files[fileName].SaveAs(Path.Combine(path, filename));
ValidateExcel(path, filename);
}
}
catch
{
isSavedSuccessfully = false;
}
return Json(isSavedSuccessfully == true ? new { Message = "Successfully Saved!" } : new { Message = "Error in saving file" });
}
So the Upload method is returning a JSON object. And I want DropZone to determine whether the save/import was successful, based on a value from the JSON. Is this possible?
Many thanks

Instead of trying to parse the JSON response and handle the error client side, I would make your server responsible for this.
Specifically: have your server return something other than a successful HTTP 200 response when an upload fails. DropZone will treat an upload as failed if it receives a 4xx or 5xx response from the server.

Transfer cookies across domains in express

I've got a proxy set up in nodejs that goes to one of our backend servers for data; some of that data (such as session id) is stored as cookies. what I want to do is have the proxy get the remote cookies, push then into the header of the response to the original request, then send the response back. I'm close, but hit a snag:
app.get(/\/json\/(.+)/, getJson);
var getJson = function(req, response1) {
response1.setHeader('Content-Type', 'application/json; charset=utf-8');
var before1stWrite = true;
utils.setCookies(response1, ["floo=flum"]) // this works
var options = {
host : config.scraperUrl.replace('http://', ''),
path : '/rwd/' + req.params[0] + '?' + querystring.stringify(req.query),
method : "GET",
rejectUnauthorized : false
};
var request = https.request(options, function(response2) {
response2.setEncoding('utf8');
// utils.setCookies(response1, ["flib=flah"]) // this fails, too
response2.on('data', function(d) {
if (before1stWrite) {
console.log(response2.headers['set-cookie']); // remote's cookies
utils.setCookies(response1, ["flib=flah"]) // this fails
before1stWrite = false;
}
response1.write(d);
});
response2.on('end', function() {
response1.end()
});
});
request.end();
request.on('error', function(e) {
console.error("error occurred: " + e.message);
response1.end();
});
}
setCookies(response1, cookies) just loops thru the cookies and does
res.setHeader('Set-Cookie', cookie)
The problem is that it looks like the headers have been baked by the time the second setCookies is called; moving the method to the 'data' event handler does not help. The error I get is:
http.js:689
throw new Error('Can\'t set headers after they are sent.');
Any way to add headers to response1 that I receive from the response2?
UPDATE
I fixed the code to be sure that the attempt to write to headers of response1 was done before any other writes; it is not a fix, however.

Yes, you cannot send headers after data has started flowing. Did you try setting the header after this line?
response.setEncoding('utf8');
Also, did you consider using streams rather than transferring in chunks? http://nodejs.org/api/stream.html

You'll need to buffer the data.
Doing this is pretty much like piping:
response.on('data', function(d) {
res.write(d);
});
so you're sending the response straight away. Haven't tried it but this should work:
var data = "";
response.on('data', function(d) {
data += d;
});
response.on('end', function() {
console.log(response.headersSent);
console.log(response.headers['set-cookie']);
utils.setCookies(res, ["flib=flah"])
res.write(data);
res.end();
});
Just remember you're buffering all that data into memory, not recommended for large responses.

Questions about the scopes, node.js, and express

I really suck at understanding scopes and other things of that nature in just about every language. Right now I am building an express application that takes user input and then queries an arbitrary api and then feeds it to the console. To handle the rest api, I am using shred. I know I can use nodes built in get request, but for some reason, I could never get it to work. The user makes the following get request to my app, /query?query=. This is what I have now. I can't really describe what I'm doing so pleas read the code comments.
var http = require('http');
var Shred = require("shred");
var assert = require("assert");
exports.query = function(req, res){
//thequery is the string that is requested
var thequery = req.query.query;
var shred = new Shred();
console.log("user searched" + " " + thequery);
console.log();
//The if statement detects if the user searched a url or something else
if (thequery.indexOf("somearbitratyrestapi.com") !== -1){
console.log("a url was searched");
//find info on the url
var thedata = shred.get({
url: "http://somearbitratyrestapi.com/bla/v2" + thequery,
headers: {
Accept: "application/json"
},
on: {
// You can use response codes as events
200: function(response) {
// Shred will automatically JSON-decode response bodies that have a
// JSON Content-Type
//This is the returned json
//I want to get this json Data outside the scope of this object
console(response.content.body);
},
// Any other response means something's wrong
response: function(response) {
console.log("ohknowz");
}
}
});
//I want to be able to see that json over here. How do?
}else{
console.log("another thing was searched");
}
/*
res.render('search-results', {
result: 'you gave me a url',
title: 'you gave me a url'
});
*/
};
I tried doing this
var http = require('http');
var Shred = require("shred");
var assert = require("assert");
exports.query = function(req, res){
//thequery is the string that is requested
var thequery = req.query.query;
var shred = new Shred();
//I created a variable outside of the object
var myjson;
console.log("user searched" + " " + thequery);
console.log();
//The if statement detects if the user searched a url or something else
if (thequery.indexOf("somearbitratyrestapi.com") !== -1){
console.log("a url was searched");
//find info on the url
var thedata = shred.get({
url: "http://somearbitratyrestapi.com/bla/v2" + thequery,
headers: {
Accept: "application/json"
},
on: {
// You can use response codes as events
200: function(response) {
// Shred will automatically JSON-decode response bodies that have a
// JSON Content-Type
//This is the returned json
//I set myjson to the returned json
myjson = response.content.body
},
// Any other response means something's wrong
response: function(response) {
console.log("ohknowz");
}
}
});
//Then I try to output the json and get nothing
console.log(myjson);
}else{
console.log("another thing was searched");
}
/*
res.render('search-results', {
result: 'you gave me a url',
title: 'you gave me a url'
});
*/
};
Sorry for the bad explanation of my problem. Can someone please help or explain what is going on.

So you think you need to move data out of your nested scope, but the opposite is true. Within the nested scope where you have access to your upstream JSON response, you need to access the res object and send it though:
myjson = response.content.body
res.send(myjson);
However, long term you'll need to do some more node tutorials and focus on how to use callbacks to avoid deeply nested function scopes.