I have a script that uses JSON.parse(datastring); to parse a stringified JSON.
datastrings look like this:
{"_id":"8b8fdd243f734b27829c92e4099f70ec.d","date":1439418654920,"player":"player1","action":"capture"}
{"_id":"a3b7d70d8a074f9ba8b13368ee947f1e.d","date":1439418074476,"player":"player1","action":"capture"}
First one works just fine, but with the second one I get a weird error, that I can't find a way to solve.
undefined:2
{"_id":"a3b7d70d8a074f9ba8b13368ee947f1e.d","date": 1439418074476,"player":"pla
^
SyntaxError: Unexpected token {
at Object.parse (native)
at Socket.<anonymous> (/home/ubuntu/workspace/lib/engine.js:12:18)
at Socket.emit (events.js:95:17)
at Socket.<anonymous> (_stream_readable.js:765:14)
at Socket.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:427:10)
at emitReadable (_stream_readable.js:423:5)
at readableAddChunk (_stream_readable.js:166:9)
at Socket.Readable.push (_stream_readable.js:128:10)
at Pipe.onread (net.js:529:21)
Any ideas?
JSON.parse expects to be passed a string containing a JSON text, not a string containing multiple JSON texts.
Either parse each line (assuming you can trust that line breaks will only be between the JSON texts in your input) separately…
var json_texts = datastring.split("\n");
… or express the data in an array to start with.
[
{"_id":"8b8fdd243f734b27829c92e4099f70ec.d","date":1439418654920,"player":"player1","action":"capture"},
{"_id":"a3b7d70d8a074f9ba8b13368ee947f1e.d","date":1439418074476,"player":"player1","action":"capture"}
]
Related
Info: Start process (2:00:25 AM)
Error:
node:internal/modules/cjs/loader:317
throw e;
^
SyntaxError: Error parsing /Users//package.json: Unexpected token in JSON at position 121
at parse (<anonymous>)
at readPackage (node:internal/modules/cjs/loader:304:20)
at readPackageScope (node:internal/modules/cjs/loader:329:19)
at shouldUseESMLoader (node:internal/modules/run_main:43:15)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:72:24)
at node:internal/main/run_main_module:17:47 {
Share your package.json file. The json data in the file is invalid.
I am currently debugging some tests written with jest over typescript and I'm having a bit of a headache.
If a test, or tested class, runs Postgres SQL and there is an error in the query, I get the wrong stack trace, for example, this:
error: invalid input syntax for type integer: ""0""
at Parser.parseErrorMessage (/Users/sklivvz/src/xxx/node_modules/pg-protocol/src/parser.ts:369:69)
at Parser.handlePacket (/Users/sklivvz/src/xxx/node_modules/pg-protocol/src/parser.ts:188:21)
at Parser.parse (/Users/sklivvz/src/xxx/node_modules/pg-protocol/src/parser.ts:103:30)
at Socket.<anonymous> (/Users/sklivvz/src/xxx/node_modules/pg-protocol/src/index.ts:7:48)
at Socket.emit (node:events:365:28)
at addChunk (node:internal/streams/readable:314:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
The "error" line is very useful, however, the stack trace only tells me that the error was thrown by the pg-protocol driver. I would like to know which line within my code generated the error.
I am exactly 82.7% sure that this is due to PG's query being async.
It is incredibly time-consuming having to step debug or (gasp) console.log my way to each error when it would only be a matter of showing the correct call stack in order to make it better.
Has anyone found a way of making this developer-friendly?
Check if this is related to brianc/node-postgres issue 2484
is (there) a preferred package, extension, or method for providing more detail when you get a syntax error back from the parser?
(for instance, one that listed line number, column of the error)
for instance, right now:
error: syntax error at or near "as"
at Parser.parseErrorMessage (/home/collspec/projects/staff-portal/sprint-server/node_modules/pg-protocol/dist/parser.js:278:15)
desired behavior:
error: syntax error at or near "as", line 5, column 7
at Parser.parseErrorMessage (/home/collspec/projects/staff-portal/sprint-server/node_modules/pg-protocol/dist/parser.js:278:15)
Possible workaround from that issue:
There are a bunch of additional fields on Error objects populated by the driver.
If you log the error object you can see them. They correspond to the error fields returned by the server:
For example with the command:
SELECT foo
FROM bar
You can get an error like this:
{
length: 102,
severity: 'ERROR',
code: '42P01',
detail: undefined,
hint: undefined,
position: '17',
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'parse_relation.c',
line: '1180',
routine: 'parserOpenTable'
}
The one you want is position. It gives you the character offset in the SQL of the error.
In this example the position value of "17" refers to the start of the bar token in the SQL.
It's not always populated though as it depends on what caused the error (generally just parse errors).
I ran into a similar issue with aws-sdk for DynamoDb. This is a stacktrace I usually get from aws-sdk.
ResourceNotFoundException: Requested resource not found
at Request.extractError (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\protocol\json.js:52:27)
at Request.callListeners (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\request.js:688:14)
at Request.transition (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\state_machine.js:14:12)
at D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\request.js:690:12)
at Request.callListeners (D:\workspaces\typescript-starters\console-app\node_modules\aws-sdk\lib\sequential_executor.js:116:18)
My workaround is simply to catch async errors, and overwrite their stack traces. On the other hand, you may append Postgres stacktrace, or error message to your own errors.
async function getPersonFromDb (personId: string): Promise<DocumentClient.AttributeMap | undefined> {
const result = await documentClient.get({ // Similar to postgres.query()
TableName: 'wrong-name',
Key: { pk: personId, sk: personId }
}).promise().catch(error => {
Error.captureStackTrace(error)
throw error
})
return result.Item
}
test('Get a person from DynamoDB', async () => {
const person = await getPersonFromDb('hello')
expect(person).not.toBeUndefined()
})
// ========= new stacktrace ========
Error: Requested resource not found
at D:\workspaces\typescript-starters\console-app\test\abc.test.ts:12:13
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at getPersonFromDb (D:\workspaces\typescript-starters\console-app\test\abc.test.ts:8:20)
at Object.<anonymous> (D:\workspaces\typescript-starters\console-app\test\abc.test.ts:18:20) // my code, and where my error is thrown
I'm tryng to send a Curl request using "request-promise" of npm. The Curl that I want to send is as follows:
`curl \
-H "Content-Type: multipart/form-data" \
-F "original=#./${parent_path}" \
-F "modified=#./${version_path}" \
-o "${out_path}" \
${URI}`
My code in node is:
BIMFile.findOne({ _id: responseDB.parent_id })
.then(parent => {
parent_path = parsePath(parent.path);
version_path = parsePath(responseDB.path);
console.log("PARENT!", parent_path, version_path);
const URI =
`${protocol}://${host_img_diff}:${port_img_diff}/diff`
out_path = version_path + '.tmp.jpg';
request.post({
url: URI,
formData: {
file: fs.createReadStream(parent_path),
file: fs.createReadStream(version_path)
}
}).then((apiResponse) => {
console.log('apiUPDATEResponse', apiResponse);
})
The result is:
Unhandled rejection StatusCodeError: 400 - "<!DOCTYPE HTML PUBLIC
\"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>400 Bad
Request</title>\n<h1>Bad Request</h1>\n<p>The browser (or proxy) sent a re
quest that this server could not understand.</p>\n"
at new StatusCodeError (/backend/node_modules/request-promise-
core/lib/errors.js:32:15)
at Request.plumbing.callback (/backend/node_modules/request-promise-
core/lib/plumbing.js:104:33)
at Request.RP$callback [as _callback] (/backend/node_modules/request-
promise-core/lib/plumbing.js:46:31)
at Request.self.callback (/backend/node_modules/request/request.js:185:22)
at emitTwo (events.js:126:13)
at Request.emit (events.js:214:7)
at Request.<anonymous> (/backend/node_modules/request/request.js:1161:10)
at emitOne (events.js:116:13)
at Request.emit (events.js:211:7)
at IncomingMessage.<anonymous>
(/backend/node_modules/request/request.js:1083:12)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1055:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9)
The server returns the following message:
xxx.xx.x.xx - - [05/Apr/2019 10:00:15] "POST /diff HTTP/1.1" 400 -
As you can see the server couldn't understand the post request. Does anyone know how to add files correctly?
Use send files using formData as you're doing, but one of the issues you have is that you're setting both files in the same property, and only the last one is set.
console.log({ file: 1, file: 2 });
So if file can receive multiple files, you need to use an array
const formData = {
file: [
fs.createReadStream(parent_path),
fs.createReadStream(version_path)
]
}
If you need additional meta-data, request module provides a way too:
Pass optional meta-data with an 'options' object with style:
{value: DATA, options: OPTIONS}
Use case: for some types of
streams, you'll need to provide "file"-related information manually.
See the form-data README for more information about options:
https://github.com/form-data/form-data
I am using the Instagram API to search for a specific tag and then display the first result's image. Here I am using the JSON.parse() function parse the JSON that comes back from making a http get request to https://api.instagram.com/v1/tags/{tag}/media/recent?access_token={access token}. Here is what the request is returning me (- everything that doesn't matter). Note: I blanked out parts of the URL's, assume that they are correct.
{
"data":[
{
"images":{
"low_resolution":{
"url":"https://scontent.cdninstagram.com/t51.2885-15/s320x320/e35/--------_--------6248_1327489376_n.jpg?ig_cache_key=--------",
"width":320,
"height":320
},
"thumbnail":{
"url":"https://scontent.cdninstagram.com/t51.2885-15/s150x150/e35/--------_--------6248_1327489376_n.jpg?ig_cache_key=--------",
"width":150,
"height":150
},
"standard_resolution":{
"url":"https://scontent.cdninstagram.com/t51.2885-15/s640x640/sh0.08/e35/--------_--------6248_1327489376_n.jpg?ig_cache_key=--------",
"width":640,
"height":640
}
}
}
]
}
Here is the code I am using to make the request and parse the JSON:
request.get("https://api.instagram.com/v1/tags/" + tag + "/media/recent?access_token=" + accessToken, function(error, response, body)
{
console.log(JSON.parse(body).data.images.standard_resolution.url)
});
});
When ever I run this code it gives me this error:
/Users/pjtnt11/Documents/NodeJs/Instagram/index.js:52
console.log(JSON.parse(body).data.images.standard_resolution.url)
^
TypeError: Cannot read property 'images' of undefined
at Request._callback (/Users/pjtnt11/Documents/NodeJs/Instagram/index.js:52:67)
at Request.self.callback (/Users/pjtnt11/Documents/NodeJs/Instagram/node_modules/request/request.js:187:22)
at emitTwo (events.js:106:13)
at Request.emit (events.js:191:7)
at Request.<anonymous> (/Users/pjtnt11/Documents/NodeJs/Instagram/node_modules/request/request.js:1044:10)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at IncomingMessage.<anonymous> (/Users/pjtnt11/Documents/NodeJs/Instagram/node_modules/request/request.js:965:12)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
I thought that maybe it could have been because console.log() was running before JSON.parse() but JSON.parse blocks the code from running anything else. I also check if it had a callback or a promise that I could use but I didn't see anything.
Now I have no idea what is going on. Does anyone know what could be causing this error and how I could fix it?
EDIT:
Okay, so I found out that data is a data array, so I need to change my code to be data[0] the problem now is I get the following error when I run the code:
/Users/pjtnt11/Documents/NodeJs/Instagram/index.js:52
console.log(JSON.parse(body).data[0].images.standard_resolution.url)
^
TypeError: Cannot read property '0' of undefined
at Request._callback (/Users/pjtnt11/Documents/NodeJs/Instagram/index.js:52:67)
at Request.self.callback (/Users/pjtnt11/Documents/NodeJs/Instagram/node_modules/request/request.js:187:22)
at emitTwo (events.js:106:13)
at Request.emit (events.js:191:7)
at Request.<anonymous> (/Users/pjtnt11/Documents/NodeJs/Instagram/node_modules/request/request.js:1044:10)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at IncomingMessage.<anonymous> (/Users/pjtnt11/Documents/NodeJs/Instagram/node_modules/request/request.js:965:12)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
data is an Array, so you need to access the first element...
JSON.parse(body).data[0].images
see following working snippet...
var someObject = '{ \
"data":[ \
{ \
"images":{ \
"low_resolution":{ \
"url":"https://scontent.cdninstagram.com/t51.2885-15/s320x320/e35/--------_--------6248_1327489376_n.jpg?ig_cache_key=--------", \
"width":320, \
"height":320 \
}, \
"thumbnail":{ \
"url":"https://scontent.cdninstagram.com/t51.2885-15/s150x150/e35/--------_--------6248_1327489376_n.jpg?ig_cache_key=--------", \
"width":150, \
"height":150 \
}, \
"standard_resolution":{ \
"url":"https://scontent.cdninstagram.com/t51.2885-15/s640x640/sh0.08/e35/--------_--------6248_1327489376_n.jpg?ig_cache_key=--------", \
"width":640, \
"height":640 \
} \
} \
} \
] \
}';
console.log(JSON.parse(someObject).data[0].images.low_resolution.url);
I am trying to store a JSON POST request into a postgresql database. I have defined the profile model to store id as an int, and match_caps as JSON.
profile_model.js
db.define('profile', {
id : Number,
match_caps : Object
});
Once a POST request has been made I would like to store into postgresql.
profile.js
req.on('data', function(chunk){
data += chunk;
});
req.on('end', function(chunk){
var obj = JSON.parse(data);
db.models.profile.create([{
id : obj.id,
match_caps : obj.match_caps
}], function(err) {//error}
});
I am POSTing the following:
{"id":101, "match_caps":{"x":1, "y":2}}
And I get the following error:
error: invalid input syntax for type json
at Connection.parseE (/home/dev/node_modules/pg/lib/connection.js:526:11)
at Connection.parseMessage (/home/dev/node_modules/pg/lib/connection.js:356:17)
at Socket.<anonymous> (/home/dev/node_modules/pg/lib/connection.js:105:22)
at Socket.EventEmitter.emit (events.js:95:17)
at Socket.<anonymous> (_stream_readable.js:746:14)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable_ (_stream_readable.js:408:10)
at emitReadable (_stream_readable.js:404:5)
at readableAddChunk (_stream_readable.js:165:9)
at Socket.Readable.push (_stream_readable.js:127:10)
I think I am POSTing the JSON correctly. I have tested POSTing the id alone, it was successful.
I have console.log(data), and received the output
{"id":101, "match_caps":{"x":1, "y":2}}
I have tried to JSON.stringify(obj.match_caps). I recieve the same error.
I am using node ORM to define my models
I am unsure if the POST syntax for match_caps is correct... or if there is a bug in orm.