I have extracted the schema of my BigQuery table from the "INFORMATION_SCHEMA" table. I get the list of all the columns in the table in a proper Javascript Object format except of the "Struct" and "Array" data types. I need a clean way to convert the "Struct" and "Array" string into a javascript object.
I am working with NodeJS v11.2 I have written a small regex which extracts the following. But it doesn't seem right to split the string and iterate through each word until I get the output. I need a cleaner way to solve this.
let structString = "STRUCT<name STRING, email STRING, time_sec INT64, tz_offset INT64, date STRUCT<seconds INT64, nanos INT64>>";
let _structSchema = structString.match(/STRUCT<([^)]+)>/)[1];
console.log(_structSchema); // name STRING, email STRING, time_sec INT64, tz_offset INT64, date STRUCT<seconds INT64, nanos INT64>
I need to write a recursive function which will parse though the string and give me the output in following manner.
{
"name": "STRING",
"email": "STRING",
"time_sec": "INT64",
"tz_offset": "INT64",
"date": {
"seconds": "INT64",
"nanos": "INT64"
}
}
The function should run irrespective of the depth/hierarchy of the nested structs/arrays.
Using a regex can be good to tokenise the input string, but you'll need more logic to perform the actual parsing.
Here is how you could do it:
function parse(structString) {
let tokenizer = /([a-z_]\w*)|\S|$/gi;
function next(identifier, expected) {
let match = tokenizer.exec(structString);
function error(expected) {
throw `Expected ${expected} but got ${match[0]} at ${match.index}`;
}
match[0] = match[0] || "<EOF>";
if (identifier && !match[1]) error(identifier);
if (expected && !expected.includes(match[0])) error(expected.join(" or "))
return match[0];
}
function getType() {
let fieldType = next("type identifier or STRUCT or ARRAY");
if (fieldType === "STRUCT") {
next(null, ["<"]);
fieldType = {};
do {
fieldType[next("field identifier")] = getType();
} while (next(null, [",", ">"]) === ",");
} else if (fieldType === "ARRAY") {
next(null, ["<"]);
fieldType = [getType()];
next(null, [">"]);
}
return fieldType;
}
let result = getType();
next(null, ["<EOF>"]);
return result;
}
// Sample input & call
let structString = "STRUCT<name STRING, email STRING, time_sec INT64, tz_offset INT64, date STRUCT<seconds INT64, nanos INT64>, phones ARRAY<STRING>>";
let obj = parse(structString);
console.log(obj);
If you can access Google Cloud Console from your environment, consider running something like: bq show --format=pretty project:dataset.table or bq show --format=prettyjson project:dataset.table. You would still need to parse the results for your purposes, but the nesting is already done for you.
Related
(ultra-Junior dev here)
I have these code.. using typeof to validate if the client puts a string or not.. Otherwise will not proced with the task and will send an error 400... The things is that with my dev-Front colegue.. He is putting a string in the username and he is having a 400 error.
-- if there is another method to check is the input is a string or not.??
its a big validator checking at the same time other type of inputs.. so maybe your answer could impact all the code.
Thanks
could someone explain why
function tappingValidator(req, res, next) {
if (typeof req.body.username !== "string") {
const msg = JSON.stringify({ msg: "username has to be a string" });
const err = new Error(msg);
err.statusCode = 400;
return next(err);
}};
Any sequence of characters, including a sequence of pure digits, is a string. It's context and/or syntax that makes the difference between what's a number and what's a string.
const a = 123;
const b = '123';
console.log(typeof a); // displays "number"
console.log(typeof b); // display "string"
req.body.username is probably a string data type (something coming from a web form) no matter what the content of that string is.
If you want to make sure the string doesn't start with a digit, this regular expression test will work:
if (/^[\d]/.test(req.body.username)) {
const msg = JSON.stringify({ msg: "username shouldn't start with a digit" });
There are other tests if you merely wanted to rule out something purely numeric, but that is allowed to start with a digit.
Add a function to your codebase
function isString(arg) {
return typeof arg === 'string' || arg instanceof String
}
and then use it in if statement like this:
if(!isString(req.body.username)) {
// it is not a string
// ...
}
I am using indexOf to see if an email contains anything other than a particular text.
For example, I want to check if an email DOES NOT include "usa" after the # symbol, and display an error message.
I was first splitting the text and removing everything before the # symbol:
var validateemailaddress = regcriteria.email.split('#').pop();
Then, I check if the text doesn't include "usa":
if(validateemailaddress.indexOf('usa')){
$('#emailError').show();
}
Something with the above check doesn't seem right. It works - I can enter an email, and if it does not include 'usa', then the error message will show.
Regardless, when I add an additional check, like if the email does not include "can", then the error message shows no matter what.
As follows:
if(validateemailaddress.indexOf('usa') || validateemailaddress.indexOf('can')){
$('#emailError').show();
}
As stated, using the above, the error message will show regardless if the email includes the text or not.
All I want to do is check if the email includes 'usa' or 'can', and if it doesn't, then show the error message.
How can I make this work?
Here is a simple JavaScript function to check if an email address contains 'usa' or 'can'.
function emailValid(email, words) {
// Get the position of # [indexOfAt = 3]
let indexOfAt = email.indexOf('#');
// Get the string after # [strAfterAt = domain.usa]
let strAfterAt = email.substring(indexOfAt + 1);
for (let index in words) {
// Check if the string contains one of the words from words array
if (strAfterAt.includes(words[index])) {
return true;
}
}
// If the email does not contain any word of the words array
// it is an invalid email
return false;
}
let words = ['usa', 'can'];
if (!emailValid('abc#domain.usa', words)) {
console.log("Invalid Email!");
// Here you can show the error message
} else {
console.log("Valid Email!");
}
You can do something like that, using includes:
const validateEmailAdress = (email) => {
const splittedEmail = email.split('#').pop();
return (splittedEmail.includes('usa') || splittedEmail.includes('can'))
}
console.log("Includes usa: ", validateEmailAdress("something#gmail.usa"))
console.log("Includes can: ", validateEmailAdress("something#gmail.can"))
console.log("Does not includes: ", validateEmailAdress("something#gmail.com"))
There are several ways to check, if a string contains/does not contain a substring.
String.prototype.includes
'String'.includes(searchString); // returns true/false
String.prototype.indexOf
// returns values from -1 to last postion of string.
'String'.indexOf(searchString);
// In combination with ~ this can work similar to includes()
// for strings up to 2^31-1 byte length
// returns 0 if string is not found and -pos if found.
~'String'.indexOf(searchString);
With the help of Regular Expressions:
// substring must be escaped to return valid results
new RegExp(escapedSearchString).test('String'); // returns true/false if the search string is found
'String'.match(escapedSearchString); // returns null or an array if found
So overall you can use allmost all methods like:
if ('String'.function(searchString)) {
// 'String' includes search String
} else {
// 'String' does not include search String
}
Or in case of indexOf:
if ('String'.indexOf(searchString) > -1) {
// 'String' includes search String
} else {
// 'String' does not include search String
}
// OR
if (~'String'.indexOf(searchString)) {
// 'String' includes search String
} else {
// 'String' does not include search String
}
I believe this regular expression match is what you're looking for
System.out.println(myString.matches("(.)#(.)usa(.*)"));
The output of this program is undefined instead of string name.
I am taking a date as input to the program and comparing the date with the existing dates of president array. In case if the date matches then i want to return the president name for that particular date
process.stdin.resume();
process.stdin.setEncoding('utf8');
var stdin = '';
process.stdin.on('data', function (chunk) {
//printing the value returned by presidentOnDate function
console.log(JSON.stringify(presidentOnDate(chunk)));
});
//This is presidents array
var presidents = [
{"number":32,"president":"Franklin D. Roosevelt","took_office":"1933-03-04","left_office":"1945-04-12"},
{"number":33,"president":"Harry S. Truman","took_office":"1945-04-12","left_office":"1953-01-20"},
{"number":34,"president":"Dwight D. Eisenhower","took_office":"1953-01-20","left_office":"1961-01-20"},
{"number":35,"president":"John F. Kennedy","took_office":"1961-01-20","left_office":"1963-11-22"},
{"number":36,"president":"Lyndon B. Johnson","took_office":"1963-11-22","left_office":"1969-01-20"},
{"number":37,"president":"Richard Nixon","took_office":"1969-01-20","left_office":"1974-08-09"},
{"number":38,"president":"Gerald Ford","took_office":"1974-08-09","left_office":"1977-01-20"},
{"number":39,"president":"Jimmy Carter","took_office":"1977-01-20","left_office":"1981-01-20"},
{"number":40,"president":"Ronald Reagan","took_office":"1981-01-20","left_office":"1989-01-20"},
{"number":41,"president":"George H. W. Bush","took_office":"1989-01-20","left_office":"1993-01-20"},
{"number":42,"president":"Bill Clinton","took_office":"1993-01-20","left_office":"2001-01-20"},
{"number":43,"president":"George W. Bush","took_office":"2001-01-20","left_office":"2009-01-20"},
{"number":44,"president":"Barack Obama","took_office":"2009-01-20","left_office":"2017-01-20"}
];
//PresidentOnDate function which should return a president name based on input date
function presidentOnDate(date) {
var output="";
for(var i=0;i<presidents.length;i++){
//console.log(presidents[i].took_office);
if((presidents[i].took_office)==date){
output+=presidents[i].president;
}
}
return output;
}
I think the problem is you are passing in a buffer instead of a string.
Try changing the chunk buffer to a string before passing it to presidentOnDate.
So instead of presidentOnDate(chunk) try presidentOnDate(chunk.toString())
Try this function it's working fine.
problem you facing when you take input it take \r\n also so when you compare both you get false that y output showing null.
EX:input string: "2009-01-20\r\n" compare with : took_office: "2009-01-20" => result false
EX: input string with trim: "2009-01-20" compare with : took_office: "2009-01-20" => result True
change only : (presidents[i].took_office) ==date.trim()
function presidentOnDate(date) {
var output="";
for(var i=0;i<presidents.length;i++){
if((presidents[i].took_office) ==date.trim()){
output+= presidents[i].president;
}
}
return output;
}
I have extracted a string from stored procedure in Microsoft SQL Server 2008 R2,
using this query:
EXEC sp_helptext 'MyStoredProcedureName'; I need to split this string into arrays or sub strings starting from the ending parenthesize " ] " and ending with the word "As".
I have to save ALL fields starting with "#" in (fieldsArray) and the types of these fields -After the space- in (typeArray) and then output them to a file in node.js with this format:
InvoiceNo: {
type: DataType.String(255),
},
Here is the extracted string:
CREATE PROCEDURE [dbo].[MyStoredProcedureName]
#InvoiceNo int
,#TransDate datetime
,#CustomerID bigint
,#CurrencyID bigint
,#SalesInvoiceTypeID bigint
,#DiscountAmount nvarchar(50)
,#DetailXml ntext
,#TotalAll float
,#TotalBefore float
,#TaxAmount float
,#OtherExpenses float
,#OutVouchersNo nvarchar(1000)
,#Notes nvarchar(1000)
,#TotalWiegts float
,#VoucherDefID bigint
,#SalesmanID bigint
,#IsSale bit
AS
BEGIN TRANSACTION
Edit: I used another query instead of the mentioned above. Now I have an object containing the result of the query which is the Names of the fields each in separate line. What I need to do now is to separate this object to a string array so that I can deal with each name separately.
the object "Names" contain the result of the query that I used:
Names = await sequelize.query(namesQuery);
And this is the output of console.log(Names);
[ [ { '': 'InvoiceNo' },
{ '': 'TransDate' },
{ '': 'CustomerID' },
{ '': 'CurrencyID' },
{ '': 'SalesInvoiceTypeID' },
{ '': 'DiscountAmount' },
I tried Names.split but I got the error "Names.split is not a function"
You could query the Sys.Parameters table instead:
SELECT Substring(Parameters.Name,2,255) + ': { type: DataType.' +
CASE types.Name
WHEN 'varchar' THEN 'String(' + CAST(Parameters.Max_Length As Varchar)
WHEN 'int' THEN 'Int'
ELSE 'COMPLETE THE REST OF THIS yourself.....'
END + '),},'
FROM Sys.Parameters
INNER JOIN sys.procedures on parameters.object_id = procedures.object_id
INNER JOIN sys.types on parameters.system_type_id = types.system_type_id AND parameters.user_type_id = types.user_type_id
Where procedures.name = 'MyStoredProcedureName'
If you consider that the symbol # is only used to identify the beginning of a field name, you can use regular expressions as follows:
let result = {}
str.match(/#\w+ [a-z]+/g)
.map(s => s.match(/#(\w+) ([a-z]+)/))
.forEach(r => {
let fieldName = r[1]
let type = r[2]
result[fieldName] = { type }
})
console.log(JSON.stringify(result, undefined, 2))
This will output
{
"InvoiceNo": {
"type": "int"
},
"TransDate": {
"type": "datetime"
},
...
}
I guess the same could be achieved with only regex, but the code is easier to understand this way:
the first match captures all the #FieldName type strings
the second match captures for each of these FieldName and type
the last one fills the object result with members of name FieldName and content { type: "type" }
the last command formats everything and outputs to the console
I have this Json string that i use for google chart visualization that needs to be in this exact format and i need to replace every value of "v" that is a number to its numeric value( the value without the ""). I should do some javascript replace function, but i couldn't find a way to move around the json object. Here is and example json string that i should modify :
{"cols":[
{"id":"r","label":"Reason","type":"string"},
{"id":"m","label":"Minutes","type":"number"}
],
"rows":[
{"c":[
{"v":"Flour - Blower","f":"Flour - Blower"},
{"v":"7","f":"7"}]},
{"c":[
{"v":"Whole Line - d","f":"Whole Line - d"},
{"v":"4","f":"4"}]},
{"c":[
{"v":"Flour - Pipework","f":"Flour - Pipework"},
{"v":"3","f":"3"}]},
{"c":[
{"v":"Horseshoe - Belt","f":"Horseshoe - Belt"},
{"v":"1","f":"1"}]}
],
"p":null
}
probably i should do something like :
var jsonStr = ...;
for (i in jsonStr.rows) {
for(j in jsonStr[i].c)
{
if (parseInt(jsonStr[i].c[j].v) != 'NaN') {
jsonStr.rows[i].c[j].v = parseInt(jsonStr.rows[i].c[j].v);
}
}
Since JSON is effectively a string, why not put the entire string through a global string.replace:
jsonStr = JSON.stringify(jsonStr);
jsonStr = jsonStr.replace(/"v":"(\d+)"/g, '"v":$1');
Jsfiddle demo
Well, the parsing seems okay to me. It's probably not working because you can't really check if a string contains a number or not by comparing something with NaN
This is because even NaN === NaN, famously, returns false.
I'd suggest that you use the isNaN method (which does use parseInt internally). So, something like this ought to work
for (i in jsonStr.rows) {
for(j in jsonStr[i].c)
{
if (!isNaN(jsonStr[i].c[j].v)) {
jsonStr.rows[i].c[j].v = parseInt(jsonStr.rows[i].c[j].v);
}
}
A function that returns string if isNaN else a number:
function convertNumberToInteger(val) {
if (isNaN(val)) {
return val;
} else {
return parseInt(val);
}
}
Usage:
convertNumberToInteger("sasdfasdf");
Output: "sasdfasdf"
convertNumberToInteger("3");
Output: 3
And if you really want to parse it you can do a forEach on the JSON object