We Keep Coding

How to Combine/merge two arrays in similar index of post request in node.js/express and convert in object to insert mongoose model

I have a page that gets two arrays of input elements and sends them by post request to my app.js:
<input type="text" name="titleAttr[]" > </input>
<input type="text" name="descriptionAttr[]"> </input>
I created a Schema that receives an array with 2 fields, titleAttr and descriptionAttr, which correspond to the <input> elements above:
const mySchema = mongoose.Schema({
titulo: String,
attrs: [{
titleAttr: String,
descriptionAttr: String
}]
});
I can insert the data manually and it works:
MyModel.bulkWrite([ { insertOne : { document: {
title : "TEST",
attrs: [
{titleAttr : "test 1", descriptionAttr: "This is a test 1"},
{titleAttr: "test 2", descriptionAttr: "This is another test"}
]
} } }
]);
Here is a screenshot of the form .
When the post request is sent from my form and I print it in app.js, I get these results:
console.log(req.body.titleAttr); //result: [ 'test 1', 'test 2' ]
console.log(req.body.descriptionAttr);// result: [ 'This is a test 1', 'This is another test' ]
This code doesn't work:
ConceitoHTML.bulkWrite([ { insertOne : { document: {
titulo : req.body.title,
attrs: [
{
titleAttr: req.body.titleAttr,
descriptionAttr: req.body.descriptionAttr
}
]
} } } ]);
I want to merge my two arrays and insert into MongoDB as an array of objects. How do I generate an array like the following?
const myArray = [
{
titleAttr: req.body.titleAttr[0],
descriptionAttr: req.body.descriptionAttr[0]
},
{
titleAttr: req.body.titleAttr[1],
descriptionAttr: req.body.descriptionAttr[1]
}
]

You can do it with this code below 👇 to get an array as expected what you want:
const {titleAttr, descriptionAttr} = req.body;
const myArray = [];
// check the length first, make sure it's same
if(titleAttr.length === descriptionAttr.length) {
for(let i=0; i<titleAttr.length; i++) {
myArray.push({ titleAttr: titleAttr[i], descriptionAttr: descriptionAttr[i] })
}
}
console.log(myArray); // this is the array result
I hope it's can help you.

Scraping Javascript variables into Python

I want to scrape the following data from http://maps.latimes.com/neighborhoods/population/density/neighborhood/list/:
var hoodFeatures = {
type: "FeatureCollection",
features: [{
type: "Feature",
properties: {
name: "Koreatown",
slug: "koreatown",
url: "/neighborhoods/neighborhood/koreatown/",
has_statistics: true,
label: 'Rank: 1<br>Population per Sqmi: 42,611',
population: "115,070",
stratum: "high"
},
geometry: { "type": "MultiPolygon", "coordinates": [ [ [ [ -118.286908, 34.076510 ], [ -118.289208, 34.052511 ], [ -118.315909, 34.052611 ], [ -118.323009, 34.054810 ], [ -118.319309, 34.061910 ], [ -118.314093, 34.062362 ], [ -118.313709, 34.076310 ], [ -118.286908, 34.076510 ] ] ] ] }
},
From the above html, I want to take each of:
name
population per sqmi
population
geometry
and turn it into a data frame by name
So far I've tried
import requests
import json
from bs4 import BeautifulSoup
response_obj = requests.get('http://maps.latimes.com/neighborhoods/population/density/neighborhood/list/').text
soup = BeautifulSoup(response_obj,'lxml')
The object has the script info, but I don't understand how to use the json module as advised in this thread:
Parsing variable data out of a javascript tag using python
json_text = '{%s}' % (soup.partition('{')[2].rpartition('}')[0],)
value = json.loads(json_text)
value
I get this error
TypeError Traceback (most recent call last)
<ipython-input-12-37c4c0188ed0> in <module>
1 #Splits the text on the first bracket and last bracket of the javascript into JSON format
----> 2 json_text = '{%s}' % (soup.partition('{')[2].rpartition('}')[0],)
3 value = json.loads(json_text)
4 value
5 #import pprint
TypeError: 'NoneType' object is not callable
Any suggestions? Thanks

I'm not really sure how to do that with the beautiful soup, yet another option might be to likely design an expression and extract our desired values:
(?:name|population per sqmi|population)\s*:\s*"?(.*?)\s*["']|(?:geometry)\s*:\s*({.*})
Demo
Test
import re
regex = r"(?:name|population per sqmi|population)\s*:\s*\"?(.*?)\s*[\"']|(?:geometry)\s*:\s*({.*})"
test_str = ("var hoodFeatures = {\n"
" type: \"FeatureCollection\",\n"
" features: [{\n"
" type: \"Feature\",\n"
" properties: {\n"
" name: \"Koreatown\",\n"
" slug: \"koreatown\",\n"
" url: \"/neighborhoods/neighborhood/koreatown/\",\n"
" has_statistics: true,\n"
" label: 'Rank: 1<br>Population per Sqmi: 42,611',\n"
" population: \"115,070\",\n"
" stratum: \"high\"\n"
" },\n"
" geometry: { \"type\": \"MultiPolygon\", \"coordinates\": [ [ [ [ -118.286908, 34.076510 ], [ -118.289208, 34.052511 ], [ -118.315909, 34.052611 ], [ -118.323009, 34.054810 ], [ -118.319309, 34.061910 ], [ -118.314093, 34.062362 ], [ -118.313709, 34.076310 ], [ -118.286908, 34.076510 ] ] ] ] }\n"
" },\n")
matches = re.finditer(regex, test_str, re.MULTILINE | re.IGNORECASE)
for matchNum, match in enumerate(matches, start=1):
print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
for groupNum in range(0, len(match.groups())):
groupNum = groupNum + 1
print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))

You can't really use json.loads because hoodFeatures object is not really a json. In a proper json, every key is surrounded with double quotes "
You could try adding quotes around keys manually (using regular expressions).
Another option is using Selenium to execute that JS and get the JSON.stringify output of it.
Answer using manual cleanup:
This one cleans up JS code and turns it into a JSON that can be parsed properly. However keep in mind that it is by no means robust, and may break at any sight of unexpected input.
import json
import re
js = '''
var hoodFeatures = {
type: "FeatureCollection",
features: [
{
type: "Feature",
properties: {
name: "Beverlywood",
slug: "beverlywood",
url: "/neighborhoods/neighborhood/beverlywood/",
has_statistics: true,
label: 'Rank: 131<br>Population per Sqmi: 7,654',
population: "6,080",
stratum: "middle"
},
geometry: { }
}]
}
'''
if __name__ == '__main__':
unprefixed = js.split('{', maxsplit=1)[1]
unsuffixed = unprefixed.rsplit('}', maxsplit=1)[0]
quotes_replaced = unsuffixed.replace('\'', '"')
rebraced = f'{{{quotes_replaced}}}'
keys_quoted = []
for line in rebraced.splitlines():
line = re.sub('^\s+([^:]+):', '"\\1":', line)
keys_quoted.append(line)
json_raw = '\n'.join(keys_quoted)
# print(json_raw)
parsed = json.loads(json_raw)
for feat in parsed['features']:
props = feat['properties']
name, pop = props['name'], int(props['population'].replace(',', ''))
geo = feat['geometry']
pop_per_sqm = re.findall('per Sqmi: ([\d,]+)', props['label'])[0].replace(',', '')
pop_per_sqm = int(pop_per_sqm)
print(name, pop, pop_per_sqm, geo)

ExtJs - How to create instance of Ext.data.Model dynamically using json file containing Model configs

I am using ExtJs Version 5.0.0.970. I have a Json file which has configs related to a model. Following is the structure of the Json file -
{
"model": {
"requires": ["Ext.data.Field"],
"fields": [
{
"name" : "carName",
"mapping" : "carName"
},
{
"name" : "carPrice.currency",
"mapping" : "carPrice.currency"
}
]
}
}
Using above Json i am trying to create a Model dynamically in the following way -
//gridModelJsonData variable contains the content of json.model object
Ext.create("Ext.data.Model",gridModelJsonData);
As soon as the above line of code executes i am getting following error in the console -
Model.js?_dc=1471922882661:2367 Uncaught TypeError: Cannot read property 'length' of undefined
Ext.data.Model#rankFields # Model.js?_dc=1471922882661:2367
Ext.data.Model#makeInitializeFn # Model.js?_dc=1471922882661:2824
Ext.data.Model#constructor # Model.js?_dc=1471922882661:378
Ext.data.Model # Class.js?_dc=1471922882661:29
Ext.create1 # VM11599:3create # ClassManager.js?_dc=1471922882661:1413
Please let me know how to solve this problem!
Thanks!

You should define your model's fields(i.e. the model's metadata) and then create the model:
var jsonData = {
"model": {
"fields": [
{
"name" : "carName",
"mapping" : "carName"
},
{
"name" : "carPrice.currency",
"mapping" : "carPrice.currency"
}
]
}
};
Ext.define('CarModel', { //Defining a model, which is like an object
extend : 'Ext.data.Model',
fields : [
{name: 'name', type: 'string'},
{name: 'mapping', type: 'string'}
]
});
var gridModelJsonData = jsonData.model;
var carModels = [];
for(var i =0; i < gridModelJsonData.fields.length; i++) {
carModels.push(Ext.create('CarModel', gridModelJsonData.fields[i]));
}
for(var i =0; i < carModels.length; i++) {
console.log('CarModel[' + i + '].name=' + carModels[i].get('name'));
console.log('CarModel[' + i + '].mapping=' + carModels[i].get('mapping'));
}
. See this fiddle

Trying to get information from JSON file dynamically in a JavaScript function

I found this function
<script>
$(document).ready(function() {
$('#container')
.TidyTable({
columnTitles : ['Column A','Column B','Column C','Column D','Column E'],
columnValues : [
['1','1A','1B','1C','1D','1E'],
['2','2A','2B','2C','2D','2E'],
['3','3A','3B','3C','3D','3E'],
['4','4A','4B','4C','4D','4E'],
['5','5A','5B','5C','5D','5E']
]
});
});
</script>
And it gives me a table where I can click on the column titles and it sorts the respective column. Furthermore I have a json file with the following structure :
{ "results": [
{
"information1": "Some info1",
"information2": "Some info2"
},
{
"information1": "Some info1",
"information2": "Some info2"
}
]}
My question is:
Can I dynamically parse this JSON file into the above function? So I imagine I have something like this:
<script>
$(document).ready(function() {
$('#container')
.TidyTable({
columnTitles : ['Information 1','Information 2'],
columnValues : [ //Something dynamic has to happen here for all entries
['information1', 'information2']
]
});
});
</script>
I am not well traversed in this area so being specific would help me a lot. Thanks.

You should be able to use jQuery's map method.
var data = { "results": [
{
"information1": "Some info1",
"information2": "Some info2"
},
{
"information1": "Some info1",
"information2": "Some info2"
}
]}
// assumes keys are the same for every object in the array
var headers = $.map(data.results[0], function(v, k) { return k })
var values = [];
$.map(data.results, function(row) {
values.push(
$.map(row, function(v, k) {
return v;
})
);
});
console.log( headers ) // ['information1', 'information2']
console.log( values ) // [['Some info1', 'Some info2'], ['Some info1', 'Some info2']]
Then it's just a matter of using your new data format in your function: $('#container').TidyTable({ columnTitles : headers, columnValues : values});
fiddle here: http://jsfiddle.net/4ftoxw16/
Documentation on .map(): http://api.jquery.com/jquery.map/

Loading Flexigrid for jQuery with JSON String

I am trying to load the Flexigrid by using a JSON String which is returned by a WCF Service.
My Service has a public string GetContacts(string CustomerID) method and it returns a Json string.
That JSON string is created from a List object by using
System.Web.Script.Serialization.JavaScriptSerializer class. So, my aim is to bind the JSON string to the my Flexigrid as objects. I convert the web service result to objects using
var customer = eval("("+result+")");
The result is the JSON string being returned from service. Is there any way to bind customer objects to Flexigrid?

Flexigrid requires a format as follows in json
EDIT Thanks to EAMann for the format update.
total: (no of rec)
page : (page no)
rows : [{cell: [ (col1 value) , (col2 value) ,.. ] },
{cell: [ (col1 value) , (col2 value) ,.. ] }]
in order to bind the data to the grid i prefer sending the data across the wire and then formatting it on the client, but thats just me heres an example
function formatCustomerResults(Customers) {
var rows = Array();
for (i = 0; i < Customers.length; i++) {
var item = Customers[i];
//Do something here with the link
var link = "alert('opening item " + item.DealGuid + "');"
rows.push({
cell: [item.DealId,
item.Created,
item.CurrentStatus,
item.LastNote,
'<a href="javascript:void(0);" onclick="' + link + '" >view</a>'
]
});
}
return {
total: Customers.length,
page: 1,
rows: rows
};
}
and then all you need is
$("#FlexTable").flexAddData(formatCustomerResults(eval(data)));
ps this last bit is jquery syntax

almog.ori's answer is almost perfect. In fact, that's just about how I had built things before trying to Google the solution. One exception, though.
The JSON object should be:
total: (no of rec),
page : (page no),
rows : [{cell: [ (col1 value) , (col2 value) ,.. ] },
{cell: [ (col1 value) , (col2 value) ,.. ] }]
If you neglect the array format of the rows element, you'll end up choking Flexigrid and throwing all sorts of errors. But I've verified that this works flawlessly as long as you remember which parts of the script take JSON objects and which parts take arrays of JSON objects.

This is an older post but I thought I would add another way to use the excellent script provided by almog.ori.
The OP said that his data was being returned from a WCF service. If you mark the operation contract body style as bare you can use the preProcess property to add your formatCustomerResults (or other function) function to initially load the grid.
Like this:
$("#gridContainer").flexigrid({
url: '/YourService.svc/..',
method: 'GET',
dataType: 'json',
preProcess: formatCustomerResults,
...
});
function formatCustomerResults(data){
...
Hope this helps someone.

Make sure also that you are using the correct HTTP method, default is POST
To use GET:
$("#gridContainer").flexigrid({
url: '/json/files.json',
method: 'GET',
dataType: 'json',
...

preProcess solution by nameEqualsPNamePrubeGoldberg works perfect.
This is how my custom function for preProcess looks like.
var rows = Array();
$.each(data.rows,function(i,row){
rows.push({id:row.val1, cell:[row.val2,row.val3]});
});
return {
total:data.total,
page:data.page,
rows:rows
};

I recommend you to follow this sample to parse your JSON code and make requests to server:
Step 1: Parse using a function
function parsedForm(json)
{
var h = "";
if (json.post.val1)
h += "<b>Value 1</b>: " + json.post.val1 + "<br />";
h += "<b>Value 2</b>: " + json.post.val2 + "<br />";
h += "<b>Value 3</b>: " + json.post.val3 + "<br />";
if (json.post.val4)
h += "<b>Value 4</b>: " + json.post.val4 + "<br />";
$('#fdata').empty().html(h);
$('.formdata').slideDown();
return json;
}
Step 2: The flexigrid view code
$("#flex1").flexigrid({
url: 'post2.php',
dataType: 'json',
colModel : [
{display: 'ISO', name : 'iso', width : 40, sortable : true, align: 'center'},
{display: 'Name', name : 'name', width : 180, sortable : true, align: 'left'},
{display: 'Printable Name', name : 'printable_name', width : 120, sortable : true, align: 'left'},
{display: 'ISO3', name : 'iso3', width : 130, sortable : true, align: 'left', hide: true},
{display: 'Number Code', name : 'numcode', width : 80, sortable : true, align: 'right'}
],
searchitems : [
{display: 'ISO', name : 'iso'},
{display: 'Name', name : 'name', isdefault: true}
],
sortname: "iso",
sortorder: "asc",
usepager: true,
title: 'Countries',
useRp: true,
rp: 15,
showTableToggleBtn: true,
width: 700,
onSubmit: addFormData,
preProcess: parsedForm,
height: 200
});
Step 3: Additonally, you can validate or serialize the data to request server
function addFormData(){
//passing a form object to serializeArray will get the valid data from all the objects, but, if the you pass a non-form object, you have to specify the input elements that the data will come from
var dt = $('#sform').serializeArray();
$("#flex1").flexOptions({params: dt});
return true;
}
$('#sform').submit(function (){
$('#flex1').flexOptions({newp: 1}).flexReload();
return false;
});
I hope it will help!

Make sure you have the dataType option set to json.
$('#gridContainer').flexigrid({
singleSelect: true,
showToggleBtn: false,
dataType: 'json'
});

I believe the latest flex code broke the solution using preProcess.
addData: function (data) { //parse data
if (p.dataType == 'json') {
data = $.extend({rows: [], page: 0, total: 0}, data);
}
if (p.preProcess) {
data = p.preProcess(data);
}
You need to flip it so that the preProcess if comes before the type JSON if. Otherwise the function listed as an answer does not work properly.

It's old, I know... But here is an example of json that works:
{
"total": 5,
"page": "1",
"rows": [
{"cell": [1, "asd", "dsa", "2013-07-30"]},
{"cell": [2, "asd", "dsa", "2013-07-30"]},
{"cell": [3, "asd", "dsa", "2013-07-30"]},
{"cell": [4, "asd", "dsa", "2013-07-30"]},
{"cell": [5, "asd", "dsa", "2013-07-30"]}
]
}
(5 results in total; first page (they are NOT zero-based); 5 lines of data, each containing { ID, "asd", "dsa", "a date" } )

Try to make total your first element in you JSON string like this.
`{"total" : 2,"page":1,"rows":[ {"cell" : ["226 CLAVEN LN", "312"]},{"cell" : ["1377 FAIRFAX PIKE","280"]}]}`