I have csv files that are generated, and I am trying to load them into d3 to graph them. The column names are based on the data, so I essentially can't know them in advance. With testing, I am able to load this data and graph it all well and nice if I know the names of the columns...but I don't in my use case.
How can I handle this in d3? I can't seem to find anything to help/reference this online or in the documentation. I can see when I log to the console data[0] from d3.csv that there are two columns and the values read for them, but I don't know how to refer arbitrarily to column 1 or 2 of the data without knowing the name of the column ahead of time. I'd like to avoid that in general, knowing my timestamps are in column 1 and my data is in column 2, if that makes sense.
Edit, my answer uses d3.entries to help learn the name of the unknown column, and then continues to access all objects with that index:
d3.csv("export.csv", function(error, data) {
var mappedArray = d3.entries(data[0]);
var valueKey = mappedArray[1].key;
data.forEach(function(d) {
...
d.value = d[valueKey];
}
}
You can use the d3.entries() function to transform an associative array into another array that contains an associative array with key and value keys for each key-value pair.
I'm glad you figured it out, #cdietschrun.
Version 4 of D3 allows you to do this a little more simply. It introduces a columns property, which creates an array of column headers (i.e. the dataset's 'keys').
So, instead of using your code:
var mappedArray = d3.entries(data[0]),
valueKey = mappedArray[1].key;
... you can use:
var valueKey = data.columns;
You can get keys (column names) using D3 v3 values() method like this:
const [dataValues] = d3.values(data)
const keys = Object.keys(dataValues)
console.log(keys);
I used d3.entries() as below:
for(i=0;i<data.length;i++)
{
var temp = d3.entries(data[i]);
for(j=0;j<temp.length;j++)
if(temp[j].key == selectedx)
myarray.push(temp[j].value);
}
Hope this helps :)
Related
I am importing an excel file using xlsx in angular, where the excel file is imported as an object of arrays, with each array being a row of the excel file and each item in each array is a cell within it's respective row. See below for how this is done
onFileChange(event: any)
{
const inputFile: DataTransfer = <DataTransfer>(event.target);
const fileReader: FileReader = new FileReader();
fileReader.onload = (event: any) =>
{
const binaryString: string = event.target.result;
const workBook: XLSX.WorkBook = XLSX.read(binaryString, { type: 'binary', sheetStubs: true});
/* sheetstubs true supposedly shows empty cells but isn't */
const workSheetName: string = workBook.SheetNames[0];
const workSheet: XLSX.WorkSheet = workBook.Sheets[workSheetName];
this.data = <Array>(XLSX.utils.sheet_to_json(workSheet,
{header: 1, blankrows: true }));
};
I am now trying to find the column which contains a manufacturer description by using a Regex search by looping through each of the arrays and searching for the term cap like so:
getManufacturerDescriptionColumn()
{
console.log(this.data)
for (const row in this.data)
{
var manDescriptIndex = row.search(/cap/i)
console.log(manDescriptIndex)
if (manDescriptIndex > -1)
{
return manDescriptIndex
}
}
}
however when ever I try this, even though the phrase cap is clearly present in some of the arrays when I run this, I am presented with all -1 values indicating that the phrase is not found in any of the arrays. Below is an attached example where CAP is clearly present in a couple of the 15 rows of the excel file but I am still met with 15 -1 values.
For example the array indexed at
Any ideas why a regex search isn't identifying the phrase in this scenario when I do console.log(this.data) I can see the phrase cap like so,
I have also tried adding another layer of iteration to isolate the strings of the individual cells in the row also to no avail.
As you can clearly read here MDN you are using the loop for..in which is a mistake, and you should be using for..of instead. Explanation below
for..in is made to iterate around objects, which actually works for arrays, but is just giving you the indexes, which in this case will be 0,1,2 and so on...
If you want to get the values you will have to use the for..of loop MDN
You will see this more clearly if you print console.log(row) before the check
Ok so #AlejandroVales was definitely right in his answer so I'm not gonna take away that well earned check mark but... I have an update to fully get the solution because while changing in to of was instrumental to me getting to the point I needed to get, there was more to be done and in the spirit of sharing information and frankly good karma, I have for others who run into a similar issue... a solution. Rather than trying to use array methods in order to see if I could find what I am looking for I tested each element in each array AGAINST a regular expression, AKA I flipped the problem on it's head a little bit. So to clarify take a look at the code block below
var cap = new RegExp(/cap/i)
for (const row of this.data)
{
for (let cell = 0; cell<row.length; cell++)
{
if (cap.test(row[cell]))
{
return cell
}
}
}
First set a variable equal to a regex, then iterate to a level where I am iterating over each cell in each array (each array is a row of the excel document in this case), then find the cell that returns a TRUE value and poof problem solved. If I didn't do a good job explaining this solution feel free to leave a comment below I am no expert but figured it's only right try my best to give value to a community that has given me so much value previously.
I am following ag-grid's official tutorial:
https://www.ag-grid.com/angular-getting-started/?utm_source=ag-grid-readme&utm_medium=repository&utm_campaign=github
I reached the part where I have to manipulate the information regarding the selected checkboxes. However, the documentation is not detailed; It does not explain how the code actually works. Perhaps, this makes sense since it is not their job to explain in detail. However, for someone like me who doesn't have solid experience with working with angular technology and who wants to truly understand how things work, this is troublesome!
In the html file, I have been asked to add this:
<button (click)="getSelectedRows()">Get Selected Rows</button>
And in app.component.ts, I have been asked to add this:
getSelectedRows() {
const selectedNodes = this.agGrid.api.getSelectedNodes();
const selectedData = selectedNodes.map(node => node.data);
const selectedDataStringPresentation = selectedData.map( node => node.make + ' ' + node.model).join(', ');
alert('Selected nodes: ${selectedDataStringPresentation}');
}
If someone could explain what the typescript code is doing exactly, that would be very generous.
Thank you!
I guess agGrid is the service storing your mock values, this simply gets an array of data from somwhere.
selectedData is another array that is created by transforming (transforms the array while providing a new reference, thus not modifying the original array) the selectedNodes array and only selecting the data property of each node.
selectedDataStringPresentation is the same, but this time it provides an array of formatted strings by merging the properties make and model of each object from selectedData.
What you probably fail to grasp is the usage of the ES6 (JavaScript standard) functions that are used here, and especially the map() function.
Here is the official documentation of the map() function for arrays : https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
Simply explained, it is a function that iterates over an array, and transforming each object by applying the function declared in the map, returning the results in a new array.
If you take the example of selectedData, you can translate the operation into this :
Loop over each object of selectedNodes, and return only the property data of the current object. Push the results in a new array.
This is especially useful when you want to work on different arrays that serve different purposes. For example, imagine you have a service that contains a list of objects. A backend service will provide you an array of numbers representing the IDs of the objects you have in your service.
You can then use the map() function to transform the array of IDs into an array of your Objects stored in your service easily.
Darn #Alex Beugnet(upvote) beat me to the punch! I'm going to post anyway as I was in the middle of writing it all.
Firstly I'm not sure how much of TypeScript you already know, I apologize if much of these becomes trivial, the purpose is only to ensure maximum clarification to the question in understanding the logic.
In the Enable Selection portion of the guide, you are essentially enabling multiple row selection in the grid and having the button return the data from those selected rows.
In order to see what's happening with the getMultipleRows() function, it would be best to visualize it via the Debugger provided in browsers, I'm using Chrome Developer Tools (hit F12), I would highly recommend it for understanding what is happening in the logic.
const selectedNodes
Let's start by selecting say 2 rows, I have selected the Porsche Boxster 72000 and Ford Mondeo 32000. After selecting them I click on the 'Get Selected Rows' button which triggers the getSelectedRows() function:
const selectedNodes = this.agGrid.api.getSelectedNodes();
The above line is assigning the constant variable 'selectedNodes' the RowNodes from AgGrid. Here you are using the AgGridNg2 method getSelectedNodes() to return the selected node data, which you would be returned an array in the form of:
[RowNode, RowNode] //(each for the row we have selected)
Looking into a RowNode we get:
These are all the RowNode properties provided by the AgGrid framework, you can ignore all of these object properties as you are only concerned with the 'data' property as you'll see in the next line of code!
const SelectedData
const selectedData = selectedNodes.map(node => node.data);
Here we are setting 'selectedData' as an array of RowNode.data, basically trying to get the data property from the RowNodes into an array.
The above line can basically be assumed as:
let selectedData = [];
for (let i = 0; i <= selectedNodes.length - 1; i++){
selectedData[i] = selectedNodes[i].data;
}
In which we are just trying to get the data property into a new constant variable 'selectedData'. Look at the documentation in order to better understand this: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map
const selectedData would be returned as:
[
{
make: "Porsche",
model: "Boxster",
price: 72000,
},
{
make: "Ford",
model: "Mondeo",
price: 32000
}
]
const selectedDataStringPresentation
const selectedDataStringPresentation = selectedData.map( node => node.make + ' ' + node.model).join(', ');
We take the selectedData array and concatenate the Make and Model as a single element for the array and adding a comma in the end. We would get "Porsche Boxter, Ford Mondeo".
let selectedData = [
{
make: "Porsche",
model: "Boxster",
price: 72000,
},
{
make: "Ford",
model: "Mondeo",
price: 32000
}
]
let selectedDataStringPresentation = [];
for (let i = 0; i <= selectedData.length - 1; i++){
selectedDataStringPresentation[i] = selectedData[i].make + ' ' + selectedData[i].model;
}
selectedDataStringPresentation = selectedDataStringPresentation.join(', ');
console.log(selectedDataStringPresentation);
alert()
And the last line,
alert('Selected nodes: ${selectedDataStringPresentation}');
You are going to send an alert in the browser that will display the selectedDataStringPresentation array.
I have 20,000 rows in a CSV which I have loaded using d3. Within this CSV there are roughly 4,000 unique category names (each being repeated across various numbers of rows).
I would like to make a list (an array or objects) of all the ~4,000 category names from my CSV, to be able to filter out categories that I do not want to work with.
See code and data sample below; the category column is called feature_id.
var rowConverter = function(d){
return{
event_date: parseTime(d.event_date),
claim_number: d.claim_number,
cause: d.cause,
detail_cause: d.detail_cause,
paid_total: parseFloat(d.paid_total),
feature_id: d.feature_id,
id: parseFloat(d.id)
};
}
d3.csv('claims_cwy.csv', rowConverter, function(dataset) {
console.log(dataset);
}
You can create an empty array, iterate over this dataset and for each iteration check this category if it exists. If not, add to the array. Something like:
const categories = []
dataset.forEach( item => {
if ( categories.indexOf(item.category) <= 0)
categories.push(item.category)
})
PS: I don't know which of this attributes in the row represents the category, it's not clear.
There are various ways to achieve what you want. If you want to keep it D3-ish you could make use of d3.set() which not only guarantees uniqueness of its values, but also allows you to provide an accessor to extract the categories' values, i.e. the field feature_id, from your data.
const categories = d3.set(dataset, d => d.feature_id);
Note, however, that this requires an additional loop through your data. As you claim to have a large set of data, you might want to do it step by step by adding to the set in the row converter function.
const categories = d3.set();
const rowConverter = function(d) {
categories.add(d.feature_id);
};
Whatever approach you prefer the unique category values are available by calling d3.values().
I have a 2d array representing rows in a database. I'm using officeJS to load and manipulate the data in Excel. I update, insert, and delete rows. The challenge I'm facing is that I need to figure out the changed rows (inserted, deleted or updated) so that I can update only those rows in the database. I'm sending one query for the updated and inserted rows and one query for the deleted rows. I'm able to do this using lodash for data with 5000 rows and 10 columns. I'd like to scale this to a much larger data set and I'm wondering if there are any alternatives to what I'm currently doing. Below is the code I'm using to find the difference.
insertedOrUpdatedRows = _.differenceWith(modifiedData, originalData, _.isEqual);
deletedRows = _.differenceWith(originalData, modifiedData, compareFunction);
function compareFunction(a, b) {
if(a[0] == b[0]) {
return true;
}
else
return false;
}
Sample data array
[ [1,data,data,data],
[2,data,data,data] ]
The first element is the primary key.
Because you have mentioned that your Javascript engine is crashing (which it should not, at 50,000 rows - so I would revisit the logic), I would recommend chunking out the data using Lodash's _.chunk function:
_.chunk(modifiedData, modifiedData.length/500).map({
...
...
});
Ok im using the following logic. Not sure why its crashing at 50K rows. OriginalData and ModifiedData are in the format of the sample 2D array mentioned above.
var originalDataStrings = [];
var modifiedDataStrings = [];
var insertedOrUpdatedRows;
originalData.forEach(function(row){
originalDataStrings.push(JSON.stringify(row));
});
modifiedData.forEach(function(row){
modifiedDataStrings.push(JSON.stringify(row));
})
insertedOrUpdatedRows = _.differenceWith(modifiedDataStrings, originalDataStrings, _.isEqual);
console.log(insertedOrUpdatedRows);
I have an array of arrays where the position of the value is important in that this data is used to ultimately layout a grid.
[[a,b,c,d],[a,b,c,d][a,b,c,d],[a,b,c,d]] // Original data
Based on an action, values within this "dataset" can change, but the size will never change.
[[a,b,c,d],[a,b,b,b][a,c,c,c],[a,b,c,d]] // Modified data
What I'd like to do is to return a dataset that only contains the delta values:
[[null,null,null,null,],[null,null,b,b],[null,c,null,c],[null,null,null,null,]]
Now, I do always know the max X and Y of the dataset, and could simply loop through the original dataset, comparing it's value to the corresponding value and build a new array, but it seems like there could be a more efficient way to accomplish this.
That said, my js-fu is minimal at best, and that's why I'm brining the problem here. Are there any language provided methods for accomplishing this? Suggested approaches? etc?
A bit of array mapping should do the trick.
Here's a fiddle: http://jsfiddle.net/C8jnr/
Given the following:
var arr_A = [[a,b,c,d],[a,b,c,d],[a,b,c,d],[a,b,c,d]];
var arr_B = [[a,b,c,d],[a,b,b,b],[a,c,c,c],[a,b,c,d]];
function deltaArrays(arr1, arr2)
{
return arr1.map(function(el_arr,i,arr){
return el_arr.map(function(el, j){
return (el == arr2[i][j]) ? null : arr2[i][j];
});
});
}
Calling deltaArrays on the two will yield the expected delta array;
deltaArrays(arr_A, arr_B) = [[null,null,null,null,],[null,null,b,b],[null,c,null,c],[null,null,null,null,]]
I don't believe there is a built in way to do this. Your best bet is to loop through each of the array elements and compare them with the new array, adding it to a list/dictionary if it is different.