I'm working on a CSV uploader that uses PapaParse as it's CSV parser. For my CSV I would like my first column to act as my header for the parsed data as opposed to the first row. In order to get the expected outcome, I've been having to manually transpose the CSV in the editor before uploading.
The reason for this is that my users find it much easier to edit the CSV when the headers are in the first column and not the first row. Is there a way I can do this in PapaParse (or even JavaScript outside of PapaParse)?
if (file != null) {
Papa.parse(file, {
header: true,
complete: function (results, file) {
console.log("Parsing complete: ", results, file);
}
});
}
I would suggest to parse the array with PapaParse and then perform transpose over the result with JS.
Using this method: https://stackoverflow.com/a/4492703/1625793
So it would look like that transpose(result.data)
-- Update --
const transposed = transpose(result.data)
const headers = transposed.shift();
const res = transposed.map(row => row.reduce((acc, col, ind) => {acc[headers[ind]] = col; return acc}, {}))
Related
I have a file upload system. It reads excel file and upload data to database (mongoose). When I console.log(sheetData) it returns array of arrays with the objects. The array inside arrays contain only 100 objects and then it create another array. Below is my code and images of the issue.
Code
//Excel Upload
const handleExcelSubmit = async (e) => {
e.preventDefault();
const reader = new FileReader();
reader.onload = async (e) => {
const data = e.target.result;
const workbook = xlsx.read(data, { type: "binary" });
const sheetNames = workbook.SheetNames[0];
const workSheet = workbook.Sheets[sheetNames];
const sheetData = xlsx.utils.sheet_to_json(workSheet, {
header: "1",
});
const headers = sheetData[0];
return convertToJson(headers, sheetData); // <--- returns array of arrays???
dispatch(importExcel(sheetData)); // currently disabled for debugging
};
reader.readAsBinaryString(excelFile);
};
//Converts data to json. IDK if this is useful I received same data without this function
const convertToJson = async (headers, data) => {
const rows = [];
data.forEach(async () => {
let rowData = {};
rows.forEach(async (element, index) => {
rowData[headers[index]] = element;
});
rows.push(rowData);
});
setTableData(rows);
console.log(tableData);
return rows;
};
Image - Array of Arrays
Summary
Actually I want that only single array is created with all the objects in one array. Currently it is creating two arrays, with one array of limit with 100 objects, after 99th object it creates 2nd array and starts from 100th object. Is there any option that it create only one array with all 108 objects in it.
(This app is production so I have to hide those data. Sorry)
Thank You
I found the issue. It was in my backend. I am sending an array of data so I used for loop to fetch and upload data to database in backend. The mistake was I was sending a response in between loop which was breaking it. The response was just for check backend data and was useless so I removed it and it worked. If anyone faces same issue in future then please check your loops if they break while the loop is running.
I am parsing a csv file with following contents using csv-parse -
userID,sysID
20,50
30,71
However, on the objects returned it isn't possible to access the property created from the first column userID.
Here is my code --
async function main(){
let systemIDs = await getSystemIds('./systems.csv');
console.log(`Scanning data for ${systemIDs.length} systems..`);
console.log(systemIDs[0]);
console.log(systemIDs[0].userID); // This prints undefined
console.log(systemIDs[0].sysID); // This prints the correct value
}
async function getSystemIds(path){
let ids= [];
await new Promise ((resolve,reject)=>{
const csvParser = csvParse({columns:true, skip_empty_lines: true});
FS.createReadStream(path)
.pipe(csvParser)
.on('readable', ()=>{
let record ;
while(record = csvParser.read()) {
ids.push(record);
}
})
.on('finish',()=>{
resolve();
});
});
return ids;
}
Output -
Scanning data for 2 systems..
{ 'userID': '20', sysID: '50' }
undefined // <== The Problem
50
I notice the first column key userID has single quotes around it in the console output where as sysID doesn't. But don't know what is causing them.
Figured it out myself in the end...
I needed the BOM option. The documentation states it should be set to true for UTF-8 files. But it defaults to false.
Excel by default generates csv files with BOM as the first character in CSV files. This gets picked up as part of the header (and key name) by the parser.
With the bom option set to true, it can handle csv files generated from excel or other programs.
const csvParser = csvParse({
columns: true,
skip_empty_lines: true,
bom: true
});
I want to parse json data from multiple url. I tried using for loop but it only parse json data from 1 url, while i want to show data from all urls.
I have a working script, but it is hardcoding. This is part of my script.
const apiex1 = 'http://www.ex.com/example1.json';
const apiex2 = 'http://www.ex.com/example2.json';
$.getJSON(apiex1, ex1Weather);
$.getJSON(apiex2, ex2Weather);
function ex1Weather(report) {
weatherData(report);
document.querySelector('tbody.ex1').innerHTML = apiWeather;
}
function ex2Weather(report) {
weatherData(report);
document.querySelector('tbody.ex2').innerHTML = apiWeather;
}
I have around 30 data/urls, using this script just overwhelming since the data will increase. What is the best way to make a better configuration for this script?
You should definitely use a loop over an array of URLs instead of hardcoding each one.
const apiUrls = ['http://ex.com/example1.json', 'http://ex.com/example2.json'];
apiUrls.forEach((url, index) => {
$.getJson(url, report => {
weatherData(report);
document.querySelector('tbody.ex' + index).innerHTML = apiWeather;
});
});
i need to make Array for data grid.
Input is XML(string). With lots of unnecessary data, i need only array "a:Client"
Here is my code which works, but I think it's not too clean to set a way like this.
parse(XML){
var parseString = require('react-native-xml2js').parseString;
var xml = XML;
parseString(xml, (err, result) => {
this.setState({rows: result["s:Envelope"]["s:Body"][0].GetClientsResponse[0].GetClientsResult[0]["a:Client"]});
});
}
Here is XML String
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><GetClientsResponse xmlns="http://tempuri.org/"><GetClientsResult xmlns:a="http://schemas.datacontract.org/2004/07/tt4t.Dispatching.ApplicationServer.Data.CommonData" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><a:Client><a:ClientID>0</a:ClientID><a:Name/></a:Client><a:Client><a:ClientID>12</a:ClientID><a:Name>Magistrát města Liberec</a:Name></a:Client><a:Client><a:ClientID>30</a:ClientID><a:Name>Krajský úřad Libereckého kraje</a:Name></a:Client><a:Client><a:ClientID>31</a:ClientID><a:Name>OC Nisa</a:Name></a:Client><a:Client><a:ClientID>32</a:ClientID><a:Name>Globus</a:Name></a:Client><a:Client><a:ClientID>33</a:ClientID><a:Name>Die Länderbahn GmbH DLB</a:Name></a:Client><a:Client><a:ClientID>34</a:ClientID><a:Name>Magistrát města Jablonec nad Nisou</a:Name></a:Client><a:Client><a:ClientID>35</a:ClientID><a:Name>Dopravní podnik měst Liberce a Jablonce n.N.</a:Name></a:Client><a:Client><a:ClientID>36</a:ClientID><a:Name>Liplastec</a:Name></a:Client><a:Client><a:ClientID>37</a:ClientID><a:Name>CBRE Česká republika</a:Name></a:Client><a:Client><a:ClientID>38</a:ClientID><a:Name>Cinestar</a:Name></a:Client><a:Client><a:ClientID>39</a:ClientID><a:Name>České dráhy a.s.</a:Name></a:Client><a:Client><a:ClientID>40</a:ClientID><a:Name>DENSO MANUFACTURING CZECH s.r.o.</a:Name></a:Client><a:Client><a:ClientID>41</a:ClientID><a:Name>Hasiči</a:Name></a:Client><a:Client><a:ClientID>42</a:ClientID><a:Name>MŠ Lísteček ((N.Ruda)</a:Name></a:Client><a:Client><a:ClientID>43</a:ClientID><a:Name>MŠ Tanvaldská (vč. pobočky Poštovní)</a:Name></a:Client><a:Client><a:ClientID>44</a:ClientID><a:Name>Policie ČR</a:Name></a:Client><a:Client><a:ClientID>45</a:ClientID><a:Name>SOŠ Kateřinky</a:Name></a:Client><a:Client><a:ClientID>46</a:ClientID><a:Name>Sportkids</a:Name></a:Client><a:Client><a:ClientID>47</a:ClientID><a:Name>SŽDC, s.o.</a:Name></a:Client><a:Client><a:ClientID>48</a:ClientID><a:Name>Záchranná služba</a:Name></a:Client><a:Client><a:ClientID>49</a:ClientID><a:Name>ZŠ Nad Školou</a:Name></a:Client><a:Client><a:ClientID>50</a:ClientID><a:Name>SKI KLUB Jizerska 50</a:Name></a:Client><a:Client><a:ClientID>51</a:ClientID><a:Name>TJ Dukla Liberec, z.s.</a:Name></a:Client><a:Client><a:ClientID>52</a:ClientID><a:Name>Jiný, viz poznámka</a:Name></a:Client><a:Client><a:ClientID>53</a:ClientID><a:Name>KORID LK, spol. s r.o.</a:Name></a:Client><a:Client><a:ClientID>54</a:ClientID><a:Name>STUDENT AGENCY k.s.</a:Name></a:Client><a:Client><a:ClientID>55</a:ClientID><a:Name>Boveraclub z.s.</a:Name></a:Client><a:Client><a:ClientID>56</a:ClientID><a:Name>Archa 13</a:Name></a:Client><a:Client><a:ClientID>57</a:ClientID><a:Name>Pekárny</a:Name></a:Client><a:Client><a:ClientID>58</a:ClientID><a:Name>MŠ Sídliště - Skloněná</a:Name></a:Client><a:Client><a:ClientID>59</a:ClientID><a:Name>Vratislavice</a:Name></a:Client><a:Client><a:ClientID>60</a:ClientID><a:Name>První festivalová s.r.o.</a:Name></a:Client><a:Client><a:ClientID>61</a:ClientID><a:Name>Městský obvod Liberec - Vratislavice nad Nisou</a:Name></a:Client><a:Client><a:ClientID>62</a:ClientID><a:Name>Preciosa</a:Name></a:Client><a:Client><a:ClientID>63</a:ClientID><a:Name>Central Europe Spartan Race ESR Enterprises Czech</a:Name></a:Client><a:Client><a:ClientID>64</a:ClientID><a:Name>Kümpers Textil, s.r.o.</a:Name></a:Client><a:Client><a:ClientID>65</a:ClientID><a:Name>Základní škola a Mateřská škola, Stráž n.N.</a:Name></a:Client><a:Client><a:ClientID>66</a:ClientID><a:Name>Základní škola a mateřská škola logopedická, LBC</a:Name></a:Client><a:Client><a:ClientID>165</a:ClientID><a:Name>Zájezd stř. 704</a:Name></a:Client><a:Client><a:ClientID>166</a:ClientID><a:Name>DPMLJ X11</a:Name></a:Client><a:Client><a:ClientID>167</a:ClientID><a:Name>DPMLJ 2 a 3</a:Name></a:Client><a:Client><a:ClientID>168</a:ClientID><a:Name>DPMLJ 5 a 11</a:Name></a:Client></GetClientsResult></GetClientsResponse></s:Body></s:Envelope>
Your code works, but I think that it have two things that can be done better....
1) It's not readable, if you'll read this code in a year it'll be difficult to understand what you are looking for, so I would wrap that:
{rows: result["s:Envelope"]["s:Body"][0].GetClientsResponse[0].GetClientsResult[0]["a:Client"]}
in a function like:
getClient(result)
2) It's not reusable, if the XML change or you need to find some other data it'll break, so try to use a function with a parameter that finds exactly what you need recursively if necessary
(You can give a look here, I know that you have array, but you can search recursively also for your arrays)
The only thing I worry about the above code is that it missed error checking. I am sure how confident that you are about your API and the schema will behave as expected. I feel it is good to have error checking when you are accessing a nested property of a nested object.
afterParsing(err, result){
if(err){
// Handle error
return [];
}
try {
const res = result["s:Envelope"]["s:Body"][0].GetClientsResponse[0].GetClientsResult[0]["a:Client"];
return res;
} catch (error) {
return [];
}
}
parse(XML){
var parseString = require('react-native-xml2js').parseString;
var xml = XML;
parseString(xml, (err, result) => {
this.setState({rows: this.afterParsing(err, result)});// if afterParsing and this function are in the class
});
}
If this piece of code is going to be run in the browser you can also try DOMParser.
const stringContainingXMLSource = `<your xml string>`;
var parser = new DOMParser();
var doc = parser.parseFromString(stringContainingXMLSource, "application/xml");
console.log(Array.from(doc.querySelectorAll('Envelope > Body > GetClientsResponse > GetClientsResult > Client')).map(n => n.textContent));
I'm trying to retrieve the filename of gists on Github, from the data obtained from Github's API. I'm using javascript to access the data.
An example result can be found here: https://api.github.com/users/blaercom/gists. I've also copied a shortened version of the data below.
{
...
id: "4468273",
files: {
firstpost.md: {
type: "text/plain",
filename: "firstpost.md",
size: 16,
language: "Markdown"
}
}
...
}
I've been trying many things, but I can't seem to access the filename. This is because the 'files' object is not a list, but a key-value pair, where the key identifier matches the filename itself.
Things I've tried include
filename = files[0]['filename']
filename = files[0].filename
filename = files['filename']
Frankly, the only methods that work are variations of filename = files['firstpost.md']['filename'] but this isn't valid since I can not know the filename beforehand.
I'm sure it is possible to access the filename, but have been spending quite a while testing different methods. Help would be appreciated!
You can use for (var key in object) {}, here is an example using the data from your api call:
var filenames = [];
for (var filename in data[0].files) {
filenames.push(filename);
}
console.log(filenames); // ["firstpost.md"]
Here is a real example using your json response
var obj='your json data';
var fileNames=[];
for(var i in obj[0]['files'])
{
var fileName=obj[0]['files'][i]['filename'];
fileNames.push(fileName);
}
document.write(fileNames[0]); // firstpost.md
Example.
Update:
Another example using jsonp/script.
<script src="https://api.github.com/users/blaercom/gists?callback=myCallback"></script>
The callback function
function myCallback(o)
{
var obj=o.data;
var fileNames=[];
for(var i in obj[0]['files'])
{
var fileName=obj[0]['files'][i]['filename'];
fileNames.push(fileName);
}
document.write(fileNames[0]); // firstpost.md
}