I'm using sheetJS to convert HTML table into excel. The table was generated using datatable jquery with pagination. The data is from sql database, When im exporting it to excel, it only generates the page that is active in DOM.
my script to generate excel from html table:
let tbl1 = document.getElementById("Table1");
let worksheet_tmp1 = XLSX.utils.table_to_sheet(tbl1);
let a = XLSX.utils.sheet_to_json(worksheet_tmp1, { header: 1 })
let worksheet = XLSX.utils.json_to_sheet(a, { skipHeader: true })
const new_workbook = XLSX.utils.book_new()
XLSX.utils.book_append_sheet(new_workbook, worksheet, "worksheet")
XLSX.writeFile(new_workbook, 'Report.xls')
How can i include the whole data not only just the active page? Thanks for answer
Documentation doesn't help at all,no Table or Grid is specified...(or I cant find it)
I tried to create a grid from inside InDesign and it shows up as TextFrame,but still I dont understand how to manage it.
The tool I need to do takes a file(CSV/JSON) and generates a Table(or whatever is called in Adobe) from it,but the problem is that I can't find anything about Table generation.
Basically you can make a table from a selected text with the method convertToTable() this way:
var doc = app.activeDocument; // your document
var frame = doc.pages[0].textFrames[0]; // first text frame on first page
frame.texts.everyItem().select(); // select all text inside the frame
var table = app.selection[0].convertToTable(); // convert the selection into a table
Before:
After:
Reference:
Text object
As for the rest... it's need more details about your workflow. JSON and CSV are quite different beasts, it would be different parsing algorithms for each of the formats. Will you copy the contents of the files manually or the script should read all csv or json files from some predefined folder? Or there should be some interface to select a file(s). Or a folder? How it supposed to handle a page size and formatting of the table? Etc...
I started to write a scraper for the site to collect data on cars. As it turned out, the data structure can change, since the sellers do not fill all the fields, because of what there are fields that can change, and during the scraper as a result in the csv file, the values are in different fields.
page example:
https://www.olx.ua/obyavlenie/prodam-voikswagen-touran-2011-goda-IDBzxYq.html#87fcf09cbd
https://www.olx.ua/obyavlenie/fiat-500-1-4-IDBjdOc.html#87fcf09cbd
data example:
Data example
One approach was to check the field name with text () = "Category name", but I'm not sure how to correctly write the result to the correct cells.
Also I use the built-in Google developer tool, and with the help of the command document.getElementsByClassName('margintop5')[0].innerText
I brought out the whole contents of the table, but the results are not structured.
So, if the output can be in json format then it would solve my problem?
innerText result
In addition, when I studied the page code, I came across a javascript script in which all the necessary data is already structured, but I do not know how to get them.
<script type="text/javascript">
var GPT = GPT || {};
GPT.targeting = {"cat_l0":"transport","cat_l1":"legkovye-avtomobili","cat_l2":"volkswagen","cat_l0_id":"1532","cat_l1_id":"108","cat_l2_id":"1109","ad_title":"volkswagen-jetta","ad_img":"https:\/\/img01-olxua.akamaized.net\/img-olxua\/676103437_1_644x461_volkswagen-jetta-kiev.jpg","offer_seek":"offer","private_business":"private","region":"ko","subregion":"kiev","city":"kiev","model":["jetta"],"modification":[],"motor_year":[2006],"car_body":["sedan"],"color":["6"],"fuel_type":["543"],"motor_engine_size":["1751-2000"],"transmission_type":["546"],"motor_mileage":["175001-200000"],"condition":["first-owner"],"car_option":["air_con","climate-control","cruise-control","electric_windows","heated-seats","leather-interior","light-sensor","luke","on-board-computer","park_assist","power-steering","rain-sensor"],"multimedia":["acoustics","aux","cd"],"safety":["abs","airbag","central-locking","esp","immobilizer","servorul"],"other":["glass-tinting"],"cleared_customs":["no"],"price":["3001-5000"],"ad_price":"4500","currency":"USD","safedealads":"","premium_ad":"0","imported":"0","importer_code":"","ad_type_view":"normal","dfp_user_id":"e3db0bed-c3c9-98e5-2476-1492de8f5969-ver2","segment":[],"dfp_segment_test":"76","dfp_segment_test_v2":"46","dfp_segment_test_v3":"46","dfp_segment_test_v4":"32","adx":["bda2p24","bda1p24","bdl2p24","bdl1p24"],"comp":["o12"],"lister_lifecycle":"0","last_pv_imps":"2","user-ad-fq":"2","ses_pv_seq":"1","user-ad-dens":"2","listingview_test":"1","env":"production","url_action":"ad","lang":"ru","con_inf":"transportxxlegkovye-avtomobilixx46"};
data in json dict
How can I get the data from the pages using python and scrapy?
You can do it by extracting the JS code from the <script> block, using a regex to get only the JS object with the data and then loading it using the json module:
query = 'script:contains("GPT.targeting = ")::text'
js_code = response.css(query).re_first('targeting = ({.*});')
data = json.loads(js_code)
This way, data is a python dict containing the data from the JS object.
More about the re_first method here: https://doc.scrapy.org/en/latest/topics/selectors.html#using-selectors-with-regular-expressions
I have a download link on a webpage which downloads a csv file. This csv file contains 100+ columns with data (numeric, text, decimal etc). The values in csv are actually taken from the webpage entered by users, when user clicks save and clicks download link these values are downloaded to csv.
How do i write a nightwatch test which can validate all of these values from web page are downloaded to csv correctly?
I have mapped all the values (that go into csv) from webpage into UI properties (such as cityName:'input[elementId]', stateName:'input[elementId]' ..so on).
I am looking for a way to perform below steps in my nightwatch tests:
Get all the values from the UI for ex: browser.getValue(cityName,function(textBoxValue){var city=textBoxValue})
Trigger Click download link
Read the downloaded file from CSV and compare the value in csv with that of the value in UI - pass if the csv value and UI value match.
Finally I was able to parse values from a csv using fs and papa parse
const papa = require('papaparse');
const fs = require('fs');
..
..
fs.stat(localFilePath, function(err, stats){
if (!err) {
console.log("FileSize: "+stats['size']);
var fileContents = fs.readFileSync(localFilePath, 'utf8');
papa.parse(fileContents,{
complete:function(results){
firstName = results.data[1][1];
}
}
}
}
Background: The value of the Summary property corresponds to the setting of the Description box in the Alternative Text dialog box, which is displayed by right-clicking on an Excel table and then clicking Alternative Text.
You can get to it the property from VBA when you have a ListObject reference like so:
Dim mytable As ListObject
Set myTable ....
MsgBox myTable.Summary
From C# / .NET / VSTO:
https://msdn.microsoft.com/en-us/library/microsoft.office.interop.excel.listobject.summary.aspx
Question: I want to be able to get to this property using the EWA namespace (https://msdn.microsoft.com/en-us/library/office/ee589018.aspx?f=255&MSPPError=-2147217396)
In JavaScript, let's say that I have:
const ewa = ewaRef.EwaControl.getInstances().getItem(0);
to get to an Excel Web Part. Now I can get to the "name items" of the active workbook, of which my table will be one of:
const wb = ewa.getActiveWorkbook();
var namedItems = wb.getNamedItems();
const table = namedItems.getItemByName('Table1');
And I can validate it's an actual table:
if (table.getNamedItemType() !== Ewa.NamedItemType.Table) {
return;
}
Sample Ewa code: https://msdn.microsoft.com/en-us/library/office/ee660116.aspx?f=255&MSPPError=-2147217396
When I look at the properties of the Table object in the browser's debugger, I don't get any clues.