I want to store data in a SQLite database directly from a javascript script. I found this SQL.js library that is a port for javascript. However, apparently it's only available for coffeescript. Does anyone know how to use it in javascript? Other ideas about how to store data in SQLite DB are welcomed too.
Update
sql.js now has its own github organisation, where both the original author and I are members: https://github.com/sql-js/sql.js/ .
The API itself itself is now written in javascript.
Original answer
I am the author of this port of the latest version of sqlite to javascript: https://github.com/lovasoa/sql.js
It is based on the one you mentioned (https://github.com/kripken/sql.js), but includes many improvements, including a full documentation: http://lovasoa.github.io/sql.js/documentation/
Here is an example of how to use this version of sql.js
<script src='js/sql.js'></script>
<script>
//Create the database
var db = new SQL.Database();
// Run a query without reading the results
db.run("CREATE TABLE test (col1, col2);");
// Insert two rows: (1,111) and (2,222)
db.run("INSERT INTO test VALUES (?,?), (?,?)", [1,111,2,222]);
// Prepare a statement
var stmt = db.prepare("SELECT * FROM test WHERE col1 BETWEEN $start AND $end");
stmt.getAsObject({$start:1, $end:1}); // {col1:1, col2:111}
// Bind new values
stmt.bind({$start:1, $end:2});
while(stmt.step()) { //
var row = stmt.getAsObject();
// [...] do something with the row of result
}
</script>
I'm using SQL.js from pure JavaScript without any problems. Simply include the following file:
https://cdnjs.com/libraries/sql.js
Related
This question might be a basic one but I am very new to SSJS so thank you for your understanding.
The data extension names JourneyA, JourneyB, JourneyC...infinity are the result of Journey Builder. Then, I got the data extension name AllJourneys from _Journey.
SELECT JourneyName as "JourneyName",
FROM _Journey j
INNER JOIN(select JourneyID, max(CreatedDate) as MaxDate FROM _Journey
GROUP BY JourneyID) sort on sort.JourneyID = j.JourneyID and j.CreatedDate = sort.MaxDate
After that, I would like to count the number of audiences in each journey and put the results in data extension name Summary using upsert. Additionally, data extension name Summary is a non-sendable data extension.
According to my understanding, the data extension name Summary can be done by SSJS (Script Activity in Automation Studio), right? or please suggest me if it can be done by any other way.
I have done some SSJS but cannot figure out what I sould do next.
<script runat="server">
Platform.Load("core","1");
var AllJourneys = DataExtension.Init('AllJourneys');
var AllJourneysData = testDE.Rows.Retrieve();
var Summary = DataExtension.Init('Summary');
for (var i = 0; i < testDEData.length; i++) {
var JourneyName = AllJourneysData[i].JourneyName;
var count = xxx
var result = Summary.Rows.Upsert({"JourneyName":JourneyName}, ['Count(CusID)'], [count]);
}
</script>
If you know the number of Journey DE's, you could also use SQL Query Activities in Automation studio to do this. Create your data extension for the Summary table, and then run a SQL query or series of queries to look up each of the journey DE's and update the table with the updated value.
I would recommend adding one additional field to your summary table for a REFRESHDATE, so that you can see in the data the last time it was updated.
I suggest you to use the WSproxy to manage data extension data.
The main problem that you have to bypass is that SSJS cannot retrieve more than 2,500 rows per "call".
This tutorial explains how to retrieve more than 2,500 rows.
https://ampscript.xyz/how-tos/how-to-retrieve-more-than-2500-records-from-a-data-extension-with-server-side-javascript/
Then I suggest you to use basic javascript function to check the length or simply manage your returned json to get the desired id count.
You can finally upsert that data with t
I started to write a scraper for the site to collect data on cars. As it turned out, the data structure can change, since the sellers do not fill all the fields, because of what there are fields that can change, and during the scraper as a result in the csv file, the values are in different fields.
page example:
https://www.olx.ua/obyavlenie/prodam-voikswagen-touran-2011-goda-IDBzxYq.html#87fcf09cbd
https://www.olx.ua/obyavlenie/fiat-500-1-4-IDBjdOc.html#87fcf09cbd
data example:
Data example
One approach was to check the field name with text () = "Category name", but I'm not sure how to correctly write the result to the correct cells.
Also I use the built-in Google developer tool, and with the help of the command document.getElementsByClassName('margintop5')[0].innerText
I brought out the whole contents of the table, but the results are not structured.
So, if the output can be in json format then it would solve my problem?
innerText result
In addition, when I studied the page code, I came across a javascript script in which all the necessary data is already structured, but I do not know how to get them.
<script type="text/javascript">
var GPT = GPT || {};
GPT.targeting = {"cat_l0":"transport","cat_l1":"legkovye-avtomobili","cat_l2":"volkswagen","cat_l0_id":"1532","cat_l1_id":"108","cat_l2_id":"1109","ad_title":"volkswagen-jetta","ad_img":"https:\/\/img01-olxua.akamaized.net\/img-olxua\/676103437_1_644x461_volkswagen-jetta-kiev.jpg","offer_seek":"offer","private_business":"private","region":"ko","subregion":"kiev","city":"kiev","model":["jetta"],"modification":[],"motor_year":[2006],"car_body":["sedan"],"color":["6"],"fuel_type":["543"],"motor_engine_size":["1751-2000"],"transmission_type":["546"],"motor_mileage":["175001-200000"],"condition":["first-owner"],"car_option":["air_con","climate-control","cruise-control","electric_windows","heated-seats","leather-interior","light-sensor","luke","on-board-computer","park_assist","power-steering","rain-sensor"],"multimedia":["acoustics","aux","cd"],"safety":["abs","airbag","central-locking","esp","immobilizer","servorul"],"other":["glass-tinting"],"cleared_customs":["no"],"price":["3001-5000"],"ad_price":"4500","currency":"USD","safedealads":"","premium_ad":"0","imported":"0","importer_code":"","ad_type_view":"normal","dfp_user_id":"e3db0bed-c3c9-98e5-2476-1492de8f5969-ver2","segment":[],"dfp_segment_test":"76","dfp_segment_test_v2":"46","dfp_segment_test_v3":"46","dfp_segment_test_v4":"32","adx":["bda2p24","bda1p24","bdl2p24","bdl1p24"],"comp":["o12"],"lister_lifecycle":"0","last_pv_imps":"2","user-ad-fq":"2","ses_pv_seq":"1","user-ad-dens":"2","listingview_test":"1","env":"production","url_action":"ad","lang":"ru","con_inf":"transportxxlegkovye-avtomobilixx46"};
data in json dict
How can I get the data from the pages using python and scrapy?
You can do it by extracting the JS code from the <script> block, using a regex to get only the JS object with the data and then loading it using the json module:
query = 'script:contains("GPT.targeting = ")::text'
js_code = response.css(query).re_first('targeting = ({.*});')
data = json.loads(js_code)
This way, data is a python dict containing the data from the JS object.
More about the re_first method here: https://doc.scrapy.org/en/latest/topics/selectors.html#using-selectors-with-regular-expressions
I am building a dummy application for testing javascript and mssql connections, ive done a similar process using c# and now tried to find a way of doing it with javascript, i found the activeXobject online and just figured out it only worked with IE not chrome.
Not sure what other objects i can use that would work here, i simply want to insert a row into the db while using simple html5 and javascript.
Here is what i have to far...
function persistInDB(){
/* Getting access to the database */
var connection = new ActiveXObject("ADODB.Connection");
var connectionstring = "Data Source=SQL5036.HostBuddy.com;Initial Catalog=DB_A1F3FA_dominicmazur;User Id=--MYID--;Password=--MYPASS--;";
connection.Open(connectionstring);
/* JavaScript obect to access a SQL query's results */
var rs = new ActiveXObject("ADODB.Recordset");
var sql = "INSERT INTO DB_A1F3FA_dominicmazur.dbo.Orders (SIZE,CHEESE,PEPPERONI,HAM,PINEAPPLE,SAUSAGE,FETA,TOMATOE,OLIVES,SUBTOTAL,GST,TOTAL) VALUES ('"+size+"','"+Cheese+"','"+Pepperoni+"','"+Ham+"','"+Pineapple+"','"+Sausage+"','"+Feta+"','"+Tomatoes+"','"+Olives+"','"+cost+"','"+GST+"','"+grandTotal+"')";
alert(sql);
rs.Open(sql, connection);
/* Closing the connections */
//rs.close;
connection.close;
}
Thank you in advance :)
There's no good reason to try to do this from a browser, unless you have some absurdly niche requirements. (And any such requirements would probably be better questioned than followed!)
Collect the data on a form and send it to your server using HTML5 & JavaScript, and then do the database operation/s from there.
During my load test, I would like to fetch values from an SQL database. How can I achieve this on load runner TrueClient protocol using JavaScript?
This would be great help…
Important: This will only work in TruClient (IE) and not in TruClient (Firefox).
Enter a new "Eveluate Javascript" step, and edit the javasctipt like so:
var connection = new ActiveXObject("ADODB.Connection") ;
var connectionstring="Data Source=<server>;Initial Catalog=<catalog>;User ID=<user>;Password=<password>;Provider=SQLOLEDB";
connection.Open(connectionstring);
var rs = new ActiveXObject("ADODB.Recordset");
rs.Open("SELECT * FROM table", connection);
rs.MoveFirst
while(!rs.eof)
{
// Here you should get the value from the 1st cell, 1st column
var value = rs.fields(1);
rs.movenext;
}
rs.close;
connection.close;
There are several options.
I'll list them in order of their complexity:
Option 1:
Use a parameter file to hold all of your data. If you need to modify it periodically, consider placing it in a shared location, accessible for all LGs.
Option 2:
Use the Virtual Table Server (VTS) provided with LoadRunner. It is dedicated to sharing test data between virtual-users. Queries are easy with a built in API.
Option 3:
You could write a custom C function, using LoadRunner DB API to query the DB, calling the function from your script using an Eval C step.
Note this can only be done in VuGen.
I'm trying to use HTML 5's local database feature on a Mac Dashboard widget.
I'm programming in Dashcode the following javascript:
if (window.openDatabase)
{
database = openDatabase("MyDB", "1.0", "Sample DB", 1000);
if (database)
{
...database code here...
}
}
Unfortunately the database-variable remains always null after the call to openDatabase-method. I'm starting to think that local databases are not supported in Widgets...
Any ideas?
/pom
No you will not be able to do the above. And even if you could then you would not be able to distribute the widget without distributing the database assuming it was a MySQL or SGLite. (not sure what you mean by HTML 5's local Db.
here are a number of ways round this:-
You can add a data source which can be a JSON file, or an XML file or and RSS feed. So to do this with JSON for example you would write a page on a server in PHP or something that accessed a database so that when the URL was called the result was a JSON string. Take the JSON string and parse it and use it in the Widget. This will let you get data but not save it.
Another way would be to use the user preferences. This allows you to save and retrieve data in the individual widget.
So
var preferenceKey = "key"; // replace with the key for a preference
var preferenceValue = "value"; // replace with a preference to save
// Preference code
widget.setPreferenceForKey(preferenceValue, preferenceKey);
You can then retrieve it with
var preferenceForKey = "key"; // replace with the key for a preference
// Preference code
preferenceForKey = widget.preferenceForKey(preferenceForKey);
The external call, you could also use REST will let you read any amount of data in and the preferences will let you save data for later reuse that will survive log out's and shut downs.
The Apple site has a lot of information about Widgets and tutorials as well thjat are worth working through.
Hope this helps.