I'm using the youtube-serach NPM library with express to find the first youtube video with a song name.
app.get("/search", (req, res) => {
var search = require("youtube-search");
var SONG = req.query.SONG;
var opts = {
maxResults: 10,
key: "[REDACTED]"
};
search(SONG, opts, function(err, results) {
if (err) return console.log(err);
res.json(results);
});
});
When I set SONG to "DJ Turn It Up", the first result when you search in the youtube search bar is the youtube video "Yellow Claw - DJ Turn It Up [Official Full Stream]" by Mad Decent.
When I use youtube-search to sear for "DJ Turn It Up" none of the 10 results are the Mad Decent video, and the first result is actually a scene from Riverdale with the song in it, with 1/33 of the views!?!
This happens with other tracks I search too.
I don't get it! I've tried other NPM packages like ytsearch with no luck either!
Is there anyway to fine tune this or a better alternative?!
You can use the REST API https://www.googleapis.com/youtube/v3/search and pass some parameter to the API call.
The parameters are q - that defines the artist name or album name, key - a key is generated by making google project use that key and the last parameter is part - part parameter in the request specifies which portions of the resource are to be included in the response. For knowing the details like publish date, channel id etc you can pass snippet in your part parameter.
For more details visit - https://developers.google.com/youtube/v3/sample_requests
I want to start accepting Bitcoin on my website.
In order to do that, I wrote the following piece of code, but I truly struggle to understand how I can implement proper business logic after that the transaction is completed.
Here is the code:
<html>
<head>
<title>Pay with Bitcoin</title>
<script>
//Gets the URL of the Webpage and gets the price value of this transaction in USD.
//For simplicity Here the Value is passed in the URL.
//However in production you wanna use POST instead of GET.
const myUrl = window.location.href;
const url = new URL(myUrl);
const usdPrice = url.searchParams.get("price");
//This is the function where all the magin happens
const showQR = () => {
//URL of the api which will provide us with current BTC exchange rate
const apiUrl = "https://blockchain.info/ticker";
const hr = new XMLHttpRequest();
hr.open('GET', apiUrl, true);
hr.onreadystatechange = function(){
//Make sure the API sent a valid response
if(hr.readyState == 4){
let ticker = JSON.parse(hr.responseText);
//Get last BTC/USD exchange value from the API , then convert Price from USD to BTC
let BTCprice = ticker.USD.last;
let btcToPay = usdPrice / BTCprice;
//Make sure you have just 8 decimal points in your BTC price!!
btcToPay = btcToPay.toFixed(8);
//Use google API (or other...) to create the QR code. Pass on your btc public address and
//the amount (btc price) dynamically created. Message and label parameters can be dynamic too.
let qrurl = "https://chart.googleapis.com/chart?chs=250x250&cht=qr&chl=bitcoin:1BAnkZn1qW42uRTyG2sCRN9F5kgtfb5Bci?amount="+btcToPay+"%26label=CarRental%26message=BookingID123456";
//Populate the 'btc' DIV with QR code and other info...
document.getElementById('btc').innerHTML = "<img src=" +qrurl+"><br> <span class = 'greenMoney'>" + usdPrice + " usd / " + btcToPay + " BTC </span>";
}
}
hr.send();
};
</script>
</head>
<body onload = "showQR()">
<h1>Pay with BitCoin</h1>
<div id = "btc">
</div>
</body>
</html>
This code does the following:
Gets current USD/BTC exchange rate using the blockchain API.
takes the price in USD for the URL and converts it into BTC
generates a QR code using google API.
Embeds the price, label and message into the QR code
Renders the QR code in a DIV
I ve also set up a web hook service which will be listening to new transactions happening in the specified wallet address. Then a callback to my server is made, by mean of a POST request.
The problem is: the label and message parameters passed to the QR code will not be written in the blockchain.
They are just a handy reference for the customer to remind him what that specific transaction paid for.
As a result the callback to my server is practically useless.
In fact, the callback doesn't return any Booking Id or any other piece of information which could help me to understand who paid for what. Needless to say, in this scenario no business logic is possible: I can't update the order status on my DB, I can't send a confirmation email to the right customer.
How can I embed relevant information (e.g. Booking ID) into the BTC payment, ideally through the QR code?
If this is possible, how can I retrieve this information later on when my server receives the callback informing me that a new payment was made to my BTC wallet?
In short, you can't.
When accepting payments, you are supposed to give each invoice a new BTC address. This way, when you receive notification of an incoming transaction, you can check the receiving address to see which invoice is being paid, and compare the received amount against the expected amount.
Note
Technically, you could embed stuff like a order ID into an OP_RETURN. However, most wallets don't support transactions like that, and any users who want to pay you from an exchange account would be unable to comply.
#Raghav Sood thank you for your input which routed me to the right direction.
Using NodeJS/Express/MongoDB in the backend, I managed to implement a solution which I would like to share here.
Before starting, I wanna make a big disclaimer: this solution is not the only one, it is not the best one, it is not the fastest and probably it is not the most elegant.
Anyway, this solution has the advantage of not relying on packaged third parties solutions. This is in line with the spirit of the whole "no intermediation" philosophy of the bitcoin community. Most imortantly, your XPub always stay in your server and is NOT shared with any external service, which is probably the wisest approach.
Having said that, here is how one can show dynamic unique BTC addresses to customers:
First of all , I put in place a counter which keeps track of how many btc addresses were created for customers from a my HD wallet.
This is important to make sure than you never present the same address twice to customers, which is good for privacy of all parties and also for the sake of implementing business logic in your app.
In order to do this, I store a "counter value" into my DB. Everytime someone visits the BTC payment page, this value is retrived from mongo using a "dealCount" function and is assigned to a "serialPay" variable, which is equal to the value gotten from Mongo + 1. In the backend, the code would be something like this:
`function dealCount(){`
return new Promise(function(resolve, reject){
Deal.find({_id: "ID_OF_OBJ_WHERE_YOU_STORE_COUNTER"}, function(err, data){
if(err){
console.log(err);
}
resolve(data[0].serialDeal + 1);
})
})
};
The new value obtained (which later on will be saved again into Mongo in order to keep track of addresses created) is used to generate the new BTC public address for the customer at hand. If you keep reading you will see how.
To create new public addresses dynamically, one needs the xPub Key of his or her HD Wallet. If one is coding in NodeJS there are a couple of libraries (which can be imported into the server) that will enable this operation rather easily: bitcoinjs-lib and/or bitcore-lib. Personally I opted for Bitcore-lib, because there are less dependencies to deal with and I found the supporting material easier to digest.
Codewise, address generation goes as follows:
const bitcore = require('bitcore-lib');
app.post("/pay.html", urlencodedParser, function(req, res){
let serialPay = dealCount();
serialPay.then(function(serialPay){
const pub = new bitcore.HDPublicKey('INSERT_HERE_YOUR_XPUB_KEY');
let derivedHdPk = pub.derive('m/0/'+serialPay);
let derivedPk = derivedHdPk.publicKey;
let myDynAddress = new bitcore.Address(derivedPk);
res.render('pay', {myDynAddress: myDynAddress});
});
});
Then, using EJS as a templating engine, I could easily make the receiving bitcoin address dynamic in the front-end (/pay.ejs):
let myDynAddress = "<%=myDynAddress%>";
let qrurl = "https://chart.googleapis.com/chart?chs=250x250&cht=qr&chl=bitcoin:"+myDynAddress+"?amount="+btcToPay+"%26label=CarRental";
This will generate the QR Code Dynamically. In the original question, one can see how to render that into the webpage. In the meantime one should also put in place a function to store the updated "serialPay" counter back to the DB.
At this point one should only start monitoring incoming (non-confirmed) payments to the dynamic BTC address generated. A simple way to do it, is using the blockchain.info websocket API. When the payment arrives, things go forward as suggested by #Raghav Sood: one checks the incoming transaction making sure the customer paid the right amount to the right address.
Now you know who paid for what and all sorts of business logics can be triggered.
I'm making a web scraper Node.js app that harvests job description text from various urls.. I currently have an array of job objects named jobObj and the code cycles through each url, makes a request for html, loads using cheerio module then finally makes a new object with jobName and jobDesc keys and pushes it onto a new array of objects that is then written as a json file....
All this currently works however the completeness of the written json file is very random and usually only contains one complete job account. I thought this may be due to the forEach loop completing much quicker than the asynchronous Request function thus resulting in execution of the fs.writefile before request callback is completed. I've added a counter to monitor at what stage the requests are at and only write the json file once counter===jobObj.length but still the json file is not fully complete.
I'm new to node.js if someone could please point out my error it would be greatly appreciated!
var jobObj = [
{
id:1,
url:"https://www.indeed.co.uk/cmp/Daffodil-IT/jobs/Lead-Junior-Website-Developer-59ea7d446bdf1253?q=Junior+Web+Developer&vjs=3",
},
{
id:2,
url:"https://www.indeed.co.uk/cmp/Crush-Design/jobs/Middleweight-Web-Developer-541331b7885c03cf?q=Web+Developer&vjs=3",
},
{
id:3,
url:"https://www.indeed.co.uk/cmp/Monigold-Solutions/jobs/Graduate-Web-Software-Engineer-a5787dc322c0ca36?q=Web+Developer&vjs=3",
},
{
id:4,
url:"https://www.indeed.co.uk/cmp/ZOO-DIGITAL-GROUP-PLC/jobs/Web-Developer-5cdde1c3b0b7b8d0?q=Web+Developer&vjs=3",
},
{
id:5,
url:"https://www.indeed.co.uk/viewjob?jk=9cc3d8c637c41067&q=Web+Developer&l=Sheffield&tk=1cf5di52e9u0ocam&from=web&vjs=3",
}
];
app.get('/myform', function(req, res){
res.send("<h1>" + `scanning ${jobObj.length} urls for job description text` + "</h1>");
//make assign input form data to node "url" variable
//Compnonents for a request counter
var jobs = new Array;
function scrapeFinished(){console.log("all websites scraped!");};
var itemsProcessed = 0;
jobObj.forEach(function(item){
request(item.url, function(err, res, html){
if(!err){
var $ = cheerio.load(html);
var newJob = new Object;
$('#job_summary').each(function(){
var data = $(this);
var textout = data.text();
newJob.jobDesc = textout;
});
$('.jobtitle').each(function(){
var data = $(this);
var jobtitle = data.text();
newJob.jobName = jobtitle;
});
jobs.push(newJob);
itemsProcessed++;
console.log(item.url + " scraped");
if(itemsProcessed === jobObj.length){
scrapeFinished();
fs.writeFile('output.json', JSON.stringify(jobs, null, "\t"), function(err){
if(!err){console.log("output.json file written")}
})
}
}
})
})
})
And finally this is what I get on fs.writefile
[
{},
{
"jobDesc": "We are a successful design and digital agency that works with some great clients on a wide range of digital projects.We simply need more developers to join our great team to deliver even more great work.The projects we work on are all php based, typically built using WordPress, Laravel or flat html.We are seen as a premium agency because of the quality and complexity of the work we do.That means you will have to do more that just manipulate a theme - you will have to code. But you will be given the space, time and support to do so.We want you to be proud of the work you do, because the reputation of the agency need you to be.Key skills we will want you to bringCSS (CSS3) & HTMLAt least some knowledge of MySQL and JavaScript.At least some knowledge PHP (seniors will be tested)PhotoshopWhat you will want that we can giveA good place to work with a friendly teamA chance to develop your coding craftA decent range of projects to challenge yourselfA senior developer on hand to coach and adviseA successful company with an optimistic outlook, growth plans and a secure futureExactly how much experience you have can vary, but you must have some. And the more experience you have, the more we will pay you.We are based in offices we own in the centre of Chesterfield will two staff that do the short commute from Sheffield.If you think this job sounds interesting, we would love to hear from you, please apply!(though not agencies please)Job Type: Full-timeSalary: £22,000.00 to £30,000.00 /yearExperience:development: 2 years",
"jobName": "Middleweight web developer"
},
{},
{},
{}
]
forEach run in synchronous fashion so no matter how much time it will get to scrap the webpage, it won't run for the next array item. What can cause trouble here is your code maybe pushing an object in the array before scrapping the webpage. This is why you are getting an empty object. What you do is to push object after your program finishes visiting the URL.
Do something like this
jobObj.forEach(function (item) {
var newJob = new Object;
request(item.url, function (err, res, html) {
// Scrap URL and save values in newJob
});
jobs.push(newJob);
});
if your code still pushes an empty object before completing the request then consider using Async module.
How can i insert a loop and view for one or more jpg screenshot-urls from json-variable "screenshotUrls"?
$.getJSON("https://itunes.apple.com/lookup?id=343200656&callback=?", function (data) {
var icon = document.getElementById("icon");
var name = document.getElementById("name");
var description = document.getElementById("description");
icon.src = data.results[0].artworkUrl100;
name.innerHTML = data.results[0].trackName;
description.innerHTML = data.results[0].description;
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div class="appicon"><img id="icon"></div>
<h3 id="name">.</h3>
<p id="description">.</p>
When I call that URL ( https://itunes.apple.com/lookup?id=343200656&callback=? <-- make sure to copy that ? too, becuase SO removes it from the hyperlink, probably because it is invalid. BUT it is included in the original URL in the script... ) , this is the response:
{
"errorMessage":"Invalid value(s) for key(s): [callback]",
"queryParameters":{"output":"json", "callback":"A javascript function to handle your search results", "country":"ISO-2A country code", "limit":"The number of search results to return", "term":"A search string", "lang":"ISO-2A language code"}
}
If I were you, I would start there before looking any further.
In the meantime (We are having a little discussion about the trailing ? after the callback=? in the original URL. That gives an error for me). I use the one without the last ? in the URL. (It is also invalid to put two ? in the url)
How to use the response?
Symply do something like this:
var myResp = data;
Next use myResp as an object with properties (name/value paisr)
For example:
myResp.resultCount <-- will return 1
The response looks like the following to me:
{
"resultCount":1,
"results": [
{
"screenshotUrls":["http://a4.mzstatic.com/us/r30/Purple5/v4/37/00/46/37004662-657e-1964-a0dd-1266ddf6e096/screen320x320.jpeg", "http://a4.mzstatic.com/us/r30/Purple69/v4/49/16/ba/4916ba99-3c0f-1b18-b873-a509e8b7a5e3/screen320x320.jpeg", "http://a2.mzstatic.com/us/r30/Purple7/v4/4e/0b/56/4e0b56da-8dcd-8be2-d07a-64631ff45295/screen320x320.jpeg", "http://a4.mzstatic.com/us/r30/Purple7/v4/e6/16/94/e6169414-818b-a86c-f37c-4bad6a9f1f6f/screen320x320.jpeg", "http://a1.mzstatic.com/us/r30/Purple69/v4/9b/a1/6f/9ba16f4f-99bd-36a2-fccd-c3edfa181ff3/screen320x320.jpeg"], "ipadScreenshotUrls":[], "artworkUrl512":"http://is3.mzstatic.com/image/thumb/Purple5/v4/80/28/4c/80284c71-3c6b-470c-00a3-b7129e4ab814/source/512x512bb.jpg", "artistViewUrl":"https://itunes.apple.com/us/developer/rovio-entertainment-ltd/id298910979?uo=4", "artworkUrl60":"http://is3.mzstatic.com/image/thumb/Purple5/v4/80/28/4c/80284c71-3c6b-470c-00a3-b7129e4ab814/source/60x60bb.jpg", "artworkUrl100":"http://is3.mzstatic.com/image/thumb/Purple5/v4/80/28/4c/80284c71-3c6b-470c-00a3-b7129e4ab814/source/100x100bb.jpg", "isGameCenterEnabled":true, "kind":"software", "features":["gameCenter"],
"supportedDevices":["iPhone-3GS", "iPhone4", "iPodTouchFourthGen", "iPad2Wifi", "iPad23G", "iPhone4S", "iPadThirdGen", "iPadThirdGen4G", "iPhone5", "iPodTouchFifthGen", "iPadFourthGen", "iPadFourthGen4G", "iPadMini", "iPadMini4G", "iPhone5c", "iPhone5s", "iPhone6", "iPhone6Plus", "iPodTouchSixthGen"], "advisories":[], "languageCodesISO2A":["EN", "FR", "DE", "IT", "JA", "PT", "RU", "ZH", "ES", "ZH"], "fileSizeBytes":"71674372", "sellerUrl":"http://www.angrybirds.com/", "averageUserRatingForCurrentVersion":4.0, "userRatingCountForCurrentVersion":1195, "trackContentRating":"4+", "trackCensoredName":"Angry Birds", "trackViewUrl":"https://itunes.apple.com/us/app/angry-birds/id343200656?mt=8&uo=4", "contentAdvisoryRating":"4+", "minimumOsVersion":"6.0", "formattedPrice":"$0.99", "currency":"USD", "wrapperType":"software", "version":"6.0.1", "artistId":298910979, "artistName":"Rovio Entertainment Ltd", "genres":["Games", "Arcade", "Entertainment", "Action"], "price":0.99,
"description":"Use the unique powers of the Angry Birds to destroy the greedy pigs' defenses!\u2028\u2028\n\nThe survival of the Angry Birds is at stake. Dish out revenge on the greedy pigs who stole their eggs. Use the unique powers of each bird to destroy the pigs’ defenses. Angry Birds features challenging physics-based gameplay and hours of replay value. Each level requires logic, skill and force to solve.\u2028\u2028\n\nIf you get stuck in the game, you can purchase the Mighty Eagle! Mighty Eagle is a one-time in-app purchase in Angry Birds that gives unlimited use. This phenomenal creature will soar from the skies to wreak havoc and smash the pesky pigs into oblivion. There’s just one catch: you can only use the aid of Mighty Eagle to pass a level once per hour. Mighty Eagle also includes all new gameplay goals and achievements!\u2028\u2028\n\nIn addition to the Mighty Eagle, Angry Birds now has power-ups! Boost your birds’ abilities and three-star levels to unlock secret content! Angry Birds now has the following amazing power-ups: Sling Scope for laser targeting, King Sling for maximum flinging power, Super Seeds to supersize your birds, and Birdquake to shake pigs’ defenses to the ground!\u2028\u2028\n\nHAVING TROUBLE? Head over to https://support.rovio.com where you can browse FAQs or submit a request to our support flock!\n\n#1 IPHONE PAID APP in US, UK, Canada, Italy, Germany, Russia, Sweden, Denmark, Finland, Singapore, Poland, France, Netherlands, Malta, Greece, Austria, Australia, Turkey, UAE, Saudi Arabia, Israel, Belgium, Norway, Hungary, Malaysia, Luxembourg, Portugal, Czech Republic, Spain, Ireland, Romania, New Zealand, Latvia, Lithuania, Estonia, Nicaragua, Kazakhstan, Argentina, Bulgaria, Slovakia, Slovenia, Mauritius, Chile, Hong Kong, Pakistan, Taiwan, Colombia, Indonesia, Thailand, India, Kenya, Macedonia, Croatia, Macau, Paraguay, Peru, Armenia, Philippines, Vietnam, Jordan and Kuwait. \u2028\u2028\n\n#1 IPHONE PAID GAME in more countries than we can count!\n\nTerms of Use: http://www.rovio.com/eula\u2028\nPrivacy Policy: http://www.rovio.com/privacy\u2028\n\nThis application may require internet connectivity and subsequent data transfer charges may apply.\n\n\nImportant Message for Parents\n\nThis game may include:\n- Direct links to social networking websites that are intended for an audience over the age of 13.\n- Direct links to the internet that can take players away from the game with the potential to browse any web page.\n- Advertising of Rovio products and also products from select partners.\n- The option to make in-app purchases. The bill payer should always be consulted beforehand.", "trackName":"Angry Birds", "trackId":343200656, "bundleId":"com.clickgamer.AngryBirds", "releaseDate":"2009-12-11T08:00:00Z", "primaryGenreName":"Games", "isVppDeviceBasedLicensingEnabled":true, "currentVersionReleaseDate":"2015-12-11T08:02:04Z", "releaseNotes":"The #1 App of all time turns 6!\nJoin the celebration in 15 all-new levels in the BirdDay episode!", "sellerName":"Rovio Entertainment Ltd", "primaryGenreId":6014, "genreIds":["6014", "7003", "6016", "7001"], "averageUserRating":4.5, "userRatingCount":822974}]
}
Now, how do you want to proceed?
EDIT: You want all screenshot url.
You can go there with something like:
myResp.results[0].screenshotUrls <-- will be an array.
I haven't checked the code, but just inspect the JSON response, and try to go to the desired element, paying attention to the craziness of the response, that is a mix of object and array notation.