Im kind of new to this and looking to expand pulling API results and displaying them on page, whether it's from a blog resource or content generation.
For example, I want to pull from VirusTotal's API to display returned content. What is the best way to capture that in an input tag and display it in a DIV. And what if it were an option to pull from different API's based on drop down selection?
An example of the API to pull content would be here https://developers.virustotal.com/reference#api-responses under the /file/report section.
To call the data from the API, you need to send a request. However, there is a problem with CORS. Basically, you can't call the website from inside your web browser from a page on your local machine, because your browser blocks the request. The web browser will only allow calls to and from the same server, except for a few exceptions.
There's two ways to approach this.
The simplest one is to make a program that calls the API and outputs an HTML file. You can then open that HTML file to read the contents. If you want to update the info, you would need to run that program once again manually. You could easily do this building off the python they provided.
The other, little bit more complex way, is where you host a server on your PC. When you go to the webpage on that server, it sends a request to the website, and then provides the latest information. There's tons of frameworks and ways to do this. For an absolute beginner on this subject, ExpressJS is a good start. You can make a hello world program, and once you do that you can figure out how to call the API whenever a page is loaded, and display the results.
Related
I'll try to explain everything the way I understand it as clearly as possible, so correct me if I'm confused with something.
I was trying to scrape the users from a member list on a website, I used Python and the first thing I did was making a post request to the Request URL with the required headers so I get a response that contains the data I need but this didn't work, so I tried to find out the reason.
From what I understand now the website uses AJAX to make XHR and JavaScript calls which respond with the content (users).
The JS code is stored on a static website from what Chrome's developer tool request initiators
tell me (Here is an image for reference), which responds with the HTML that contains the users
The idea is to create a script that runs this static JS script that's stored online and fetch the data about the users from it. (Image for clarification)
How do I achieve this, I'm using python. What libraries do I need etc.? Any help/advice is greatly appreciated!
Based on your questions, I think you're trying to load data from a website that uses AJAX to load data.
In my opinion, have a look at Scray and some Headless Browers.
Check the flowing links for more information
https://scrapy.org/
https://github.com/puppeteer/puppeteer
https://github.com/pyppeteer/pyppeteer
We have developed a website and it uses JavaScript library to query database and display the data in HTML page. When you go to the website, you need to search for something in order to retrieve the data.
so by default website doesn't display any data and it needs users to perform action.
The search result data is not visible in HTML view source as it uses JavaScript.
So, the search engines have no visibility as to what our website used for and data used in order to redirect more visitors.
Secondly, I wonder how search bots/engine crawl the websites with non-static content and understand enough about the website to redirect users.
from what i see from your question what you need to do is send requests to your server to query data from your database and show it to you client in real-time.For that i would recommend that you use web sockets(such as socket.io) or AJAX so that you could update your website seamlessly
From what I have researched, crawlers actually don't read dynamic content. Instead, they use this technique called dynamic rendering.
Dynamic rendering has to do with the server itself. It checks each request and if it determines it to be a bot, then it will send static HTML content to the bot. Otherwise, it will send normal dynamic content to the user.
Also, google and other search engines make use of meta tags. With meta tags you can define a short description of the webpage which will oftentimes be shown in the search results page.
As for question in the title, you would need to send the search information to a server. From there, you would process the data server-side and send the results back to the client where JavaScript would render it based off of the results.
You should use AJAX for this.
Resources:
https://ignitevisibility.com/dynamic-rendering-seo-details-need-know/
https://developer.mozilla.org/en-US/docs/Web/Guide/AJAX
https://developer.mozilla.org/en/docs/Web/HTML/Element/meta
I have an html page that has a form and is communicating to a server via JSON requests, when loading (e.g. in order to remember user's previous answers) and when submit button is pressed (e.g. in order to save his new answers). Every user has a specific user_id that is located as an attribute in the url of the website. I have only HTML pages, CSS and Javascript that makes some simple functions, as well as received and sends the requests.
The problem is that the server needs an api-key for the request to happen, that I need to be kept hidden, but instead is easily discovered when the user sees the source code. I want this specific line to be hidden, but I guess this is not possible without any backend in the game.
I would like to know the easiest and fastest way to get from this state (just frontend, where every piece of information in the source code is totally insecure) to another where the api-key (at least) is not on the open.
I guess the solution is to use a server for that part but I need suggestion on the easiest transition from my code to another. I tried to use GWT, as I am a bit more familiar with JAVA backend application (but not with GWT), but seems like a big trouble, where I need to change my HTML, my Javascript and also the CSS that I have may not be useful, as well as I face a lot of problems when trying to read my parameters.
I know that it is not the best way but I do not have a lot of time to make it work, so sorry if it seems lazy (I am new to this type of programming), but I haven't found anything helpful and I cannot study for 2 weeks in order to be able to begin implementing it.
Is node.js (which is Javascript and I already have implemented the request sending/receiving in this language) easier than GWT for that matter? Will my sensitive data be secure in that way? I would be grateful if there was a similar sample, that I could start using for my implementation, as I haven't find anything that is specifically helpful for my situation.
Thanks in advance!
NodeJs is not javascript, NodeJs is specific javascript "interpreter" whose is purpose is mainly to be executed server-side. If you have an HTML page, it is likely to be loaded in a web browser (client-side), so not in a NodeJs environnement.
Everything that is hard-coded in the javascript of you web page is visible from the client, there is no way around that. So yes, you need some server-ish thing somewhere.
If you are not to manage a server by yourself or via PaaS, you can go for a serverless architecture. For instance, If you consider AWS which I know the most, you can easilly add some user management to your web page using Aws Cognito User Pool. Once a user is connected and have the good permission, he can be granted access to some other resources via a JWT token that you send along with your request.
I am making a small payment system, basically it's just a point system, you pay say 1 USD and you get 100 points which is used later on in a game project to get bonuses. It's a script for game servers, something like a user panel.
Now, the script system is ready, but I'm afraid to give it away, since than someone will share it and it will spread all over the gaming area. What would be the solution keeping it working only if I give them a permission?
I thought about re-making whole code and make it work on my website but I don't think that people will want to put their SQL data to website that located NOT on their host. Please help me out, at least with some clues, maybe its possible to make some widgets? or maybe some license system?
I'm really lost.
You should implement the logic on the server side as an api REST call and include in the script only an ajax call to the api. You can limit the use of the api through an api key that you'll provide only to qualified sites.
You'd need to implement some sort or serverside authentication/api so that only varified users can use the script. Much like how software checks a licence.
On script load your javascript could make a ajax call to a server passing through the users IP, auth key, username etc etc.
This can then be varified on the server, maybe returning a dynamically generated url containing a javascript file which contains your business logic
(so that urls are dynamically generated for that users session only)
That way people cant hot link the script, and the script you give out is solely the ajax call
(With the business logic script injected on auth)
At the moment, im building a Website with Ruby on Rails.
Problem:
My Website is using different foreign API's for getting Data, for example the Amazon Product Advertising API. If i load e.g. 10 objects at once, it tooks to mutch time.
Its possible to load each object particular? (If one request finished push it with javascript on the page, or something like that) The user should be able to read the first objects while the rest of the content is loading in the background.
simple example:
list.each do |object|
result << AmazonRequest.getItem(object)
[And now push the changed result list to the view]
end
Is this possible? If yes, how?
Thanks :)
I don't really know RoR, but if you do not need to have all the different API results on the server before sending them out (which seems to be the case), you could just make multiple ajax requests and display content from the different APIs independently.