Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 1 year ago.
Improve this question
I explain my problem to you:
When I do PHP curls on some site or want to display the source code of the page element is missing a lot. I think some part is called by a script or something. Could someone help me view the entire code with Curl PHP.
To duplicate my problem go to Facebook or LinkedIn and right click on the page and "View the source code of the page", in this you don't see all the page content but when for example you right click and "inspect an element" You can.
Thank you in advance
CURL can't do this. It's not designed to render HTML or execute JavaScript.
A lot of the content on Facebook, LinkedIn, Twitter and many other pages is loaded through different ways. (like fetch()-requests or WebSocket-Events)
Some nodes you can see in the inspector are not part of the original document (which you are viewing with "view source" or curl downloads). What you see on the inspector is everything currently held in memory, which was partially (or completely) created with a scripting language.
This is basically done to
reduce the load on servers as it doesn't have to generate the whole page on every request
reduce traffic on clients and servers (no need to reload the header-data and/or scripts over and over again)
If you need data from a rendered site, you should either check if the website provides an API which gives you the data you are looking for or use one of the cli-rendering-engines from this answer.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I need to copy data (each field has different ElementIds) from site A and paste to site B (similar to a login page). In the past I used ctrl+c and ctrl+v to repeatedly copy paste information between two sites. The structures of site A and site B always keep the same but the context in site A changed according to different people.
I have been thinking about creating a chrome extension to copy the data from site A and paste to site B.
How should I work on it?
[Additional Explanation]
Although I am new to JavaScript and Chrome extension, I know how to create a chrome extension and how to read the date of a certain page. The main problem is how can I pass the data I read in the site A (www.a.com/XXX) to site B (www.b.com/XXX), so I can use the data to fill in the site B?
Since I need to repeatedly do this process, I hope I can make a extension so when I click it in site A it retrieves the data. Then I click it in site B it will fill the date into the site B.
It is not for batch process and I need it can be easily used by other people.
You need to use content scripts for copy/pasting and background page/event pages for communication. Inject the first script into page A. This script will be responsible for copying data. Inject the second script to page B. This script will be inserting data into the page. Use chrome.runtime.sendMessage to communicate between the scripts through main background/event page.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Does fb pick up on keywords? For example, if you try posting this link you will see that it displays "how to ask Q&A for professional and enthusiast programmers" and then the link at the bottom. Your help is much appreciated :)
They query the link server-side, scrape it, and send that information to the browser as a widget.
You can watch it happen:
Copy a link to your clipboard.
Open Facebook.
Open your browser's Dev Tools and go to the Network tab.
Click in the Post box (so it loads the various things it loads when you do that).
Wait a few seconds.
In the Network tab, click Clear so you can see what's new.
Paste your link in the Post box.
You'll see a request for something like https://www.facebook.com/react_composer/scraper/?composer_id=obscured&target_id=obscured&scrape_url=http%3A%2F%2Fstackoverflow.com%2Fquestions%2F36964511%2Fi-would-like-to-know-how-facebook-changes-something-you-post-like-a-weblink-into&entry_point=obscured&source_attachment=obscured&av=obscured&dpr=1. The response includes the information for the box.
Facebook uses the Open Graph protocol:
The Open Graph protocol enables any web page to become a rich object in a social graph. For instance, this is used on Facebook to allow any web page to have the same functionality as any other object on Facebook.
If you implement Open Graph metatags on your webpage Facebook will use these fields when creating the widget you see after posting a link.
For example, if you view the source of this page you'll see Open Graph tags for og:image, og:title, og:description, og:type and og:url. If you share this URL on Facebook, the widget will use the og:title, og:description, og:image, and og:url. You can view this info in Facebook's Open Graph Debugger tool.
A helpful tool is the Open Graph Debugger, which allows you to post a URL and view how Facebook scrapes the page.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I want my users to be able to view content on my website, and when they see content they like (My website randomly generates content using JQuery.load() ), I would like them to be able to store it onto a personal page that they have to log in to, to see.
My page loads content with the line:
$(".resultContent").load(skillList[skillChoice]);
The variable "SkillList" is assigned "skills.html #id"
the '#id' part is identifying a <div>, and that's the part I want to be stored, and I want the user to be able to store multiple parts of the page skills.html on their personal page.
This sounds fairly complex, but I was wondering if it's doable, if so, what language(s) would I need to use?
You will need to save those html snippet on the server, mostly likely in a database. This can be done by client-side javascript, an ajax POST. The server side can use any language.
Since you are saving some html on the server and will load that in a page. The server side must sanitize the html snippets for security.
When the html snippet is saved, the server needs to know who sends the request. That is the user needs to log in before or when s/he saves the content for later use.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am very familiar with programming, MySQL, PHP, JavaScript, jQuery, HTML and CSS, and this issue of mine has nothing to do with what I just mentioned. It all boils down to three things:
Defining a menu layout with link options.
Rendering a dropdown menu on-demand with position calculated live.
Determining if the current user with current session is owner of the object clicked.
Nr. 2 is not a big problem, very simple indeed. Nr.1 & 3 are a bit problematic. And it is this part that I do not fully understand, I know that for Nr. 3 Ajax can be used to communicate with the PHP in the backend and determine ownership of an object through IDs.
As for Nr.1 should this be put in PHP or as a JavaScript AMD definition, I am not so sure about this myself.
I hope things are much clearer now.
Yes it is. Ajax can be used to load any data (html also) it depends on the programmer that what he wants to do with the data. For example, if you send a request to server to send some html, and in response it sends HTML to you then you can just simply render that html code in your DOM. It is as simple as that...
It actually depends on the requirement that you should use an ajax call or just use jquery to append or show some html.
(For simplicity and for your question):
Use Ajax:
To get data from server.
Append html using ajax if the html contains the data which needs to be get from the server.
Use Jquery:
To append/show html which is present on page ,no need of ajax call.
Like for example in above quora dropdown ,
-There is no ajax when you click on that "..." image(Although you can use ajax here but it depends on the application requirement,here quora is getting this drpdown values on page load).
-There is ajax when you click on "Make anonymuous" ,"Add to reading list" etc etc.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
can my page Javascript read same page which itself is loaded? Like other parts of page are dynamically loaded by other provider. I have tried many things, google as well, but now I am in doubt that it is posible. Or it is.
Thank You!
If the page has loaded and the javascript you are running is client-side (which it should be), you should be able to access everything on the page via the document object. I would advise reading about the DOM to familiarise yourself with this.
EDIT: removed link
Server side code (whether written in JavaScript or otherwise) is not capable of determining the final rendering of the page in the user's browser.
You could build the entire page yourself (and you could use a headless browser, like PhantomJS, to do it) but that could give different results to a visitor's as you would have a different set of cookies, a different source IP address, and so on.