We Keep Coding

Trouble accessing Instagram post caption in Python Selenium

Good afternoon:
I'm currently attempting to access the description (caption) section of certain instagram posts using python selenium. For instance, in the picture below it refers to the part that starts with the word "Thanks" and that goes all the way down to the hashtags.
Well, it caught my attention the following things:
Consider this post (https://www.instagram.com/p/CNYQkxADO8p/). When the inspector is checked, we can see that the comment is inside a class call "Edited" so we can retrieve all the text with the following code:
desc = driver.find_element_by_xpath("//span[#title='Edited']").text
The code above works well for the mentioned post. However, consider this second post: (https://www.instagram.com/p/CN5dOopjNEk/). The description seems to be located in a different HTML class, and the line of code above throws a NoSuchElementException. I was therefore wondering how can I access the caption of that post, considering that it is not located in a section by the title "Edited". How can I access said part of the post?
Thanks for your attention

For that specific post you can try using:
desc = driver.find_element_by_xpath("//div[#class='C4VMK']/h2/following-sibling::span")
print(desc.text)
Since there is no direct way to access that element, we drill down from the class to h2 and then take the next element sibling of span, which is where the text resides.
I am not sure why it would be different from the other posts that have //span[#title='Edited'], but it seems to be.

How to update HTML after page is loaded?

I have a script that takes text from a hidden div:
var content = column.find(".hidden").text().split('');, and splices it with random 1s and 0s to give a decryption effect. The script writes to another div: column.children(".code").text(content);
So I'm ripping text from .hidden, turning it into an array, splicing it (within setInterval() so it's animated) with a binary array, and putting it into a visible div. The problem here is I can't get any HTML inside the visible div, so I can't include linebreaks or any other useful things. I tried using .html() instead of .text(), but then it puts html into the visible div, instead of actually formatting the page with it. I tried to .append() the changes, but it never even showed up because I probably did it wrong. I was told I may be able to do this through node manipulation, but nothing I tried worked.
So the question is: How can I update the DOM so it shows the final HTML, change the text without destroying the html, insert properly formated HTML at the end, or some other solution I can't think of because I have no idea what I'm doing.
Edit: Here is a relevant fiddle. http://jsfiddle.net/13t31w5p/

The correct syntax to add content to an existing element is:
/**
* .append()
* {#link http://api.jquery.com/append/}
*/
$('div.mydiv').append('this text will be added at the end of the div. ')
Note that the .text() and .html() methods are more or less "conversion" scripts that manipulate existing content. Click the links to read more about them.
If this doesn't answer your question, please provide a bit more of a verbose example of what you're doing -- perhaps you could include the relevant code snippets and put them in something like JSFiddle or Gist.

How To find out what is hiding/acting on a html element

I have a simple <a> tag that is getting hidden from some JS. (display:none)
I have looked into the page source and I can see it is not hidden, however the inspector shows it as display:none ( inline style )
No result finding out the class/id in JS code in order to isolate the part of the code that is hiding the <a>.
Is there a tool or fixed procedure that can help me to debug this?

Chrome lets you break code when an attribute on an element is changed.
Open the developer tools using F12, and select the "Elements" tab. Find the element that gets hidden. Right click on it, "Break on", "Attributes Modification".
Refresh the page, keeping Developer Tools open.
If the element is being hidden using JavaScript, you'll break at that point.
Otherwise, it's done via CSS. If this is the case, using the "Elements" tab again, select the element you're interested in and look at the "Styles" applied to it in the right hand column. Chrome will show which styles are applied by which definition in which stylesheet. It should then be trivial to find the one hiding the element.

First of all, look if it is hidden from inline style or by a css class.
Then, if is by a css class, search in all your project for that class (you should find a javascript function that adds this class to the element).
If is hidden by inline style property, look inside your project for any .style.display = property.
If you are using jquery try like this:
// Search by class
.addClass(".hiddenClass
// Search by css
.hide(), or $(".elementSelector").hide()

Firstly make sure that it is indeed javascript that hides your element, as it could easily be css. The easiest first step is to check the element and see if by any chance its default css is hiding it.
Second. Is your js code in one file or do you import multiple js files in your page?
If you have multiple js files you could try.
Import 1 file
then use javascript to Show your element
then import the rest of the files.
If the code that hides your element is located in the first file then your element will be visible (because you made it visible after hiding it first)
if the element is not visible it means that the hiding takes place in a subsequent file.
Move your javascript code showing the element after the second import and so on...
Last but not least make sure your code does not import external css files as well.
I recommend using Chrome Dev Tools for any javascript debugging you do.

How to 'unrender' MathJax equations?

We all know that MathJax renders elements on window onload by default (and can be refreshed using MathJax.Hub.Queue(["Typeset",MathJax.Hub]); Reference Link), but is there a way to 'unrender' the elements? So for example, after the page just loaded, I can click a button, and all the elements will turn back into their TeX code. Is that possible?

Well, the original TeX code is stored by MathJax, so you can use some javascript to put it back. There is an example of how to do this on the MathJax users forum that I think may be what you are looking for.

It seems that MathJax hides the original LaTeX code in an element called MathJax-Element-x, which contains the original code. So what I did was simply hide all elements with the class MathJax_Display, in which the formatted version appeared, and showed all elements starting with MathJax-Element-. Seemed to work fine.
EDIT: Instead of selecting MathJax_Display elements, I had to select all the elements inside MathJax_Display, like MathJax_Display *. I also had to change the type of element the LaTeX code was put in, as it was in a script.

Dynamically creating and loading a dynamic page into a browser window

I hope I can explain what I would like to do a bit better than how I did with the question's title :)
I have a page whose content is dynamically generated. I would like to:
a- grab certain divs out of it,
b- wrap them with the markup of a standard html page (html, head, body, ...)
c- load the result into a separate browser window
I can do a & b but not sure how to do c. Any thoughts on how this can be done would be appreciated.
thanks

You can open an "empty" html document in a popup window, and then add the elements to that document. So, rather than creating a document and then displaying it, display an empty shell, and then fill in the content.
Found this tutorial after some quick googling: http://www.openjs.com/tutorials/advanced_tutorial/popup.php
(just don't use global variables like they do)