How can I automatically retrieve texts of emails and further scrape them? - javascript

My business has hundreds of incoming emails daily and my plan was to have the sorting and answering at least partly automated. I know that using JavaScript it would be possible to select those elements on the webpage (i.e. in my inbox) that are email tabs but, as far as I'm concerned, I can't implement cursor movement and clicking in JavaScript to open up the emails one-by-one and copy-paste their contents into a separate file. I want to collect and analyze the texts from incoming emails, classify them based on topics using a large set of keywords, and, once the grouping is finished, assign sample answers to these messages that only have to be proofread and then can be sent out.
My idea was to use Python because it is quite convenient to move the cursor in Python. However, I can't seem to figure out how can I analyze information that is currently on the screen, so that the program can "see" if there are any new emails. In JavaScript this seemed easier, I don't know if it is even possible using Python though.
I am using Windows.
Am I on the right track? Or totally wrong? Maybe I should consider another programming language? Thank you for your insights in advance.

As far as i understand you need to automate the functionality of collecting information in emails to a separate file for further processing. For this I think you can use Selenium Web automation tool (Python) . It is normally used for web site testing. But can be used in use cases like you mentioned. Hope this helps.
https://selenium-python.readthedocs.io/
https://pypi.org/project/selenium/

Related

How to create a template creator with moveable objects?

Good day,
I am currently working on an automatic system that generates invoices with data that it receives from an API. I am currently doing this with Django (Python) to create some variation. Well, I want to create a system with which you can easily create templates for these invoices. You have seen these kinds of systems before. You can move blocks with items such as a logo or text wherever you want. Well I know that these templates are further stored as HTML. Only nowhere I can find clear information about how I can easily assemble such a system or how such a system works. Below I show a GIF of the system what I want. If anyone has any information on this I will be very happy if you can share that with me :)
Only nowhere I can find clear information about how I can easily assemble such a system
Yes, because making such a system is not an easy feat. You shouldn't do it yourself unless you know what you're doing and you are ready to deal with a lot of edge cases.
That being said, there are libraries that enable you to create such interfaces. One being https://interactjs.io/ (not affiliated with them). Then you need a WYSIWYG/Markdown editor that can be enabled on click as a tooltip. For example, https://www.tiny.cloud/
Then you need to find a way to save and load everything. Depending on the library you use, you might be able to get away saving and loading the HTML. However, it's more likely you'll need to implement a proprietary way of saving data. For example, using JSON or XML.
Best of luck!

How to change html content with URL?

I'm a beginner at coding, I know javascript but not super advanced objectd,
I'd like to know how to change html content with its URL. For example,I am on a website like GMAIL, it has different page of registring and logging in. These two pages have different URLs.
What I'd like to know is how do they change the URL along with HTML when I click on the button "Log in". Is this possible through server-side like node.js and express, or just with front-end javascript?
One last thing, do websites have multiple web pages or it's just in one single HTML file?
Well, I have set up a practice project, but I don't know what I am doing.
I changed HTML content with jQuery library but I don't know how to change URL.
First I made a homepage with some text and two links to two forms.
I showed registration form when click on "Sign in", and log in form on "Log in", and hid the homepage with the show() and hide(). The URL doesn't change in order to work with it with express. I tried it with history.pushState() but it messed up things: I can't return to homepage, and it didn't change the URL i wanted based on the form. So i deleted it, and I am stuck and don't know I could find some tutorials online.
My code doesn't contain anything other than what I described.
So, please can you explain to me how websites do that.
And one other thing, my express server now is very slow, it takes nearly 5min to start. I don't know if it's because my pc which is old and not super good unfortunately.
Can you please advice me with some tutorials and tips?
I agree that your question is too broad. Even there is many years invested in unversity to know these stuff well, I believe in self learning, so I will give you some light for your next steps in this world.
Here are some questions you may ask Google or research where ever you want:
There's both applications that hosts entire html documents in a server and reacts to http requirements responding with different ones. These are the first ones in existence.
Today the trend is to host information on distributed servers (Even cloud) as services to interact with just as information repositories, and entire client side applications that handles that information to show to the user in a more interaction friendly way.
So here are 4 first questions you can ask:
How does HTTP protocol works (with html documents e.g.)?
What's the difference between thin client and fat client applications?
What are web services?
How can I do a simple client side application with different routes using a public web service?
There is a lot of information to read about, and that's not the way I learned in university, so I can not tell you that's the right way or even a good one. Anyway, you should consider taking a web programmer beginner course, if you already know about basic algorithmic composition.
Wish you the best in this extensive path...

searching a large amount of text using javascript and html5 storage

I have a web app that relies on html5 offline storage features so that it can be accessed by the user without an internet connection. The app essentially just serves html pages and a little bit of css and javascript.
I am trying to add the ability to search the text served on these pages for key words, but because the app isn't guaranteed access to the server it needs to be able to perform these searches on the client side.
My thought is I can store the searchable text in the browser's web sql database and perform the search either through javascript or through the browser's sql api. I have a few question about the best way to do this:
1) I vaguely remember an article about how to implement something like this, maybe from airbnb? Does anyone remember such an article?
2) The text is 2,000,000+ words so I would assume that indexOf is going to break down at this data size. Is there any chance regex will hold up? What are some options for implementing the actual search? (libraries, algorithms, etc.) Any article suggestions for understanding the tradeoffs of string search algorithms if I need to go down that road?
Well, I just wrote a quick benchmark for you and was surprised to find that you could probably get away with using String.indexOf(). I get about 35ms per search, which is about 30 searches per second.
EDIT: a better benchmark. There appears to be some sort of initialization delay, but it looks like indexOf is pretty fast. You could play around with the benchmark and see if it looks like it will work for you.

Google Maps JavaScript Files on Server Side

I wonder whether someone may be able to help me please.
I've been working on a Google maps application that will allow users to geocode and reverse geocode from the information they provide on a input form.
As it stands at the moment I have separate HTML and Javscript files, and, with thanks to some of the people on this forum they work fine.
However, I now would like to run these from my SQL server. Because I'm very new to programming in general and especially that around Javascript and google maps I naively thought that it would be a simple job of putting the relevant Javascript, HTML and PHP files on my server and everything would work.
How I was wrong!
But having looked at the web there seems to be a number of solutions to this problem.
As I said earlier this is very new to me, so I just wondered whether someone, with a bit more knowledge of this could point me in the right direction please, and whether indeed, it is better to amalgamate the Javascript and HTML files rather to perform some sort of conversion to enable me to use the Javascript files.
Many thanks
Chris
I'm not sure of your level of understanding, so I'm just going to start at the top and I apologize for covering something you already understand.
When you create your HTML and Javascript, let's assume it's in the same file for now. Your web application is going to get hit, and your server is going to take the request and serve the page up to the client. This will be in the form of the same HTML and script you typed into your IDE. Each inidividual browser is going to interpret that, format it, and basically build the page up to the user.
All of the javascript comes along for the ride, and is hooked up client side. Note that if your javascript is in a different file, it still comes along for the ride, as long as it's referenced correctly by your page. At this point your job is done. They can play with your page even without internet access, as long as they don't do anything that requires a post back to the server.
This means that as long as you can get your javascript to the user, your job is done.
In short, when you say you want to run javascript on your SQL server, we aren't sure what you're implying. There is no reason to run javascript from a SQL box. Do you want the SQL server to also serve your web page? Or do you have some code in javascript that you want to run over a dataset?

What precautions should I take before I let client add javascript to a webpage?

Question: What precautions should I take when I let clients add custom JS scripts to their pages?
IF you want more details:
I am working on a custom CMS like project for a company, The CMS has number of "groups" that each subscriber "owns" where they do their own thing.
The new requirements is that some groups want to add google analytics to see how they are doing. So I naturally added a column in the table and made code adjustements so if there is some data in that column, I just use the following line in master page to set the script out:
ScriptManager.RegisterClientScriptBlock(Page, typeof(Page), "CustomJs", CustomJs, true);
It works just fine, only, It got me thinking...
It's really easy for someone with good knowledge of how to access cookies etc from from js. Sure, each group is moderated and only super admin can add this javascript, sure, they wouldn't be silly enough to hack their own group. Each group has their own code so its not possible to hack other groups BUT STILL
I am not really comfortable in letting user's add their own javascript codes.
I could monitor each group myself, but the groups are growing really quick and I will hit a time when I will no longer be able to do that.
So, to brief it up: What precautions should I take to avoid any mishaps ?
ps: did try to google, no convincing answers anywhere.
Instead of allowing the users to add their own Javascript files, and given that the only requirement here is for google analytics, why not just let them put their analytics ID into the CMS and if it's present, output the relevant Google Analytics code?
This way you fulfill the users requirement and also avoid the need to protect against malicious scripting.
Letting users use Javascript is in general, a very bad idea. Don't do it unless you have to.
I once I had a problem where I need to let clients use Javascript, but, the clients weren't necessarily trusted, so, I modified cofeescript so that only a small subset was compilable to javascript, and it worked pretty well. This may be waaaay too overkill for you.
You should not let your users access cookies, that's always a pain. Also, no localStorage or webSQL if you're one of the HTML5 people, and, no document.write() because that's another form of eval as JSLint tells you.
And, the problem with letting people have javascript is that even if you believe you have trusted users, someone may get a password, and you don't want that person to get access to all the other accounts in the group.
Automatically recognizing whether some JavaScript code is malicious or sandboxing it is close to impossible. If you don't want to allow hacking your site you are left with only few options:
Don't allow users to add JavaScript at all.
Only allow predefined JavaScript code, e.g. for Google Analytics.
Have all custom JavaScript inspected by a human before it is allowed to display on the site. Never trust scripts loaded from third party sites - these can change from one day to another and turn malicious.
If you have no other choice, you may consider separating path/domain of user javascripts (and cookies).
For example your user have page:
user1.server.com
and you keep user pages at
user1.server.com
So, if you set session cookies to the user1.server.com, it'll render them unobtainable for user scripts from other domains (e.g. user2.server.com).
Another option may be executing all user's javascript at server JS engine (thus controlling all it's I/O and limiting access to browser resources).
There is no simple and easy solution anyway, so better consider using options from other answers (e.g. predifined script API, human inspection).

Categories