Fetching YouTube live-stream comments into a React app - javascript

I'm trying to build a React Webapp to group and display superchats while a livestream is running.
What I've tried so far (unsuccessfully) is to use the YouTube LiveChat API, however this requires authentication from the live stream owner (which I do not have)
The second thing I've tried is to use youtube-chat, however this does not work. (I ran into CORS issues: Access to XMLHttpRequest at 'https://consent.youtube.com/m?continue=https%3A%2F%2Fwww.youtube.com%2Fchannel%2FUCdQPeeJ0qGK6wWBiEJWcdsQ%2Flive&gl=NO&m=0&pc=yt&uxe=23983171&hl=en&src=1' (redirected from 'http://localhost:3000/channel/UCdQPeeJ0qGK6wWBiEJWcdsQ/live') from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.. I've proxied all requests to https://www.youtube.com. Even without the CORS issues, the library does not seem to work as it should, so I'm probably looking for another option.
The intent of the library seems to be to scrape the YouTube native HTML element and read the chat messages from there, but this library seems to be deprecated and not take into consideration the YouTube cookie-consent screen.
I was thinking of mabye using Selenium and scrape the data that way, but I'm not sure if that would work. Any help on this issue would be greatly appreciated.

CORS will block any requests from your website to another website, that didn't authorize this. To solve this, write a small backend server, that uses youtube-chat to fetch your comments and relays it to your frontend. That way, you bypass CORS.

Related

Youtube Api Upload Request Blocked By Cors [duplicate]

For several years we have successfully been uploading videos via the YouTube API using some custom JavaScript code. The code was based on some samples provided by Google (cors_upload.js). It's not something we use a lot, just every couple of weeks.
Things were working fine a couple weeks ago, but it has come to my attention that things recently stopped working. We login fine, we obtain the channel info fine. But when we start the upload (which happens via XHR POST), we are getting a CORS error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://www.googleapis.com/upload/youtube/v3/videos?part=snippet%2Cstatus&uploadType=resumable. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).
But we haven't changed anything in our code or on our server and it appears as if we are doing all the things necessary as documented by Google to have their service respond with the required CORS headers.
According to the dev console, the XHR request actually generates 2 network requests. First I see an "OPTIONS":
Request URL:https://www.googleapis.com/upload/youtube/v3/videos?part=snippet%2Cstatus&uploadType=resumable
Request Method:OPTIONS
Remote Address:172.217.9.42:443
This actually DOES return the "access-control-allow-origin" header that I expect. However, this is immediately followed by the "POST" request:
Request URL:https://www.googleapis.com/upload/youtube/v3/videos?part=snippet%2Cstatus&uploadType=resumable
Request Method:POST
Remote Address:172.217.9.42:443
And according to the dev console, it does NOT have the "access-control-allow-origin" header set. So, I understand why my browser is rejecting things. It just seems like Google broke something.
I did successfully deploy a NodeJS "CORS Anywhere" server on a server we manage. So, I can use that to work around the issue. But that really isn't the correct solution to the problem. Since uploading videos programmatically isn't something we do a ton, this work around will probably be sufficient for us. I would love to understand what went wrong or how to fix. So, if someone is successfully uploading videos with the YouTube API and JavaScript, I would love to hear about it. Hopefully this will help educate others if/when they run into the issue.
The same problem just started to happen with my services.
It seems to be a bug on googleapis or maybe youtube.v3.apis had a policy change and started to block some clients.
Google has fixed the problem and our long-standing code is back to working again without any changes.
Here is the tracked issue:
https://issuetracker.google.com/issues/158718687
Same here. I have no Solution, just a workaround and I hope they fix it soon. You can disable the CORS safetycheck in chrome. Run chrome.exe with --disable-web-security and set the user-data Directory temporary to another location, with the following parameter and the folder you want --user-data-dir=c:/anyfolderName/`
For security reasons you should disable-web-security only if you have to and switch back if youre done.
I dont know why this get a downvote, because it helped me to get around. Now google fixed the issue, no need to use this workaround longer. But maybe it helps when another CORS issue arise...

Consuming REST service with Ajax - Same origin policy

I'm writing my first Knockout Js application and I'm stuck trying to make an ajax request to my service (I'm new to web development in general).
I already found out that the problem is same-origin policy, and the reason I'm getting blocked by this I think has to do with my development setup: I'm using WebStorm to write my html/js and launching the page with its built-in webserver, which serves at port 63342; and my REST service is self-hosted, written in go, and running at 8080.
When the application is finished, I'd like to serve both the REST api and the Web app from my go server, but while developing the WebStrom server is really convenient.
Do any of you guys have similar problems? How do you work it out? Should I try to serve everything from my go server even during development? My server is not ready to serve any static content yet. Or should I try to use PJSON, even though I don't think I need it in my final app?
This is the error I get in my chrome develoment tools:
XMLHttpRequest cannot load http://localhost:8080/lines/03/pos. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:63342' is therefore not allowed access.
You could CORS-enable your REST service, and make sure that your web app is sending CORS request headers.
I'm not proficient in either Go or WebStorm, but I recommend investigating CORS.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
Turns out it took only a couple of lines of code to serve static content from my go server, so I just did that and now everything is working fine.
Thanks for your help though!
Best regards

How can I override the Origin header in Chrome when connecting to a WebSocket?

I am trying to connect to an external web socket server, which is not run by myself. I would like to connect to it from a localhost javascript file, therefore the origin header has null value.
I understand that this is a measure against cross-site forgery. However, since I am on localhost, I should be able to fake this, by getting Chrome to send a custom Origin header.
Is it possible? (if I need an extension, that is fine)
If not, what is my best option to achieve the above? Thank you.
Web pages cannot change the Origin header, but extensions can modify the request headers via the chrome.webRequest API. But ws:// and wss:// are not supported by this API, so this doesn't help unless the server also supports other means of communication via http(s) (e.g. long-polling).
There is still a solution though: Simply load a (known) web page at the desired origin in an iframe (e.g. https://example.com/favicon.ico or https://example.com/robots.txt) and use a content script to open the WebSocket from there.
The Origin header is one of the headers that are set automatically by the user agent (as part of the browser implementation), and cannot be altered programatically or through extensions. This makes sense because web service providers cannot allow random connections from localhosts.
You can connect to an external WebSocket only if you do it from a host explicitly accepted by the web service provider. Many headers cannot be trusted (because they can be overridden), but this is not the case with Origin as it offers security not only for users, but also for service providers against unwanted connections.
As far as I know this will not be possible, it would break the security guards against CSRF in Chrome.
If you were able to do that the whole concept of XHR would fall apart.
Here is an Extension you can use to manipulate header on the fly, but so far I have not been able to get it to manipulate socket headers.
Look here if you want to read more about this.
But this doesn't stop you from implementing your own client (in place of chrome) where you can literally send whatever headers you want, not sure if this helps you, sorry.
It depends how you want to use your chrome browser. Since you mention localhost I assume you develop and will use this for some kind of scraping. I suggest that you explore Chrome DevTools Protocol which will render (almost) any kind of protection useless because you use a real browser. CORS, Origin, Cookie or any arbitrary header value will be under your control, and you can send a custom header for xhr/websocket request(s). If you want to manipulate in a more advanced way you can use Network.continueInterceptedRequest. You might only want to start chrome using parameters like "--disable-web-security, --disable-xss-auditor, --disable-client-side-phishing-detection, --allow-insecure-localhost" more about such options at peter.sh. However, the last option require a plugin in order to spoof origin header so I recommend the first option.

Is there a cross site issue when accessing a webservice from a phonegap app?

I would like to access an external web service from a phonegap app using ajax without having to resort to CORS or JSONP to circumvent the cross origin issue. Looking at this question, it would appear that cross site http calls are not an issue with phonegap based apps. However, there are some conflicting comments in the answer. Is anyone able to provide a definitive answer to this question within the context of phonegap for ios/andriod platforms?
With phonegap, the only thing you have to do is allow access to the server by configuring access origin in config.xml.
No need of CORS or JSONP like you would have to do for example with rhodes.

Cross-domain XMLHttpRequest from an essentially domainless mobile app?

I'm making a mobile app using the PhoneGap framework, which is to say that the entire app is written in HTML, CSS, and JavaScript.
Part of the app requires me to fetch some information from a remote database.
I've spent the last hour reading up on how to make an XMLHttpRequest() to a remote domain, and I can't figure it out for the life of me.
As a bonus, since the goal of the request is to retrieve some database content, I need to send 3 parameters to the server for querying with.
I keep seeing things about the same-origin policy, but I can't find anything clearly saying whether it would apply to a phonegap app which has no actual host. I've also seen about 6 fairly overcomplicated workarounds. Before I go to the trouble of implementing one of those, I'd like to confirm that there isn't nowadays some simple way of doing this. Can anyone show an example, if so?
The same origin policy does not apply when you are running your XHR from the file:// protocol of the mobile device. Here is a small example I used to show how to make a XHR request to twitter.
http://simonmacdonald.blogspot.ca/2011/12/on-third-day-of-phonegapping-getting.html

Categories