I've read a lot of topics about redirecting Tumblr to WordPress, but I still can’t find a suitable solution.
Here is the problem: I want to redirect jeby.tumblr.com, a Tumblr blog, to the new jeby.it, a WordPress (WIP) blog with a custom domain and web space etc. I’ve already imported all contents, now all I want is to “automagically” redirect every single post from
jeby.tumblr.com/post/[POST ID]/some-slug
to
jeby.it/2012/05/some-slug
I know that the post year and month are available in the Tumblr HTML code, as they are used to compose the permalink. I can’t use .htaccess redirects because the Tumblr blog is hosted by Tumblr.
I’ve done the same thing with Blogspot, where I found a plugin that created the right JavaScript code to paste into the Blogspot model and get automatic redirection.
As you have realized already, you will need to do the redirection client side. How to achieve that is a matter of Tumblr’s theme templating system’s possibilities and limitations.
The year and month of a post are available as {Year} and {MonthNumberWithZero} tokens respectively; that gets you 2012 and 05.
The post slug, however, is not available as a token (not sure if this is an intentional omission – post slugs are optional on Tumblr; you can manually set one, but if you don’t your post just goes by its numeric ID) – so you will have to parse it out of the link inserted by the {Permalink} token.
Redirection can only take place on single-post pages. Unluckily, there is no template based way to make sure you target only these, as the beauty and limitation of Tumblr’s theme DSL is that it is defined as one page: you get a {block:Posts} block and Tumblr does the figuring out how many posts to display inside that. That means that meta http-equiv="Refresh" tags are out of the question, as they would also be included in all non-single-posts pages.
Luckily, Tumblr does let you include arbitrary JavaScript in your template, and that is the way to go:
add the following line inside the {block:Posts} block:
<span class="redirdata">{Year},{MonthNumberWithZero},{Permalink}</span>
Style your redirdata spans with display:none in your CSS to hide them (or style them inline – considering their intended use, there is little point in being dogmatic).
now add a script to your <head> to parse these and redirect to the WordPress URL. As you want to do this for every possible visitor, i.e. as cross-browser compatibly as possible, I recommend using jQuery. Include it with:
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js" type="text/javascript"></script>
or any other hosted source (see the jQuery docs for CDN URLs). Then add the following script:
<script type="text/javascript">
$(function(){
if ( $('.redirdata').length === 1) {
var redirdataTokens = $('.redirdata').text().split(',');
if (redirdataTokens.length === 3) {
var redirectTo = 'http://jeby.it/'+redirdataTokens[0]+'/'+redirdataTokens[1]+'/'+redirdataTokens[2].replace(/.*\//, '');
window.location = redirectTo;
}
}
});
</script>
and your single-post page should automatically redirect all visitors to the WordPress post page (if they have JavaScript enabled, of course). I am sure you will figure out to adapt this to tag archives, searches and such, if need be.
Caveat Empteor: this will only work if the slugs of your WP posts match the slugs of your Tumblr posts – crucially also in the case of Tumblr posts with no textual slug (meaning you WP slug needs to match the numeric Tumblr post ID in that case).
Related
I have an AngularJS application that is injected into 3rd party sites. It injects dynamic content into a div on the 3rd party page. Google is successfully indexing this dynamic content but does not appear to be crawling links within the dynamic content. The links would look something like this in the dynamic content:
Link Here
I'm using query parameters for the links rather than an actual url structure like:
http://www.example.com/support/title/Example Title/titleId/12345
I have to use the query parameters as I don't want the 3rd party site to have to change their web server configuration to redirect unfound URLs.
When the link is clicked I use the $locationService to update the url in the browser and then my angular application responds accordingly. Mainly it shows just the relevant content based on the query params, sets the page title and meta description.
Many of the articles I have read use the route provider in angularJS and templates but I'm not sure why this would make a difference to the crawler?
I have read that google should view urls with query parameters as separate pages so I don't believe that should be the issue:
https://webmasters.googleblog.com/2008/09/dynamic-urls-vs-static-urls.html
The only things I have not tried are 1. providing a sitemap with the urls that have the query parameters and 2. adding static links from other pages to the dynamic links to help google discover those pages.
Any help, ideas or insights would be greatly appreciated.
This happens because google crawlers are not able to get the static html from your url since your pages are dynamically rendered with Javascript, you can achieve what you want using the following :
Since #! is deprecated, You can tell google that your pages are rendered with javascript by using the following tag in your header
<meta name="fragment" content="!">
On finding the above tag google bots will request your urls with the _escaped_fragment_ query parameter from your server like
http://www.example.com/?_escaped_fragment_=/support?title=Example Title&titleId=12345
Then you need to rebuild your original url from the _escaped_fragment_ on your server and it will look like this again
http://www.example.com/support?title=Example Title&titleId=12345
Then you will need to serve the static HTML to the crawler for that url.
You can do that using a headless browser to access the url. Phantom.js is a good option to render your page using the javascript and then give the contents into a file to create a HTML snapshot of your page. You can save the snapshot as well on your server for further crawling, so when google bots visit can you can directly serve the snapshot instead of re-rendering the page again.
The web crawler might be running at a higher priority than the AngularJS interpretation of your dynamic links as the web crawler loads the page. Using ng-href makes the dynamic link interpretation happen at a higher priority. Hope it works!
If you use urls with #
Nothing after the hash in the url gets sent to your server. Since Javascript frameworks originally used the hash as a routing mechanism, that's a main reason why Google created this protocol.
Change your urls to #! instead of just using #.
angular.module('myApp').config([
'$locationProvider',
function($locationProvider) {
$locationProvider.hashPrefix('!');
}
]);
This is how Google and bing handle the ajax calls.
The documentation is mentioned here.
The overview as mentioned in the docs is as follows
The crawler finds a pretty AJAX URL (that is, a URL containing a #! hash fragment). It then requests the content for this URL from your server in a slightly modified form. Your web server returns the content in the form of an HTML snapshot, which is then processed by the crawler. The search results will show the original URL.
Step by Step guide is shown in the docs.
Since the Angular JS is designed for the Client Side so you will need to configure your Web server to summon a headless html browser to access your web page and deliver a hashbang url which will be given to the special google URL.
If you use hashbang URL then you would need to instruct the angular application to use them instead of regular hash values
App.config(['$routeProvider', '$locationProvider', function($routes, $location) {
$location.hashPrefix('!');
$routes.when('/home',{
controller : 'IndexCtrl',
templateUrl : './pages/index.html'
});
as mentioned in the code example here
However if you do not wish to use hashtag url but still inform the google of the html content but still want to inform the google then you can use this meta tag as this
<meta name="fragment" content="!" />
and then configure the angular to use the htmlURL's
angular.module('HTML5ModeURLs', []).config(['$routeProvider', function($route) {
$route.html5Mode(true);
}]);
and then whichever method to be installed via module
var App = angular.module('App', ['HashBangURLs']);
//or
var App = angular.module('App', ['HTML5ModeURLs']);
Now you will need a headless browser to access the url
You can use phantom.js to download the contents of the page ,run the javascript and then give the contents into a temporary file.
Phantomrunner.js which takes any url as input,downloads and parses the html into DOM and then checks the data status.
Test each page by using the function defined here
SiteMap can also be made as well as shown in this example
The best feature is you can use search console of verify your site url using
Google search console
Full attribution goes to the website and the author mentioned in this site
.
UPDATE 1
Your crawler needs the pages as -
- com/
- com/category/
- com/category/page/
By default, however, Angular sets your pages up as such:
- com
- com/#/category
- com/#/page
Approach 1
Hash bang allows Angular to know which HTML elements to inject with JS which can be done as mentioned before but since it has been depericated hence the another solution would be the following
Configure the $locationProvider and set up the base for relative links
You can use the $locationProvider as mentioned in these docs and set the html5mode to true
$locationProvider.html5Mode(true);
This lets Angular change the routing and URLs of our pages without refreshing the page
Set the base and head of your document as <base href="/">
The $location service will automatically fallback to the hashbang method for browsers that do not support the HTML5 History API.
Full attribution goes to the page and the author
Also to mention there are also some other measures and tests that you can take care of as mentioned in this document
I'm planning to build a site where I can share my handpicked curated contents and I couldn't wrap my head around the basic idea of getting those data fed into my site without going through API.
I first thought maybe I should inspect the source HTML of the page I want to embed on my site and access it with something like $('div.post').find('img').attr('src').
But I can't imagine myself doing that every time so I guess there must be a better way.
It's what Google+ does with their post. Once you add a url link, after a second it pulls featured image and some text snippet from the linked page.
Many sites use the Open graph protocol to get the meta-title, meta-description, image etc. for any url.
For example open: view-source:https://blog.kissmetrics.com/open-graph-meta-tags/ and search for "Open Graph Protocol Meta".
They are contained in the page source. You will have to send a request to the URL you want to crawl from, and read the appropriate meta tags through Regular Expr / HTML Parsers.
You can't make this with javascript. You need a server-side script that downloads the page you need and then parse it with a DOM parser.
With PHP you can get the content of one URL with cURL.
See more:
http://php.net/manual/es/book.curl.php
I want to add facebook like buttons to different pages and use different titles, descriptions and images.
The problem now is: Facebook uses meta tags from the header to determine this values, e.g.: . I use GWT and therefore I only have one host page (e.g. index.html) and different content is rendered in this page:
"www.myurl.com#blogpost:1" would load the blogpost with id "1". Therefore every blogpost would have the same title, description, image. I could change the metatags with javascript according to which blogpost is requested. But I guess javascript is not executed by the facebook parser. Is there a way to realize different like buttons with only one host page?
I now generate a special link for facebook. So if my GWT URL looks like "www.myurl.com#blogpost:1", I will generate the URL "www.myurl.com/fb/blogpost/1". Now I check in a Servlet Filter for URLs starting with "fb". If I find a Request with a URL like that, I just write out the Meta tags and a java script forward to my actual page: "www.myurl.com#blogpost:1". The facebook crawler just sees the meta tags and doesn't use the javascript forward.
Normal Users on the other hand are forwarded to the regular page. This works pretty good for me. Thanks CBroe for the hint.
I have dynamic forms in my urls .i want to write a javascript such that when the users paste that javascript in their web pages the form should dispay.
<script type="text/javascript">
window.location.href = "http://myserver/2a249ccf59e272abb8e7b8582560a45c"
</script>
The problem is using the above code , The url redirects.I want to avoid redirection
I'v tried iframes , but i dont want to use them as per requirements .
You need to use AJAX to fetch a remote page. You must get the page from the same server that your webpage was delivered from to avoid same origin policy issues.
I would recommend using a library to handle AJAX requests. jQuery has AJAX functions built in, but you could use others also. With jQuery...
// be sure to include jQuery library in the head of your document...
// general wrapper to execute code only after all page elements have loaded
$(function(){
$.get("http://myserver/2a249ccf59e272abb8e7b8582560a45c",function(data,status){
// put the fetched data (your html form) at the end of the existing body content
// you could put it somewhere else by changing the `body` selector
$("body").append(data)
})
})
Here's the problem: On an .ascx control, there is a short Javascript block. The script, among other things, refers to a method in the control and generates DoP meta tags for different pages on a website.
Problem is, Google crawler reads the Javascript in the pages and thinks every DoP meta tag is an url of some kind. This is probably because there is a forward slash in the tag(?). In the end, Google has indexed massive amounts of non-existent urls on the website and given them 404 status. Is there a way around this? For example, is it possible to put nofollow tags directly to the Javascript?
<script language="javascript" type="text/javascript">
<!-- DoP tag -->
var v = {};
v["sample"] = '/Product_A_';
</script>
Google crawler seems to index an url "http://www.my web site.com/Product_A_" from above script.
How about setting up a site map and a robots.txt?
http://en.wikipedia.org/wiki/Sitemaps
That way, the search engines will only crawl and index the content specified in the sitemap and you can use the robots.txt to exclude the extranious content, especially if it all follows a specific pattern.
Or use th meta tag "nofollow" on these pages, meaning the search engines will not follow any additional links:
http://www.robotstxt.org/meta.html