yoast seo

Make the scrapers work for you!

Make the scrapers work for you!

January 10th, 2008 – 130 Comments

Some of you had already noticed that I’ve added a link back to my blog in the footer of articles in my RSS feed, like this:

RSS Footer in action

I didn’t do that because I thought you guys couldn’t find my blog. It was actually a plugin requested by Shoemoney, who wanted to make the scrapers work for him too, by getting some more backlinks from them. I thought it was a great idea, and this is actually quite a simple plugin: you upload it, enable it, maybe change the default text (which is “This is a post from <link><blog name></link>”), and you’re done.

It’s called RSS Footer, and a few people, like my friend Marcus, picked it up and ran with it already, I hope it works for you too!

Update: based on a quote from Matt Cutts in a post by Patrick Altoft, “How to foil scrapers on your blog”, I’ve added the option to add a link back to the post itself, with the title of the post as anchor text. This looks like this:

RSS Footer with link back to post

130 Responses to Make the scrapers work for you!

  1. slobjones
    By slobjones on 27 June, 2009

    Joost, I’d like more text options (and not just a text box) for link back to the post.

    As it is now, link back to the post reproduces the post title in the link. This can be confusing for the reader. I’d like to be able to link back to the post, but change the link text.

    Can you add that capability to RSS Footer?

  2. Dave Dugdale
    By Dave Dugdale on 20 May, 2009

    I really liked your plugin so I talked about it in my latest video:

    Dave Dugdale

  3. Ian
    By Ian on 27 October, 2008

    I just installed this but it doesnt appear to change my rss output

  4. Charles
    By Charles on 2 July, 2008

    Thank you for this great plugin. This is really helpful.

  5. diani
    By diani on 1 July, 2008

    That’s great, thank you very much.

  6. Netty Gritty
    By Netty Gritty on 13 May, 2008

    As a wordpress.com user, I don’t have access to plugins. But I am gonna use a Feedflare that will link to my blog url under every post.

    It’s a great idea. Thanks for the post! :D

  7. johncarter
    By johncarter on 13 May, 2008

    Hey Friends!
    I have developed a new web directory and need your valuable comments to know if something needs to be inculcated.
    It will be of great pleasure to have your valuable comments.

  8. freelance webdesign
    By freelance webdesign on 4 May, 2008

    Great plugin. Thank you very much.

    Best Regards from Denmark :)

  9. Justin
    By Justin on 4 May, 2008

    Awesome idea Joost.

    I have installed your plugin, but my results is showing only plain text, no live links.

    This is the address of my feed.

    Any advice appreciated.

    Thank you.

  10. Michael
    By Michael on 17 April, 2008

    Great plugin, that I have just installed on my website. Superb work.

    However, there is a small glitch: When you enter text in German – as I did – although it works fine in the newsfeed, in the options-panel of the plugin itself, it’s displayed not correctly. When you save again without cleaning this mess up, of course the humbug of characters goes into your newsfeed…

    And as many others said before: A great plugin (actually, also the second from this website that is in use on 3th.be)! Thanks.

  11. zylla3
    By zylla3 on 13 April, 2008

    Thank G*d I found this post and comments. Every time I read about plugins, I do a 90d turn when I get to the words: code, php, html, etc. This business of scraping is giving me headaches, but the coding stuff doesn’t make it any better for me.

    At least it’s good to know I’m not alone. It was by accident that I discovered 90% of my posts at this particular site [luckily at an early stage]. I thought I was crazy losing sleep googling this scraper and blogging about it. I have learned to reshape my blogging at this point to accommodate the experience…until I could learn any of the good stuff you’re all talking about. Thanks!

  12. pozmu
    By pozmu on 10 April, 2008

    In previous post, I meant that output in administration panel is garbage after saving RSS footer text.
    Another thing is that it doesn’t work with RSS excerpts – in “after” mode nothing is displayed, and in “before” mode footer and link to article (if enabled) is displayed, but it’s plain text, not link.

  13. pozmu
    By pozmu on 10 April, 2008

    Nice plugin, but it doesn’t work well with internationl characters (e.g. Polish) :(

  14. Joost de Valk
    By Joost de Valk on 4 April, 2008

    You’re very right John, fixed that in 0.7.5!

  15. John Honeck
    By John Honeck on 2 April, 2008

    Since the 2.5 update I’ve noticed that in the feed the post to the article itself isn’t using permalinks but rather links to ?p=xxxx instead.

  16. Glenn Dixon
    By Glenn Dixon on 30 March, 2008

    An awesome idea and approach – if you can’t beat ’em, fuck ’em! I like it.

  17. Gunnar andreassen
    By Gunnar andreassen on 23 March, 2008

    Scraping has been a big problem, hope this thing fixes it. Thanks!

  18. Jack
    By Jack on 16 March, 2008

    Doesnt work for me me either. Using wp 2.33 with feedburner for feeds.

  19. Ephena
    By Ephena on 14 March, 2008

    I do something similiar on all webpages because of scrapers. even though my sites links are relative links. I hardcode some on all pages with the full http :// structure so that as my sites are scraped or more commonly 302 redirected to as belonging on their sites, they unwittingly publish at least some hard coded urls to the original content. I do think it helps safeguard against scrapers.

  20. Beth
    By Beth on 3 February, 2008

    Great idea for a plugin! I’ve had issues with scrapers for a few months and have tried everything including a warning type plugin that provides readers the information about the post being scraped and to contact me. But I see this as something I can capitalize on- with as many scrapers using my content I should be number 1 on Google and Technorati!

  21. colbert low
    By colbert low on 2 February, 2008

    Looks like a great plugin. I will try it

  22. BillinDetroit
    By BillinDetroit on 1 February, 2008

    I doubt if scrapers are a large problem for me, but what the heck … every one-way inbound link is a good one, no? While my content is ~100% original, it is hardly authoritative.

    Just ask my wife. ;-)

  23. Darren
    By Darren on 26 January, 2008

    You got to love this. These scrapers are killing my sites. I can’t wait to see my traffic fly back from this.

    Does this work for those who already scrapped my blogs? or only new scrappers?

  24. Bianca
    By Bianca on 21 January, 2008

    Thanks! This is a great plugin! Now to sit back and hope it works. :)

  25. nottingham honda
    By nottingham honda on 20 January, 2008

    i hate scrapers sites with a vengance!

    mainly because i think their adsense is making more money than mine.

  26. SusanneUK
    By SusanneUK on 20 January, 2008

    Just adding my eternal thanks to the bottom of the pile, I will send others your way as well..

    Thanks again

  27. Joost de Valk
    By Joost de Valk on 20 January, 2008

    @Brian: which version of WP are you using? Could you email me that?

  28. Brian
    By Brian on 17 January, 2008

    This is a fantastic idea but unfortunately the plugin doesn’t seem to work on my blog. I enable the plugin and edit the footer message but it doesn’t appear at the end of each post. Could it be that my theme is incompatible?

    Big respect for this excellent concept just the same. You’re one of my favourite plugin authors!

  29. Joost de Valk
    By Joost de Valk on 17 January, 2008

    @JLH: yeah, thx… Version 0.7 fixes that!

  30. JLH
    By JLH on 17 January, 2008

    I made a minor tweak, I changed line 18 from …’RSS Footer’, 1, basename… to …’RSS Footer’, 10, basename… just so that subscribers couldn’t see and change the options.

  31. Shycon Web Design
    By Shycon Web Design on 16 January, 2008

    Ah, great plug-in. There isn’t much you can do to keep people from scraping your stuff, but why not get some links out of it! :D Thanks

  32. Joost de Valk
    By Joost de Valk on 16 January, 2008

    @funky: glad I inspired you, have fun, and share the cool stuff you create :)

  33. funky
    By funky on 16 January, 2008

    I am considering to start with plugins develop for a long time.

    Thank you for inspiration!

  34. Joost de Valk
    By Joost de Valk on 16 January, 2008

    Glad to hear that Marko!

  35. Marko Pyhajarvi
    By Marko Pyhajarvi on 16 January, 2008

    This is great, I’ve been looking for something like this one. Thanks! Gotta install it tonight..

  36. Joost de Valk
    By Joost de Valk on 16 January, 2008

    @Simon & Alfred: glad you like it!

  37. Alfred
    By Alfred on 16 January, 2008

    got here from http://www.tenaciouscreations.com/

    Man this is one superb plugin, was just thinking of something like this when I found it.
    Will upload it later today, thanks for the plugin

  38. simon
    By simon on 15 January, 2008

    Great plugin!! seen a good response to my blog reactions since adding this RSS footer…

  39. Joost de Valk
    By Joost de Valk on 15 January, 2008

    @Erik: good idea… That would mean doing some different stuff though, I’ll have a think about it.

  40. Erik Heidt
    By Erik Heidt on 15 January, 2008

    Joost –

    A useful feature would be the ability to change the message without effecting older posts. This would enable changing the settings without causing the RSS feed to appear as all new posts.


    Art of Information Security

  41. Joost de Valk
    By Joost de Valk on 14 January, 2008

    @Marc: well, in this plugin you can alter the text every once in a while. Should make it harder to remove it.

  42. Marc
    By Marc on 14 January, 2008

    I use a similar plugin and have noticed that scrapers have started removing my custom text and permalink link text.

  43. CBR
    By CBR on 14 January, 2008

    Yep this is a good idea but I wonder how long it will be until the scraper sites catch on and block the url’s. Until then I will give this a try, nice work.

  44. Ken Teegardin
    By Ken Teegardin on 14 January, 2008

    Excellent idea. I will start using that on all my blog entries..

  45. Ryan
    By Ryan on 13 January, 2008

    Great plugin Joost. Its been working wonders on all of my sites.

  46. kevin
    By kevin on 13 January, 2008

    Nice plugin. I have downloaded it.
    Your google analytics also works great.

    Plenty of Software: Free downloads center

  47. Halifax SEO
    By Halifax SEO on 13 January, 2008

    Good work Joost! Thanks for sharing it.

    Just subscribed to your feed also.

    Sam Dodd, using the technorati tags for some important keywords, will help.

  48. Sam Dodd
    By Sam Dodd on 12 January, 2008

    Makes me think I need to figure out a way to have more scrappers attempting to steal my content. Does anyone have any suggestions for attracting scrappers?

  49. Jonathan
    By Jonathan on 12 January, 2008

    Great suggestion – I’m using a Feedburner Feedflare called “attribution” now, but when I move over to a WordPress installation (currently on MT), this will be one of the first plugins I install.

  50. skuub
    By skuub on 12 January, 2008

    Nice Plugin Joost – now I won’t mind if some of the scrapers find my new published blog posts ;)

  51. Rasmus
    By Rasmus on 12 January, 2008

    That’s clever Joost, very clever! Cool plugin, thanks for sharing it :)

  52. Scarf*oo
    By Scarf*oo on 12 January, 2008

    This solution will help, but I find it somehow obtrusive. Still a great tip, thanks!

  53. Rekzai
    By Rekzai on 12 January, 2008

    Will this work if I’m using feedburner for all my feeds?

  54. Andrew
    By Andrew on 12 January, 2008

    Does this work with the spammers who advertise medication and other things on older blogs as well Joost?

    I’m getting so absolutely sick of getting those and having to log in and spam filter them.

    I’d love to know.


  55. get paid to Blog
    By get paid to Blog on 11 January, 2008

    Thats a cool idea…I’ll check it out and see if i can get it on my site fr the authors..they are always asking me about concerns like this.

  56. Joost de Valk
    By Joost de Valk on 11 January, 2008

    Great to hear Allan! Welcome as a new reader, I hope you’ll enjoy your stay!

  57. FireflySEO
    By FireflySEO on 11 January, 2008

    Hi Joost,

    Good work! Looks like a very useful plugin, I have just installed it and lookforward to seeing if it helps with the SERPS.

    Just subscribed to your feed also.


    Allan Stewart

  58. Joost de Valk
    By Joost de Valk on 11 January, 2008

    @All: I could use your help on the Digg above!

  59. Joost de Valk
    By Joost de Valk on 11 January, 2008

    Well you can ask Andrew, who commented above. I got his SEOBook affiliate account shutdown because he was scraping me :P

  60. David Bradley
    By David Bradley on 11 January, 2008

    I have to admit, I do too. I got a whole scraper network shutdown just before Xmas. I’d sent the old C&D to the whois registered owner of the domain, and when that didn’t work, I emailed the registration/hosting company. They said it wasn’t their problem, but then got back to me a day or two later to say they’d shut the domain down. Obviously, something seriously dodgy going on above and beyond scraping Sciencebase!


  61. Joost de Valk
    By Joost de Valk on 11 January, 2008

    @Sucker: nice.
    @David: so true, I do occasionally send out cease and desists though.
    @Rob: glad you like it!

  62. Rob
    By Rob on 11 January, 2008

    Thanks Joost, this is a great plugin to fight against scraping.

  63. David Bradley
    By David Bradley on 11 January, 2008

    It’s irrelevant whether one is happy or not about a site being scraped. There is very little you can do to prevent it unless you want to spend valuable blogging time serving cease and desist letters. Much better to make the scrapers work to your advantage than trying to get rid of them one by one (it’s like playing wack-a-mole anyway, get rid of one, another pops up). Sciencebase seems to be scraped all over the place and I have tried to get rid of those sites that try to pass themselves off as “the” original site, but for the general news scrapers, you just have to put up with it and count it as more eyeballs seeing your words!


  64. Sucker
    By Sucker on 11 January, 2008

    Yes! Thanks for the update, I’ll definitely use this now!

  65. Joost de Valk
    By Joost de Valk on 11 January, 2008

    @Andrew: I think that’s a bit too much…

    @Dave: thx :)

  66. Dave
    By Dave on 11 January, 2008

    Joost. You the Man!!!

    Love the plugin.

  67. Andrew
    By Andrew on 11 January, 2008

    I think it’s a smart solution. Maybe you could include some javascript to disable mouse clicks if the post does not appear on a certain domain?

    Or is that overkill?

  68. Joost de Valk
    By Joost de Valk on 11 January, 2008

    @Andrew: no, but if there’s assholes who do it, the result is a bit less of a catastrophy :)

  69. Andrew
    By Andrew on 11 January, 2008

    Does this mean you are now happy for your blog to be scraped?

  70. Joost de Valk
    By Joost de Valk on 11 January, 2008

    @B10G: good! If you have any ideas for plugins feel free to let me know ;)

  71. B10G
    By B10G on 11 January, 2008

    Thanks Joost, this is just what I have been looking for, up until now I have just been manually updating blog posts which I know have been scraped with keyword links.

  72. Joost de Valk
    By Joost de Valk on 11 January, 2008

    @David: good to hear that!

  73. David Bradley
    By David Bradley on 11 January, 2008

    Nice one, I’d previously been using a hack suggested by Patrick Altoft at http://www.blogstorm.co.uk, this is obviously much slicker, and Patrick led me to your page and thence the download!


  74. Joost de Valk
    By Joost de Valk on 11 January, 2008

    @Mendax: then ALL outbound links in articles won’t work, unfortunately, there’s not much you can do about that. What you CAN do though is add a specific string to your footer with this plugin, and run Google Alerts, Blog Search queries etc. specifically for that string.

  75. Mendax
    By Mendax on 11 January, 2008

    Ok, the idea is nice, but what if a scraper uses strip_tags function to remove all the html stuff?

  76. Joost de Valk
    By Joost de Valk on 11 January, 2008

    Thx for all the great feedback guys and girls!

    @Martijn: you can add javascript, but most of the time it will be removed, and it won’t work in feedreaders… You’ll never know for scrapers though!

    @Wiep: I already thought of that too, I’ll think of a way to include that in the next version.

  77. Wiep
    By Wiep on 11 January, 2008

    Great plugin Joost! Now if only it were possible to vary the anchor text… :P

  78. Nicholas James
    By Nicholas James on 11 January, 2008

    Great plugin! Thanks for this :)

  79. Clint Lenard
    By Clint Lenard on 11 January, 2008

    This is an awesome plugin! Thank you VERY much!

  80. youfoundjake
    By youfoundjake on 11 January, 2008

    This is the second plugin that I’ve gotten from you Joost. Please keep coding and coming out with stuff like this.

  81. AzAkers
    By AzAkers on 11 January, 2008

    Sweet – one of my favorite plug-in authors & one of my favorite bloggers working togather to take the sting out of geating scraped – nice!!

    Thanks for the work that went into this!

  82. Pablo
    By Pablo on 11 January, 2008

    Good plugin, in my case related posts just after the feed do the work ;)

  83. martijn
    By martijn on 11 January, 2008

    is javascript possible in the rss-footer?

  84. Chris Jacobson
    By Chris Jacobson on 11 January, 2008

    Nice plugin… I’ve been looking for something like this.

  85. martijn
    By martijn on 10 January, 2008

    nice plugin! using it. thx

  86. Eduard Blacquière
    By Eduard Blacquière on 10 January, 2008

    Brilliant plugin, Joost!

  87. Feydakin
    By Feydakin on 10 January, 2008

    Dude, awesome, thanks..

  88. Frank N. Stein
    By Frank N. Stein on 10 January, 2008

    Nice plugin, thanks for sharing.

  89. Fatgadget
    By Fatgadget on 10 January, 2008

    Hi Joost

    This is a great plugin, thanks :)

  90. Rudy
    By Rudy on 10 January, 2008

    Thank you for the plugin. I’m using it right now.

  91. Sam I Am
    By Sam I Am on 10 January, 2008

    Did something like this on our own blogging platform not too long ago as well. Another benefit, outside the scrapers, is for those importing blogs as notes into their Facebook profiles. Not as great for back links (although the benefit from scrapers is also limited I’d say) that are crawled, but very good for referrals – at least in our case where it links to how to set up a similar blog :)

    Probably just a matter of time before all the scraper sites start parsing out the urls; luizen!

Check out our must read articles about Analytics