Plugin Compatibility data now crowdsourced

compatibilityOne of the things Mark Jaquith and I talked about when Mark was on Press This a month back, in the episode aptly titled the Future of WordPress, was that Automattic would be adding a way for people to indicate whether a certain plugin was working for them or not, regardless of what the plugins “Compatible up to” version is.

This seems to have gone in beta now on, where you’ll see a block like you see on the right appear next to plugins. I think this is an awesome addition, as it makes the compatibility data a lot more reliable.

Tags: runs on the Genesis Framework

Genesis theme frameworkThe Genesis Framework empowers you to quickly and easily build incredible websites with WordPress. Whether you're a novice or advanced developer, Genesis provides you with the secure and search-engine-optimized foundation that takes WordPress to places you never thought it could go.

Read our Genesis review or get Genesis now!

11 Responses

  1. donnacha | WordSkillBy donnacha | WordSkill on 27 October, 2009

    Very smart addition.

    I wonder if they could also use this data to pinpoint incompatibilities between particular plugins i.e. send a list of installed plugins with each vote, process the amassed data to discover commonalities between those who report a plugin to be broken and, then, highlight possible conflicts under the Consensus section.

  2. Ray WaldoBy Ray Waldo on 27 October, 2009

    There is a slight problem with individual issues with a particular plugin. Added to what Donnacha said above about the incompatibilities between plugins, is the issue of other outside influences. At one time, I tried a half dozen new plugins and all caused an error. It took me a while to learn that there is a PHP memory limit and that I had reached it. Likewise, some newer users may have issues unrelated to the plugins but report them as incompatible anyway.

  3. Mark JaquithBy Mark Jaquith on 27 October, 2009

    Ray — We expect that there will be some “dirtying” of the data due to things like plugin intercompatibility. We’ve also braced ourselves for potential malicious incompatibility reporting, reporting plugins as incompatible for minor bugs, etc. We’ll see how it plays out. Once we have enough data, we may be able to find what ratio between “works” and “broken” gives us the most accurate measurement. We’ve also considered using data from to measure a person’s WP influence and weigh their vote more heavily. But right now we’re just gathering data.

  4. Ray WaldoBy Ray Waldo on 27 October, 2009

    Thanks Mark: I have a statistics background so I am always a bit fearful of uncontrolled studies . I appreciate that you guys will be taking these outside influences into consideration. It will certainly make the results more useful. Actually, I don’t put a lot of weight on the present voting results. AND, even if the notes say that a particular plugin has not been tested with my version of WP, if I want the functionality, I still TRY the plugin to see if it will work for me. Almost always it works. A more valuable “rating” for me is when was the last update. If the developer is not keeping the plugin updated, then I will look elsewhere.

  5. archshrkBy archshrk on 27 October, 2009

    I think Ray and donnacha have very valid concerns which should be considered. Maybe a disclaimer warning people that “other factors” may be responsible and how many people voted. If the result is based on 5 votes versus 500 votes, the data can have a different reliability.

  6. Drew StrojnyBy Drew Strojny on 28 October, 2009

    This looks like a promising feature. Once the quirks in the system are worked out I think it will serve as a valuable barometer for evaluating popular plugins. As archsrk points out, the results will be significantly less accurate for plugins with only a few votes.

    Are there any plans to use this in the theme directory?

  7. LuciBy Luci on 28 October, 2009

    This is a smart idea that’s useful, AND in practice! I agree with archshrk, I always find sort of ‘#votes against’ and ‘#votes for’ significant, sometimes more so that whether the overall result is good or bad (or in this case, working or broken)

  8. Matthew HaberBy Matthew Haber on 2 November, 2009

    “Crowdsourced”? Is that some Web 2.5 super pre-meeting term I’m not familar with? What does any of this mean?

  9. MileHighBy MileHigh on 14 November, 2009

    I’d like to see a way to indicate compatibility with WPMU versions as well.

    Another useful option would be to allow users to submit the name of known plugins that are incompatible with the plugin on this page. That way an incompatibility list could be accumulated and posted, possibly saving some folks from potentially breaking their site by installing this pugin.