Sunday, October 30, 2011

IMDb Search Suggestions with jQuery Autocomplete

In conjunction of my previous posts on IMDb Scraper API and jQuery Autocomplete with multiple search suggestions, on popular demand, I’ve created yet another search suggestion tool to fetch suggestions from IMDb.com. I’ve implemented a simple PHP Proxy script to get search suggestions from IMDb and format it according to jQuery Autocomplete Widget’s JSON data format.

For downloading and a guide to setup the suggestions tool on your website please visit the demo page on my labs section. You will need the suggest.php and imdbImage.php Proxy files for pulling content from IMDb.

Demo: http://lab.abhinayrathore.com/imdb_suggest/

IMDb’s suggestion API provides Title Name, Title Id, Cast, Type and Poster URL to display in the dropdown. You can customize the JavaScript code for your needs. IMDb has a leechers policy in place for media images so I’ve also included a PHP Proxy script to show IMDb Posters on search dropdown.

30 comments:

  1. Hi,

    thanks for the code.
    But what would really interest me is how to set up a similar auto suggest system. I don't want to make a query to the database each time the user types in the search field. That's slow and puts pressure on the database.
    It seems that imdb caches the results in JSON format as a file which then is returned as you type. I would guess they have thousands of these cached files each one depending of the first letters you type... Just a guess though, i'm not sure exactly how they impelemented it. All i know is that it's lightning fast and it searches within thousands and thousands of results. I doubt it queries their database...

    Any idea how their autosuggest works and how i could build a similar system?

    ReplyDelete
  2. Abhinay, One problem I see with your demo is that if there is no json file on the imdb suggestions api, your search suggestions disappear. So, for example, if you type "Empir", you're good up until you hit that final letter "r".

    ReplyDelete
  3. To build a similar system:

    It appears that IMDb has created a structure of 36 folders (one for each letter of the alphabet and number), and inside each folder holds the results for UP TO 4 characters of searching (notice that on IMDB the 5th character does NOT change your results, and on Abhinay's demo, the suggestions disappear).

    Now, they do not have a file for EVERY possible 4 characters, only the ones that would have any suggestions. So, the file for "HOLL" would contain anything with the word "hollow", "holly", "holler", etc. But the file "HOLW" would not even exist. Now, on IMDd, as you type "HOLW" you would see the results of file "HOL", and those results WOULD STAY as you hit the "W", since "HOLW" contains nothing. Abhinay, you may want to update your demo to do the same.

    It also appears that each can contain a "hit" if ANY word in the title matches based upon the first x chars.

    So, the only question that remains is how to get the files out there to begin with? Either periodic routines that loop through the database, or in the record add/update script that maintains the records. Notice that IMDb ONLY displays up to 6 matches (at least in my quick tests). How does it decide WHICH matches to show for the letter "H"? There must be some sort of weighting system, and as the system tries to update the "H" file, with, for instance "Sherlock Holmes", it compares the weight of the new entry with the weight of existing entries.

    Obviously this is only my intuition based upon what I've seen, and I'd appreciate any feedback on my suggested methodology :)

    ReplyDelete
    Replies
    1. Thanks for the input. It seems doable.
      The only challenge is the weighing algorithm they use.
      It shows the top 6 results but mostly matches the most popular matches.

      Delete
  4. As mentioned above, the search suggestions disappear if there is no json file on imdb. I therefore added some code to trim the search term until a valid result is obtained. I replaced lines 14 and 15:

    $url = "http://sg.media-imdb.com/suggests/${firstchar}/${search}.json"; //format IMDb suggest URL
    $jsonp = file_get_contents($url); //get JSONP data

    With the following code:

    do {
    $jsonp = @file_get_contents("http://sg.media-imdb.com/suggests/${firstchar}/${search}.json"); //get JSONP data
    $search = substr($search,0,-1);
    } while (!$jsonp);

    ReplyDelete
    Replies
    1. Thanks a lot for sharing your info, it was really useful :)

      Delete
  5. Is there a way to limit the results that IMDB returns to only the titles of TV shows and Movies? If not, is there a way to determine if the result selected is an actor/tv show/movie/etc?

    ReplyDelete
    Replies
    1. Nevermind... q gives all the info I need...

      Delete
    2. @Rob: Could you fix it? (I am trying to get only movie titles and Not actor names, or tv-series.. but I don't know how :|)

      Delete
  6. Is there a way so when I click on a movie on the input search to be displayed the movie name instead of something like "tt0367677"?

    Thank you.

    ReplyDelete
  7. hi, I just check your nice work, but it seems, in the time something is broken, now I got just names without images, so it seems it is just normal autocomplete, if possible, please look at it, and fix. Also when I load the page, I get error in firebug:

    TypeError: $(...).autocomplete(...).data(...) is undefined
    [Break On This Error]

    }).data("autocomplete")._renderItem = function( ul, item ) {

    I am new to jQuery, so I am unable to fix your work for now.

    ReplyDelete
  8. spent 1 hour or so on this. I really "like" this...anyway, read here more: http://bugs.jqueryui.com/ticket/8996 the fix is just replace here " }).data("uiAutocomplete")._renderItem = function( ul, item ) {" then it works like it should!

    ReplyDelete
  9. Thanks a lot for the research Brano. I've updated the code to work with jQuery UI v1.10 and later. For users below v1.10, you'll have to use ".data("autocomplete")._renderItem" instead of ".data("uiAutocomplete")._renderItem".

    ReplyDelete
  10. Hi!

    I really like the script. Unfortunately, there is one thing that I can not solve. So far, the search worked properly but now unfortunately completely slips and does not work properly. I saw the demo site that was the problem, but it solved the problem. May I ask what was the problem?

    Menta

    ReplyDelete
  11. Hi,

    I am new to javascript. Is there any way to NOT display the tt0000000 on hover and select? Just to click on the suggestion, hit enter or click and it submits the search?

    I can find myself if I knew what I was trying to google for ;)

    ReplyDelete
  12. To show the movie name in search box and get clickable links to the movie change the following lines;


    focus: function( event, ui ) {
    $(this).val( ui.item.label );
    return false;
    },
    select: function( event, ui ) {
    window.location = "http://imdb.com/title/" + ui.item.value;
    return false;
    }

    ReplyDelete
  13. It has some problems with some movie posters.For example for Don Jon the poster is 1271X1991.

    ReplyDelete
    Replies
    1. I also faced this issue, Could you find a solution for this? :|

      Delete
  14. Thx for great work, but I have problem with drop down menu. When I test I can see images and text but I can see drop down menu. Strange ....pls help...

    ReplyDelete
  15. This comment has been removed by the author.

    ReplyDelete
  16. I noticed that when we type something in the search box, for example "Gone", (in case that I am looking for "Gone Girl") when we are going up or down in dropdown menu what we wrote in search box is changed .. for example if I write Gone .. and my mouse goes to "Zoe Gone", the Gone that I wrote in textbox will be replaced with "Zoe Gone".. While in IMDb, whatever you type in the textbox will be kept..

    ReplyDelete
  17. How to get this to work with jQuery v2.1.1 ?

    ReplyDelete
  18. im pretty new at this but i s there anyway to keep the autofill? but somehow if i were to type in Brad Pitt id nevigate to my own custome html file?

    ReplyDelete
  19. Sir I am new to php and jquery I will be very very thankful to you if you can upload it on GitHub and can give link of GitHub.

    ReplyDelete
  20. take a look at this autocomplete, it's awesome, http://justwebcode.blogspot.com/2017/07/autocomplete-textbox-with-jquery.html

    ReplyDelete
  21. Hi, I was looking for some specific information and I fortunately came across your blog. It absolutely superb. Thanks for sharing so much information. What more one can expect when someone finds the appropriate answer to his/her query. Know more about premium proxy service @ https://www.proxyrotator.com/

    ReplyDelete

Thanks a lot for your valuable comments :)