Latest Tweets

Follow Us

Magento - Apache Solr Integration - Part IV (Ajax Search Form)

Following with the Magento - Apache Solr Integration series (this was the previous post), we'll work now in the UI side. To make this simple, we'll implement the following screen outside Magento, just to play around with Solr in an interference free way, without worrying about Magento specific things. Once we got that done, we'll implement this 'the Magento way'! (in another short post).

We'll use Solr as it was left in the previous post. That is, with the Charity information already indexed. Our goal is summarized in this UI sketch:

post-4-solr.png

This tutorial is separated in two parts and two documents. In the first, we set up the javascript search engine and the ajax call. In the second, we create the file which makes the request to ApacheSolr and returns its response to our search page.

In this example I am using a database of charities which we will filter by name, state and category.

1. Create a folder for our project and add jQuery (download it for free at www.jquery.com), or include it directly from google code (https://ajax.googleapis.com/ajax/libs/jquery/1.5.1/jquery.min.js).

2. Create two files, one is our index.php where we make the search form and display the results. The other file is ajaxSearch.php where we want to handle the interface with ApacheSolr.

3. In the first file we will make a simple form, and a div to show the results.

<form id="searchForm">
      <label for="tags">Tags: </label>
      <input id="charity_name" name="charity_name" value="<?php echo $_GET[charity_name] ?>" />
      <label for="region_id">State</label>
      <select title="State" name="region_id" id="region_id">
        <option value="">Please select a state</option>
        <option value="AL">Alabama</option><option value="AK">Alaska</option><option value="AS">American Samoa</option><option value="AZ">Arizona</option><option value="AR">Arkansas</option><option value="AF">Armed Forces Africa</option><option value="AA">Armed Forces Americas</option><option value="AC">Armed Forces Canada</option><option value="AE">Armed Forces Europe</option><option value="AM">Armed Forces Middle East</option><option value="AP">Armed Forces Pacific</option><option value="CA">California</option><option value="CO">Colorado</option><option value="CT">Connecticut</option><option value="DE">Delaware</option><option value="DC">District of Columbia</option><option value="FM">Federated States Of Micronesia</option><option value="FL">Florida</option><option value="GA">Georgia</option><option value="GU">Guam</option><option value="HI">Hawaii</option><option value="ID">Idaho</option><option value="IL">Illinois</option><option value="IN">Indiana</option><option value="IA">Iowa</option><option value="KS">Kansas</option><option value="KY">Kentucky</option><option value="LA">Louisiana</option><option value="ME">Maine</option><option value="MH">Marshall Islands</option><option value="MD">Maryland</option><option value="MA">Massachusetts</option><option value="MI">Michigan</option><option value="MN">Minnesota</option><option value="MS">Mississippi</option><option value="MO">Missouri</option><option value="MT">Montana</option><option value="NE">Nebraska</option><option value="NV">Nevada</option><option value="NH">New Hampshire</option><option value="NJ">New Jersey</option><option value="NM">New Mexico</option><option value="NY">New York</option><option value="NC">North Carolina</option><option value="ND">North Dakota</option><option value="MP">Northern Mariana Islands</option><option value="OH">Ohio</option><option value="OK">Oklahoma</option><option value="OR">Oregon</option><option value="PW">Palau</option><option value="PA">Pennsylvania</option><option value="PR">Puerto Rico</option><option value="RI">Rhode Island</option><option value="SC">South Carolina</option><option value="SD">South Dakota</option><option value="TN">Tennessee</option><option value="TX">Texas</option><option value="UT">Utah</option><option value="VT">Vermont</option><option value="VI">Virgin Islands</option><option value="VA">Virginia</option><option value="WA">Washington</option><option value="WV">West Virginia</option><option value="WI">Wisconsin</option><option value="WY">Wyoming</option>        </select>
      <label for="area">Area of Interest</label>
      <select name="area" id="area" class="" title="">
        <option value="">Select a category</option>
        <option value="K">Agriculture </option><option value="D">Animals and Wildlife</option><option value="A">Arts, Culture and Humanities</option><option value="O">Children and Teens</option><option value="X">Church and Religion</option><option value="R">Civil Rights</option><option value="S">Community Improvement</option><option value="G">Disease</option><option value="B">Education and Schools</option><option value="C">Environment</option><option value="Q">Foreign Affairs</option><option value="L">Housing</option><option value="J">Jobs and Employment</option><option value="I">Legal</option><option value="H">Medical Research</option><option value="F">Mental Health</option><option value="E">Nutrition and Wellbeing</option><option value="Z">Other</option><option value="T">Philanthropy</option><option value="W">Public Charites</option><option value="M">Public Safety</option><option value="U">Science and Technology </option><option value="Y">Social Organizations</option><option value="V">Social Science Research</option><option value="P">Social Services</option><option value="N">Sports and Recreation</option>
      </select>
    </form>
    <div id="searchResult">

    </div>

 

Search form

4. In the head, make a script tag. We dont have a submit button, so we need to listen for changes in our input fields and select dropdowns. If our elements change, a event is fired and refreshResult is called.

$(function(){
    $("#region_id,#area").change(function(){
      refreshResult();
    });
    $("#charity_name").keyup(function(){
      refreshResult();
    });
});
5. Create a function to make the ajax call, but first we need some validation and time limits. Because these functions are fired when the input changes, we don't want to make a call for each pushed key, so we add a short delay, in this case 300 ms. And in the validation, we check if the input has at least three characters.
function refreshResult(page){
  var timer;
  if(!page){
    page=1;
  }
  // If the timer is running we stop it.
  clearTimeout(timer);
  // Restart the timer, set the ajax call for 300 ms (If this function hasn't been called again during that time)
  timer=setTimeout(function(){
    timerRunning=false;
    // Make the call if the length is more than two characters
    if($("#charity_name").val().length>2){
      // Here you make the ajax call
    }
  }, 300);
}
6. Now make the ajax call, we want to pass our values and the page number to ajaxSearch.php. And put the response data in the #searchResult div.
$.get("ajaxSearch.php",
{
  search:1,
  page:page,
  charity_name:$("#charity_name").val(),
  region_id:$("#region_id").val(),
  area:$("#area").val()
},function(data){
  $("#searchResult").html(data);
});
7. Go to the other file, first we prepare our request, escaping any Solr special characters. For this, I use a function taken from this post http://e-mats.org/2010/01/escaping-characters-in-a-solr-query-solr-url/
A lot of this code is for the pager. I'm not going to explain that here, but it's not too complicated.
if ($_GET["search"]) {
  $rows = 0;
  $itemsPerPage = 50;
  $startItem = ($_GET["page"] - 1) * $itemsPerPage;
  // The url of my solr
  $requestUrl = "http://127.0.0.1:8983/solr/select/?";
  $queryElements = array();
  // Here we have all the query elements, data type, order, first item and max number of items to show.
  $queryElements[] = "wt=json";
  $queryElements[] = "sort=score+desc";
  $queryElements[] = "start=" . $startItem;
  $queryElements[] = "rows=" . $itemsPerPage;
  // Escape the values
  $charity["name"] = escapeSolrValue($_GET["charity_name"]);
  $charity["region"] = urlencode(escapeSolrValue($_GET["region_id"]));
  $charity["area"] = urlencode(escapeSolrValue($_GET["area"]));
  $fqField = array();
  // If a region or area exists add it to the query.
  if ($charity["region"]) {
    $qField[] = "charity_state_en:" . $charity["region"];
  }
  if ($charity["area"]) {
    $qField[] = "charity_category_en:" . $charity["area"];
  }
  if ($charity["name"]) {
    // When a user puts more than one word, by default ApacheSolr searches one word OR the other word. So we separate the words and join them with AND as this suits our use case.
    $_namesArray=explode("\ ",$charity["name"]);
    $_namesArray[count($_namesArray)-1].="*";
    foreach ($_namesArray as $key => $value) {
      $_namesArray[$key]='charity_name_en:'.$value;
    }
    $_namesParsed=implode(" AND ",$_namesArray);
    $qField[] = $_namesParsed;
  }
  $qFieldsParsed = implode(" AND ", $qField);
  // We pass the string to a validate url
  $queryElements[] = "q=" . urlencode($qFieldsParsed);
  $queryElementsParsed = implode("&", $queryElements);
  $requestUrl.=$queryElementsParsed;
  // We make the request
  $jsonResponse = file_get_contents($requestUrl);
  // We pass the json response to an array and we take the docs and the number of items found and calculate the number of pages
  $response = json_decode($jsonResponse, true);
  $docs = $response["response"]["docs"];
  $rows = $response["response"]["numFound"];
  $pages = floor($rows / $itemsPerPage);
  if ($rows % $itemsPerPage > 0) {
    $pages++;
  }
}
8. Now we just need to display the content, which we do with a simple foreach for the result, and a bunch of code for the pager.
<div>
  <?php
  if ($pages > 1) {
    $firstItem = $startItem + 1;
    $lastItem = $startItem + $itemsPerPage;
    if ($lastItem > $rows) {
      $lastItem = $rows;
    }
    echo "Viewing " . $firstItem . " to " . $lastItem . " from ".$rows."</br>";
    echo "Page: <select id=\"pager\">";
    for ($i = 1; $i <= $pages; $i++) {
      echo '<option value="' . $i . '" ';
      if ($_GET["page"] == $i

        )echo "selected=\"selected\"";echo'>' . $i . '</option> ';
    }
    echo "</select>";
  }
  ?>
</div>
<table>
  <tr>
    <td>ID</td>
    <td>Name</td>
    <td>Area</td>
    <td>State</td>
  </tr>
  <?php
  if ($_GET["search"]) {
    foreach ($docs as $key => $doc) {
      echo "
              <tr>
                <td>" . $doc["id"] . "</td>
                <td>" . $doc["charity_name_en"] . "</td>
                <td>" . $doc["charity_category_en"] . "</td>
                <td>" . $doc["charity_state_en"] . "</td>
              </tr>
      ";
    }
  }
  ?>
</table>
9. In the first file we add this line, which changes the data when we change the page.
$("#pager").live("change",function(data){
  refreshResult($(this).val());
});
Our finished files:

index.php

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
    <title>Charities Locator</title>
    <script type="text/javascript" src="jquery-1.4.4.min.js"></script>
    <script type="text/javascript">
      $(function(){
        function refreshResult(page){
          var timer;
          clearTimeout(timer);
          if(!page){
            page=1;
          }
          timer=setTimeout(function(){
            timerRunning=false;
            if($("#charity_name").val().length>2){
              $.get("ajaxSearch.php",
              {
                search:1,
                page:page,
                charity_name:$("#charity_name").val(),
                region_id:$("#region_id").val(),
                area:$("#area").val()
              },function(data){
                $("#searchResult").html(data);
              });
            }
          }, 300);
        }
        $("#region_id,#area").change(function(){
          refreshResult();
        });
        $("#charity_name").keyup(function(){
          refreshResult();
        });
        $("#pager").live("change",function(data){
          refreshResult($(this).val());
        });
      });
    </script>
  </head>
  <body>
    <form id="searchForm">
      <label for="tags">Tags: </label>
      <input id="charity_name" name="charity_name" value="<?php echo $_GET[charity_name] ?>" />
      <label for="region_id">State</label>
      <select title="State" name="region_id" id="region_id">
        <option value="">Please select a state</option>
        <option value="AL">Alabama</option><option value="AK">Alaska</option><option value="AS">American Samoa</option><option value="AZ">Arizona</option><option value="AR">Arkansas</option><option value="AF">Armed Forces Africa</option><option value="AA">Armed Forces Americas</option><option value="AC">Armed Forces Canada</option><option value="AE">Armed Forces Europe</option><option value="AM">Armed Forces Middle East</option><option value="AP">Armed Forces Pacific</option><option value="CA">California</option><option value="CO">Colorado</option><option value="CT">Connecticut</option><option value="DE">Delaware</option><option value="DC">District of Columbia</option><option value="FM">Federated States Of Micronesia</option><option value="FL">Florida</option><option value="GA">Georgia</option><option value="GU">Guam</option><option value="HI">Hawaii</option><option value="ID">Idaho</option><option value="IL">Illinois</option><option value="IN">Indiana</option><option value="IA">Iowa</option><option value="KS">Kansas</option><option value="KY">Kentucky</option><option value="LA">Louisiana</option><option value="ME">Maine</option><option value="MH">Marshall Islands</option><option value="MD">Maryland</option><option value="MA">Massachusetts</option><option value="MI">Michigan</option><option value="MN">Minnesota</option><option value="MS">Mississippi</option><option value="MO">Missouri</option><option value="MT">Montana</option><option value="NE">Nebraska</option><option value="NV">Nevada</option><option value="NH">New Hampshire</option><option value="NJ">New Jersey</option><option value="NM">New Mexico</option><option value="NY">New York</option><option value="NC">North Carolina</option><option value="ND">North Dakota</option><option value="MP">Northern Mariana Islands</option><option value="OH">Ohio</option><option value="OK">Oklahoma</option><option value="OR">Oregon</option><option value="PW">Palau</option><option value="PA">Pennsylvania</option><option value="PR">Puerto Rico</option><option value="RI">Rhode Island</option><option value="SC">South Carolina</option><option value="SD">South Dakota</option><option value="TN">Tennessee</option><option value="TX">Texas</option><option value="UT">Utah</option><option value="VT">Vermont</option><option value="VI">Virgin Islands</option><option value="VA">Virginia</option><option value="WA">Washington</option><option value="WV">West Virginia</option><option value="WI">Wisconsin</option><option value="WY">Wyoming</option>        </select>
      <label for="area">Area of Interest</label>
      <select name="area" id="area" class="" title="">
        <option value="">Select a category</option>
        <option value="K">Agriculture </option><option value="D">Animals and Wildlife</option><option value="A">Arts, Culture and Humanities</option><option value="O">Children and Teens</option><option value="X">Church and Religion</option><option value="R">Civil Rights</option><option value="S">Community Improvement</option><option value="G">Disease</option><option value="B">Education and Schools</option><option value="C">Environment</option><option value="Q">Foreign Affairs</option><option value="L">Housing</option><option value="J">Jobs and Employment</option><option value="I">Legal</option><option value="H">Medical Research</option><option value="F">Mental Health</option><option value="E">Nutrition and Wellbeing</option><option value="Z">Other</option><option value="T">Philanthropy</option><option value="W">Public Charites</option><option value="M">Public Safety</option><option value="U">Science and Technology </option><option value="Y">Social Organizations</option><option value="V">Social Science Research</option><option value="P">Social Services</option><option value="N">Sports and Recreation</option>
      </select>
    </form>
    <div id="searchResult">

    </div>
  </body>
</html>

ajaxSearch.php

<?php

function escapeSolrValue($string) {
  $match = array('\\', '+', '-', '&', '|', '!', '(', ')', '{', '}', '[', ']', '^', '~', '*', '?', ':', '"', ';', ' ');
  $replace = array('\\\\', '\\+', '\\-', '\\&', '\\|', '\\!', '\\(', '\\)', '\\{', '\\}', '\\[', '\\]', '\\^', '\\~', '\\*', '\\?', '\\:', '\\"', '\\;', '\\ ');
  $string = str_replace($match, $replace, $string);

  return $string;
}

if ($_GET["search"]) {
  $rows = 0;
  $itemsPerPage = 50;
  $startItem = ($_GET["page"] - 1) * $itemsPerPage;
  $requestUrl = "http://127.0.0.1:8983/solr/select/?";
  $queryElements = array();
  $queryElements[] = "wt=json";
  $queryElements[] = "sort=score+desc";
  $queryElements[] = "debugQuery=true";
  $queryElements[] = "indent=true";
  $queryElements[] = "start=" . $startItem;
  $queryElements[] = "rows=" . $itemsPerPage;
  $charity["name"] = escapeSolrValue($_GET["charity_name"]);
  $charity["region"] = urlencode(escapeSolrValue($_GET["region_id"]));
  $charity["area"] = urlencode(escapeSolrValue($_GET["area"]));
  $fqField = array();
  if ($charity["region"]) {
    $qField[] = "charity_state_en:" . $charity["region"];
  }
  if ($charity["area"]) {
    $qField[] = "charity_category_en:" . $charity["area"];
  }
  if ($charity["name"]) {
    $_namesArray=explode("\ ",$charity["name"]);
    $_namesArray[count($_namesArray)-1].="*";
    foreach ($_namesArray as $key => $value) {
      $_namesArray[$key]='charity_name_en:'.$value;
    }
    $_namesParsed=implode(" AND ",$_namesArray);
    $qField[] = $_namesParsed;
  }
  $qFieldsParsed = implode(" AND ", $qField);
  $queryElements[] = "q=" . urlencode($qFieldsParsed);
  $queryElementsParsed = implode("&", $queryElements);
  $requestUrl.=$queryElementsParsed;
  $jsonResponse = file_get_contents($requestUrl);
  $response = json_decode($jsonResponse, true);
  $docs = $response["response"]["docs"];
  $rows = $response["response"]["numFound"];
  $pages = floor($rows / $itemsPerPage);
  if ($rows % $itemsPerPage > 0) {
    $pages++;
  }
}
?>
<div>
  <?php
  if ($pages > 1) {
    $firstItem = $startItem + 1;
    $lastItem = $startItem + $itemsPerPage;
    if ($lastItem > $rows) {
      $lastItem = $rows;
    }
    echo "Viewing " . $firstItem . " to " . $lastItem . " from ".$rows."</br>";
    echo "Page: <select id=\"pager\">";
    for ($i = 1; $i <= $pages; $i++) {
      echo '<option value="' . $i . '" ';
      if ($_GET["page"] == $i

        )echo "selected=\"selected\"";echo'>' . $i . '</option> ';
    }
    echo "</select>";
  }
  ?>
</div>
<table>
  <tr>
    <td>ID</td>
    <td>Name</td>
    <td>Area</td>
    <td>State</td>
  </tr>
  <?php
  if ($_GET["search"]) {
    foreach ($docs as $key => $doc) {
      echo "
              <tr>
                <td>" . $doc["id"] . "</td>
                <td>" . $doc["charity_name_en"] . "</td>
                <td>" . $doc["charity_category_en"] . "</td>
                <td>" . $doc["charity_state_en"] . "</td>
              </tr>
      ";
    }
  }
  ?>
</table>
And this is the result:
Search filtered by name:
Search filtered for name
As we wanted, the information in the results changes as the user types. No need to manually refresh it by clicking on search buttons! how cool is that? :)
This is another screenshot, this time adding a filter by state:
Search filtered for name and state
The whole list now is narrowed down to those items in the "AK" state.
The apache solr console also shows interesting info. Note the time of the query, the result and the query itself!
Apache Solr console
Hope you find this useful! Coming up next, the season finale! Creating the final "Charity Locator" in Magento. 

Author

roque
PHP Developer
I'm a young developer from Argentina, since I was a child I have a computer under my control. I am allways searching for new challenges, and learning new technologies. My specialty is drupal and...
Other post from this author

Add comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.

Filter by tags

Acceptance Test
1
Agile
1
ajax
1
Algorithm
2
Android
1
Android
1
Apachesolr
6
C#
2
CMS
14
codesniffer
1
Coding
1
coding standard
1
Dependency Injection
1
Design and Theming
1
design patterns
1
Development
34
DI
1
DOM
1
DotNet
8
Drupal
7
Drupal 7
1
Drush
1
E-Commerce
20
Flash Builder
1
Interface
1
IoC
1
iOS
1
Javascript
4
jQuery
2
Kiosk
1
Magento
24
MagentoImagine
3
math
1
Mobile
1
netbeans
1
panels
1
pathfinding
1
php
1
phpmd
1
search engine
1
SEO
1
Software architecture
2
Unity
1
views
1
zend
1

Top Bloggers

aldo (7)
matias (7)
christian (5)
roque (5)
agustin (2)
jose (1)
tomas (1)

Related Content

Magento - Apache Solr Integration - Part I (intro)
Magento - Apache Solr Integration - Part II (setup)
Magento - Apache Solr Integration - Part III (indexing custom data)
Muddling, meddling and getting it right with Apache Solr
Muddling, meddling and getting it right with Apache Solr - part 2