Jump to content

fuzzy search


sean1688

Recommended Posts

Well, it seems that for this specific part of the program, it does not include the sorter data. Let's fix that.

In /classes/catalogue.class.php, near line 1571, find:
if ($search_mode == 'fulltext' && $search_str_len >= $max_word_len) {

Next, find:
} else {
that corresponds to the above if().

Next, find the statement that begins with:
$q2 = "SELECT I.* FROM

At the end of that statement, it should look like:
AND I.status = 1 ".$whereString.$rlike;

Change to:
AND I.status = 1 ".$whereString.$rlike." ".$order_string;

 

  • Like 1
Link to comment
Share on other sites

  • 4 months later...

A few things:

Does this v6 search fix allow for AND/OR?  This and search by Product ID were features of Cubecart v3 - http://www.cubecartforums.org/docs/CubeCart-v3/Understanding-CubeCart/Features.html

I had a v3 search mod set to return AND by default, so search URLs for AND were like:
/index.php?searchStr=address+book+violet&act=viewCat&Submit=Go

but these would work OK:
/index.php?searchStr=address+AND+book+AND+violet&act=viewCat&Submit=Go
/index.php?searchStr=address+AND+violet+OR+address+AND+blue&act=viewCat&Submit=Go
/index.php?searchStr=address+OR+album+OR+dragon&act=viewCat&Submit=Go

(I have Google Adwords keywords that link to specific search results.)

Also, any suggestions for implementing search changes into the admin/products side for v6?  It's really handy (using an earlier modded cart) to be able to filter products by category, then field, then ASC/DEC, then by search term with AND/OR (makes finding and editing a subgroup of matching products much easier).

Would be great if NOT was also an option so that some terms could be excluded, eg. fish NOT fishing, or PT15 NOT PT157 AND NOT PT159 (to list all PT15x items except two).

Edited by jasehead
Additional info
Link to comment
Share on other sites

To answer my own questions - here is the search behaviour:

  • address book violet = no results found (search for ANY PHRASE returns no results, would be better if search was converted to AND by default - eg. address AND book AND violet) - FIX AVAILABLE BELOW
  • address AND book AND violet = works ok
  • address AND violet OR address AND blue = works ok
  • address OR album OR dragon = works ok
  • address NOT violet = no results found (NOT isn't supported) - FIX AVAILABLE BELOW
  • +address -violet = no results found (+/- operators not supported)
  • default sort set by admin does not apply to results from BASIC search, but the sort control in advanced search IS set by the default sort setting - so an ADVANCED search will respect the default sort set by admin (Store Settings > Layout > Product List Sorting).  The sort control on the search results is set to Product List Sorting after a basic search, but it has not been applied to the search results. - PARTIAL FIX AVAILABLE BELOW
  • _a=category key appears at end of URL for first search, but moves to start of query if results are sorted using the sort control on the search results page
  • including extra search fields like seo_meta_keywords and short_description will cause search to FAIL if you have no data in these fields - so you already need to be using them for data before you add either of these to search code in catalogue.class.php

Would be great if:

  • Basic search would adhere to the Product List Sorting set by admin - PARTIAL FIX AVAILABLE BELOW
  • Search included NOT as an option - FIX AVAILABLE BELOW
  • Search words remain in the search field when results are shown (easier to edit and try again)
  • Someone more skilled than me in regexfu could craft a RewriteRule that would convert incoming v3 mod search links to new v6 mod (including default sort) that would work with existing RewriteRules for v4 SEO in .htaccess:
    • FROM THIS - /index.php?searchStr=address+AND+book+AND+violet&act=viewCat&Submit=Go#AndDeleteAnythingAfterGo
    • TO THIS - /search.html?search%5Bkeywords%5D=address+AND+book+AND+violet&_a=category
    • OR THIS - /search.html?_a=category&search[keywords]=address%20AND%20book%20AND%20violet&sort[stock_level]=DESC
Link to comment
Share on other sites

FIX FOR SEARCH PHRASES - ADD THE 'AND', COPE WITH NOT

This line doesn't appear to be working:

$rlike .= " AND ";

I think this is supposed to insert AND between search words, so:

address book violet

is supposed to become:

address AND book AND violet

but it doesn't.

 

The fix below copes with: address book violet, address and book and not violet, address book not violet

 

Find:

if ( ($ucSearchTerm != 'AND') && ($ucSearchTerm != 'OR') && (strlen($ucSearchTerm) > 0) ) {

 

Replace with:

if ( ($ucSearchTerm != 'AND') && ($ucSearchTerm != 'OR') && ($ucSearchTerm != 'NOT') && (strlen($ucSearchTerm) > 0) ) {

 

After:

$rlike .= "(I.name LIKE '".$regexp."' OR I.description LIKE '".$regexp."' OR I.product_code LIKE '".$regexp."')";

insert:

$rlike = str_replace(array(')(',') NOT ('), array(') AND (',') AND NOT ('), $rlike);

 

FIX FOR SEARCH RESULTS ORDER

After:

                    $rlike .= " )";
                    }

 

Insert:

                    if(!$order_string){$order_string = 'ORDER BY stock_level DESC';}

Ideally, the $order_string should be the Product List Sorting set by admin but I couldn't work out how to get the data so it's just hardcoded.

Link to comment
Share on other sites

  • 8 months later...

@bmither - It's been 12 months and the default search code in CubeCart 6.1.5 is very different to what it was in 6.0.12.

In 6.0.12, searching by relevance did not allow me to restrict a search to specific products - I haven't yet tested 6.1.5 to see how search has changed, but I expect the modifications discussed here will still need to be implemented in some way.

Could you please check this modified 6.0.12 search code from catalogue.class.php and suggest how to implement it for 6.1.5?

Around 1413:

public function searchCatalogue($search_data = null, $page = 1, $per_page = 10, $search_mode = 'like' /* 'fulltext' */) { // search changed from fulltext to like

And this is the lump of search code starting around 1604:

/* START modified search
					} elseif ($search_mode == 'fulltext') {
						return $this->searchCatalogue($original_search_data, 1, $per_page, 'like');
END modified search */
					}

				} else {
					$this->_sort_by_relevance = false;
/* START modified search
					$rlike = '';
					if (!empty($search_data['keywords'])) {
						$searchwords = preg_split( '/[\s,]+/', $GLOBALS['db']->sqlSafe($search_data['keywords']));
						foreach ($searchwords as $word) {
							$searchArray[] = $word;
						}

						$noKeys = count($searchArray);
						$regexp = '';
						for ($i=0; $i<$noKeys; ++$i) {
							$ucSearchTerm = strtoupper($searchArray[$i]);
							if (($ucSearchTerm != 'AND') && ($ucSearchTerm != 'OR')) {
								$regexp .= '[[:<:]]'.$searchArray[$i].'[[:>:]].*';
							}
						}
						$regexp = substr($regexp, 0, strlen($regexp)-2);
						$rlike = " AND (I.name RLIKE '".$regexp."' OR I.description RLIKE '".$regexp."' OR I.product_code RLIKE '".$regexp."')";
					}
END modified search */

					if (!empty($search_data['keywords'])) {
						$rlike = ' AND ( ';
						$searchwords = preg_split( '/[\s,]+/', $GLOBALS['db']->sqlSafe($search_data['keywords']));
						foreach ($searchwords as $word) {
							$searchArray[] = $word;
						}

			        $noKeys = count($searchArray);
					for ($i=0; $i<$noKeys; ++$i) {
						$ucSearchTerm = strtoupper($searchArray[$i]);
						if ( ($ucSearchTerm != 'AND') && ($ucSearchTerm != 'OR') && ($ucSearchTerm != 'NOT') && (strlen($ucSearchTerm) > 0) ) {
							$searchArray[$i] = str_replace(array('%','_'), array('\%','\_'), $searchArray[$i]);
							$regexp = '%'.$searchArray[$i].'%';
							$rlike .= "(I.name LIKE '".$regexp."' OR I.description LIKE '".$regexp."' OR I.product_code LIKE '".$regexp."')"; // modified search - name, description, product code
							$rlike = str_replace(array(')(',') NOT ('), array(') AND (',') AND NOT ('), $rlike); // modified search - add AND between words and cope with NOT
//							if($i+1 < $noKeys){$rlike .= " AND ";} // modified search - add AND between words (line disabled)
						} else {
							if(strlen($ucSearchTerm) > 0){
              					$rlike .= " ".$ucSearchTerm." ";
                			} else {
                				$rlike .= " AND "; // THIS LINE IS NOT WORKING
                			}
		                continue;
            			}
        			}
			        $rlike .= " )";
			        $rlike = str_replace(') AND )',')', $rlike); // modified search - added to fix trailing AND (still a problem)
					}
//					if(!$order_string){$order_string = 'ORDER BY stock_level DESC';} // modified search
					if(!$order_string){$order_string = 'ORDER BY stock_level DESC, IF (G.sale_price IS NULL, IF (I.sale_price = 0, I.price, I.sale_price), IF (G.sale_price = 0, G.price, G.sale_price)) DESC';} // modified search - order by stock first, then sale price
					$q2 = "SELECT I.* FROM ".$GLOBALS['config']->get('config', 'dbprefix')."CubeCart_inventory AS I LEFT JOIN (SELECT product_id, MAX(price) as price, MAX(sale_price) as sale_price FROM ".$GLOBALS['config']->get('config', 'dbprefix')."CubeCart_pricing_group $group_id GROUP BY product_id) as G ON G.product_id = I.product_id $joinString WHERE I.product_id IN (SELECT product_id FROM `".$GLOBALS['config']->get('config', 'dbprefix')."CubeCart_category_index` as CI INNER JOIN ".$GLOBALS['config']->get('config', 'dbprefix')."CubeCart_category as C where CI.cat_id = C.cat_id AND C.status = 1) AND I.status = 1 ".$whereString.$rlike." ".$order_string;
// END modified search
					$query = $q2.' '.$limit;
					if (($search = $GLOBALS['db']->query($query)) !== false) {
						$count = $GLOBALS['db']->query($q2, false, 0);
						$this->_category_count  = (int)count($count);
						$this->_category_products = $search;
						return true;
// START modified search
					} elseif ($search_mode == 'like') {
						return $this->searchCatalogue($original_search_data, 1, $per_page, 'fulltext');
// END modified search

You may notice that I'm still having issues with a trailing AND in the search string - it throws errors but the search works.

The goal is to search for words or parts of words in name, description and product code. Operators like AND, OR, NOT all work OK (and in combination) and searching for a phrase like black ducks returns results for black AND ducks rather than "black ducks" as a strict phrase.

Any constructive suggestions are welcome.

Link to comment
Share on other sites

" In 6.0.12, searching by relevance did not allow me to restrict a search to specific products "

For example? Like only from a specific manufacturer? Featured Products only? Please refresh my memory on what you were trying to do.

In CC615, I suggest you change the initial search mode to 'rlike'. But if you do this, you should really remove the search sort option "Relevance". A call to this function must therefore specifically use the argument 'fulltext' in order to use it.

public function searchCatalogue($search_data = null, $page = 1, $per_page = 10, $search_mode = 'rlike') {

If nothing found, then a search on character sequences anywhere will be made ('like').

I had forgotten this:

Near line 1806, find:

if (($ucSearchTerm != 'AND') && ($ucSearchTerm != 'OR')) {

Change to:

if (($ucSearchTerm != 'AND') && ($ucSearchTerm != 'OR') && ($ucSearchTerm != 'NOT') && (strlen($ucSearchTerm) > 0)) {

Near line 1805, find:

$ucSearchTerm = strtoupper($searchArray[$i]);

Add after:

if(strtolower($search_mode) == 'like') $ucSearchTerm = str_replace(array('%','_'), array('\%','\_'), $ucSearchTerm);

The following are programming mistakes.

Near line 1810, find:

if($search_mode == 'RLIKE') {

Change to:

if(strtolower($search_mode) == 'rlike') {

Near line 1824, find:

} elseif($search_mode=="RLIKE") {

Change to:

} elseif(strtolower($search_mode) == 'rlike') {

Near line 1825, find:

return $this->searchCatalogue($original_search_data, 1, $per_page, 'LIKE');

Change to:

return $this->searchCatalogue($original_search_data, 1, $per_page, 'like');

The result will be a series of groups ANDed together. Each group searches on the keyword appearing in any of the code or name or description.

In one edit, you are passing over NOT. But in another edit, you are 'coping' with it.

I think there is no good way to deal with NOT as the search words are split on spaces and commas.

It would be interesting to try a preg_split() with NOT as an optional capture.

Same with quotes.

Link to comment
Share on other sites

Searching for specific products... Basically, I wanted PRECISE search results that were restricted to only the items I wanted to show to customers so that I could send a link (or a shortened version via bit.ly).

If I want to advertise five specific items then I can provide a link to search results for those specific items.

Or, if a customer asks for help with choosing then I can provide a link to recommendations based on their requirements.

I didn't want a trailing list of other "relevant" items in the search results, and I wanted to be able to use AND, OR and NOT to further refine search results.

My current 6.0.12 attempt DOES work (despite filling the error log) and, for me, is an improvement on the original relevant search which didn't behave the way I wanted it to.

Link to comment
Share on other sites

I don't see how a search could get that specific. But here is a workaround:

Create a specific category. Call it We Recommend. Set its visibility to off. Set its Custom SEO URL Path to recommended.

For each of the recommended products, on the Category tab, check the We Recommend category as an Additional category.

Send to your customer the link: www.mystore.com/recommended.html

If you are doing this for more than one customer at a time, you may need to create distinct categories.

Link to comment
Share on other sites

"We Recommend" won't work in my case - I want to be able to make as many links to custom search results as I like (for instance, grouping three product designs together but only showing a specific size - sizes are separate products in my store so it's easy).  If I want to search for a secret tag then I can just put that in <!--comments--> in the product description and my search will pick it up OK.

Like I said, my custom search using AND, OR & NOT is working just fine in 6.0.12 with only a syntax error (an extra AND) but that doesn't cause it to fall over - just have to get it working for 6.1.5.  My search is very specific and shows only the desired products without further items.  If I check my search log and see that people are searching for particular words or phrases that clearly wouldn't return good results, then that's a cue to me to improve my copy text for SEO - which is probably good practice anyway.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...