Dirty Butter Posted November 15, 2014 Share Posted November 15, 2014 I tried Disallowing the category pages with View All (?page=all) chosen from robots.txt, because Google kept warning that these pages have duplicate metadata to the Category pages. Well, of course they do - they are the same thing, just looking at all items on one long very slow loading page. I also "removed" all those View All Category pages from Google's Search. (I was successful in disallowing in robots.txt all the page=1,2,etc pages without any issues popping up.) What did that get me???? A Google Webmaster Tools notice of "severe health issues on your site - some important page is being blocked by robots.txt" AND WORSE... I now have an 80% DROP in searches over the last month! Blocking those pages has hurt us far more than being concerned about Google's stupid warnings of duplicate metadata. So it now becomes necessary to plead for help satisfying Google with some way to give the View All page for each Category its own set of metadata. It's nonsensical, I know, but being in almighty Google's bad graces has hurt our bottom line terribly. Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 15, 2014 Share Posted November 15, 2014 "they are the same thing, just looking at all items on one long very slow loading page" What are your thoughts on removing the "All" (page=all) link from the pagination? Personally, I rarely!!! click the "Show All" link at any site I visit. Quote Link to comment Share on other sites More sharing options...
disco_ii_disco Posted November 15, 2014 Share Posted November 15, 2014 Would the best option be to change the canonical urls for each category page to either the first page or the view all page? I am addressing this in my new Cubecart 5 store now. Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 15, 2014 Author Share Posted November 15, 2014 Now that you have recently greatly improved our Search function so much, Bsmither, getting rid of View All might make an easy way around it. I do not think changing the canonical for the Category page to the page=all is the way to go for us, as some of our categories have so many items in them. Maybe that would work for disco... and others, but not for us. I think I'll try commenting out the View All button for now in element.paginate.php and give it a week to see what happens. No, that's not going to work. I took the View All button off, but the page=all DOES actually exist, and thus can still be linked to. So wherever Google is getting that page from - maybe some aggregate sites?? - they will still show up as search links. Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 15, 2014 Share Posted November 15, 2014 Of course, removing a link to a page does not invalidate the actual page. (Removing the link is a trick still used by some in the CC3/4 crowd to "fake" the unavailability of the "Tell a Friend" function which was being abused.) So, if we are sticking to this plan of invalidating the page=all querystring, how would you want CubeCart to respond when this querystring is present? Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 15, 2014 Author Share Posted November 15, 2014 Based on the persistence of Google's "serious site issue" when I disallowed page=all - I do NOT want to invalidate the page=all url's. I need to either create metadata for page=all OR allow page=all and ignore Google's duplicate meta warnings. Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 15, 2014 Share Posted November 15, 2014 "I need to either create metadata for page=all OR allow page=all and ignore Google's duplicate meta warnings." We will try to come to some solution with custom metadata when the querystring includes page=all. Being aware that pagination generally happens only on a list of products (that is, does CubeCart have pagination when the number of a category's subcategories exceed a limit?), will there be different custom metadata for page=all for saleitems, search results, and each specific category? So what do you propose the custom metadata for page=all will say? On what screen in admin will this metadata be entered? Store Settings for saleitems and search results? Per Category? Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 15, 2014 Author Share Posted November 15, 2014 I had assumed I would need to create a different page=all for each category and subcategory. Otherwise I'm STILL going to get duplicate metadata warnings from Google. So it would make sense for it to be with each of the Categories/Subcategories. I have your Meta Data code for SaleItems in use at this time. And I have Search results Disallowed in robots.txt, and that appears to be working to Google's satisfaction at this time. IF the page=all metadata could be on the same Search Engines tab currently found with categories/children - that would make copy/paste from the Category to the Page=all SO much easier. All I would have to do then would be to slightly alter the wording. IF NOT? That's fine, too! Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 15, 2014 Share Posted November 15, 2014 I wonder of the robots file could be written so that the robot must look at page=all and no other? Or maybe a snippet that if the visitor is identified as a robot, send a 302 to page=all? Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 15, 2014 Author Share Posted November 15, 2014 I wonder of the robots file could be written so that the robot must look at page=all and no other? Or maybe a snippet that if the visitor is identified as a robot, send a 302 to page=all? We must have posted simultaneously. I defer to you on that!!! Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 15, 2014 Share Posted November 15, 2014 "copy/paste from the Category [seo form field] to the Page=all [new form field]" You haven't given a specific example. But, maybe a universal tweak to the phrase? Such as: <link rel="canonical" href="http://dirtybutter.com/plushcatalog/food-related-advertising.html" /> <link rel="canonical" href="http://dirtybutter.com/plushcatalog/food-related-advertising.html?page=2" /> To be: <link rel="canonical" href="http://dirtybutter.com/plushcatalog/all-of-food-related-advertising.html" /> But how does this solve anything? This unique canonical phrase still has the same content found on all the other pages. Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 15, 2014 Author Share Posted November 15, 2014 <link rel="canonical" href="http://dirtybutter.com/plushcatalog/all-of-food-related-advertising.html" /> But how does this solve anything? This unique canonical phrase still has the same content found on all the other pages. Well, the content on page1, etc., really does NOT have the same content as ?page=all. BUT as it stands, all those page1, =all, etc. pages per category DO have the same seo title, description, and keywords. Let me try to give a complete example for Special Occasions Category>Search Engines Tab: Meta Title --- Special Occasions Plush Custom SEO URL Path --- special-occasions Meta Keywords --- holiday,christmas,halloween,valentine's day,valentines,easter,toy,plush,stuffed,animal,doll,soft,birthday,first birthday,1st birthday,special,occasion Meta Description --- We carry a wide variety of plush animals and soft dolls perfect for Special Occasions like Birthday, Christmas, Valentine's Day, Easter, and Halloween celebrations. NEW VIEW ALL Meta Title --- All Our Special Occasions Plush Custom SEO URL Path --- all-our-special-occasions Meta Keywords --- holiday,christmas,halloween,valentine's day,valentines,easter,toy,plush,stuffed,animal,doll,soft,birthday,first birthday,1st birthday,special,occasion Meta Description --- We carry a wide variety of plush animals and soft dolls perfect for Special Occasions like Birthday, Christmas, Valentine's Day, Easter, and Halloween celebrations. See all our Special Occasions Plush on one page. Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 15, 2014 Share Posted November 15, 2014 Oops. I was getting the seo metadata confused with the canonical URL. Let me reread everything. You are concerned about duplicate metadata? So, what is your response to: "BUT as it stands, all those page1, =all, etc. pages per category DO have the same seo title, description, and keywords." If you accept the "sameness of the metadata" situation for (implied page=1) vs. page=2 vs. page=3, why make a special case out of page=all? Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 15, 2014 Share Posted November 15, 2014 If you think that these general additions can apply to each and every category, then that would be a boon. NEW VIEW ALL General Addition: Meta Title --- All Our whatever already exists (code will add a space) Custom SEO URL Path --- all-our-whatever already exists Meta Description --- Whatever already exists. See all our META_TITLE on one page. (Code will add spaces) Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 16, 2014 Share Posted November 16, 2014 Cannot do Custom SEO URL Path --- all-our-whatever already exists, because all-our-special-occasions would have to be in the seo_path database table for the actual URL, from which the path is used to create, to work: www.store.com/all-our-special-occasions.html Although, with some more code in the SEO class, this could be made to work. Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 16, 2014 Author Share Posted November 16, 2014 The unique urls are already there, as each is made by taking the category url and adding ?page=all to the end of it. And #14 looks like it would work, not only for me, but for anyone who feels the need to do this. But the SEO URL is not a problem. The sameness of page=1, page=2, etc. doesn't matter because I have that disallowed in robots.txt and so far Google has respected that. The only one that seems crucial is page=all. Past my bedtime, but I'll work on this again after church tomorrow. As always, your efforts to help are very much appreciated!!! Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 16, 2014 Share Posted November 16, 2014 Here is the first version of enhancing the HTML meta data tags for the case when the View Catageory page has page=all. This includes pseudo categories such as Sale Items and (maybe) Search Results. Do not do this. See post #50. In the admin template file categories.index.php: Find: {/if} {if isset($MODE_ADDEDIT)} Add above it: {* NEW *} <div id="addl_seo" class="tab_content"> <h3>Additional page=all {$LANG.settings.title_seo}</h3> <fieldset> <div><label for="seo_meta_title_all">Additional page=all Meta Title</label><span><input type="text" name="seo[meta_title_all]" id="seo_meta_title_all" class="textbox" value="{$seo_meta_title_all}" /></span></div> {* <div><label for="seo_path_all">Additional page=all SEO Path</label><span><input type="text" name="seo[path_all]" id="seo_path_all" class="textbox" value="{$seo_path_all}" /></span></div> *} <div><label for="seo_meta_description_all">Additional page=all Meta Description<br /><br />%X for existing, %Y for Meta Title<br /> Example:<br /> All Meta Title-- <tt>All Our %X</tt><br /> {* All SEO Path-- <tt>all-our-%X</tt><br /> *} All Meta Desc-- <tt>%X See all our %Y on one page.</tt></label><span><textarea name="seo[meta_description_all]" id="seo_meta_description_all" class="textbox">{$seo_meta_description_all}</textarea></span></div> </fieldset> </div> {* NEW *} Do not do this. See post #50. In the file /admin/sources/categories.index.inc.php: Find: if (!empty($update) && is_array($update) && Admin::getInstance()->permissions('categories', CC_PERM_EDIT)) { Add above it: /* NEW */ if (isset($_POST['seo']) && is_array($_POST['seo'])) $GLOBALS['config']->set('config', 'catsAddlSEO', $_POST['seo']); /* NEW */ Find: $GLOBALS['main']->addTabControl($lang['settings']['title_category_add'], null, currentPage(null, array('action' => 'add'))); if (($categories = $GLOBALS['db']->select('CubeCart_category', false, array('cat_parent_id' => (isset($_GET['parent'])) ? (int)$_GET['parent'] : 0), array('priority' => 'ASC'))) !== false) { $i = 1; Add above it: /* NEW */ $GLOBALS['main']->addTabControl($lang['settings']['tab_seo'], 'addl_seo'); /* NEW */ Find: } $page_content = $GLOBALS['smarty']->fetch('templates/categories.index.php'); Add above it: /* NEW */ $catsAddlSEO = $GLOBALS['config']->get('config', 'catsAddlSEO'); $GLOBALS['smarty']->assign("seo_meta_title_all",$catsAddlSEO['meta_title_all']); /* $GLOBALS['smarty']->assign("seo_path_all",$catsAddlSEO['path_all']); */ $GLOBALS['smarty']->assign("seo_meta_description_all",$catsAddlSEO['meta_description_all']); /* NEW */ In the file /classes/catalogue.class.php, in the displayCategory() function: Find: $GLOBALS['seo']->set_meta_data($meta_data); Add above it: /* NEW */ $page = (isset($_REQUEST['page']) && !empty($_GET['page'])) ? $_REQUEST['page'] : 1; $catsAddlSEO = $GLOBALS['config']->get('config', 'catsAddlSEO'); if(!empty($catsAddlSEO) && $page == 'all'){ $meta_data['description'] = str_replace('%X',$meta_data['description'],$catsAddlSEO['meta_description_all']); $meta_data['description'] = str_replace('%Y',$meta_data['title'],$meta_data['description']); $meta_data['title'] = str_replace('%X',$meta_data['title'],$catsAddlSEO['meta_title_all']); // $meta_data['title'] = str_replace('%Y',$meta_data['title'],$meta_data['meta_title_all']); } /* NEW */ PLEASE SEE POST #31 regarding the following. In the file /classes/config.class.php, we need to fix (in my opinion, this) a bug. In the __construct() and _fetchConfig() functions: Find: $array_out = $this->_json_decode($result[0]['array']); Add below it: foreach($array_out as $key => $val){ if(is_null($this->_json_decode($val))) {continue;} $array_out[$key] = $this->_json_decode($val); } // Github Issue #198 In admin, Categories, there will be a new Search Engines tab. Just add the enhanced text. This will not directly replace any existing meta_data text, but will incorporate it. The bug fix has not been extensively tested. I do not know how it may impact other modules' ability to retrieve their config data. So use with all due caution. Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 16, 2014 Author Share Posted November 16, 2014 I tried it without using your bug fix code. The %X text and %Y text did NOT Save. Instead a lower case e was saved. Loading a View All page DID work, although the browser tab shows the title as "e" and this debug warning shows: PHP: [Warning] /home3/butter01/public_html/plushcatalog/classes/catalogue.class.php:143 - Illegal string offset 'meta_description_all' [Warning] /home3/butter01/public_html/plushcatalog/classes/catalogue.class.php:145 - Illegal string offset 'meta_title_all' WITH your bug fix for Github #198 it works PERFECTLY!! I did take a space out, not that it really matters. "%X See all our %Yon one page." Not sure how I got the space after my period in there to begin with. Will test with some other categories. In randomly testing I found a category that still has an extra space. So I'll go back to the original %Y on one page and make the corrections from cPanel to get rid of all trailing spaces in category titles. You have taken a HUGE weight off my shoulders, Bsmither. There is NO WAY I can thank you enough!!!!!!! Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 21, 2014 Author Share Posted November 21, 2014 I have just realized that sub-category page=all pages are NOT using the new metadata code you created. Could you help with that, too? Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 21, 2014 Author Share Posted November 21, 2014 NO - that's not the problem - the Search Engine page your code created is now EMPTY! Not sure what happened. I'll do some testing and get back to you. Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 21, 2014 Share Posted November 21, 2014 "sub-category page=all pages" A sub-category is no different than a top-level category except there is one or more extra crumbs in the breadcrumb trail. Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 21, 2014 Author Share Posted November 21, 2014 I've cleared all cache, restarted computer, optimized the database, done everything I can think of that might have deleted the info on that new Search Engines page. So somehow I must have deleted it all myself. VERY weird. Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 22, 2014 Share Posted November 22, 2014 Maybe not so much deleted, as it wasn't included on the latest save of the general Config array() - or, more likely, it didn't get properly 'decoded' so that it didn't get properly loaded back onto the form, which then saved the config missing those values. (See github issue 198 and Post#18 above.) Quote Link to comment Share on other sites More sharing options...
Dirty Butter Posted November 22, 2014 Author Share Posted November 22, 2014 I read the GitHub #198, but this is all way over my head. Thank goodness there are people like you who can debug such issues and hopefully find a solution. Quote Link to comment Share on other sites More sharing options...
bsmither Posted November 22, 2014 Share Posted November 22, 2014 Did you re-enter the phrases, and have they disappeared again? Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.