Jump to content

6.1.14 - Main Page blank except for "2018" after return from basket


jasehead

Recommended Posts

I've upgraded from 6.0.12 to 6.1.14 and I'm testing before going live, but I have an issue that breaks the store.  Here is the order of events:

  • Home page
  • Scroll to first item and add to basket
  • Scroll to second item and add to basket
  • Click View Basket - seems fine
  • Click Secure Checkout - seems fine
  • Changed my mind - click store logo to return to buy more
  • Scroll to first item and click again (forgot that I already did this)
  • BLANK PAGE with only 2018 visible
  • Refresh the page (because you would) - basket is now empty
  • Scroll and click first item - blank page with 2018 again
  • Refresh again
  • Scroll to another item I know I haven't clicked - 2018 blank page again

I've tested this on the Foundation skin as well, so it's not a skin issue.  I've also tried the vanilla 6.1.14 catalogue.class.php and it breaks with that too.  I'm wondering if CubeCart is over sensitive to the whole CSRF thing.

Safari 11.1 and Chrome 65.0 both break, but Firefox 59.0.1 correctly displays the shopping basket with a "You can't buy more than we have" alert.

Here are some headers from Safari for the broken page:

Request Headers

Name: Value
Accept-Encoding: br, gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15
Accept-Language: en-au

Resonse Headers

Name: Value
Content-Type: text/html; charset=UTF-8
Date: Sun, 15 Apr 2018 18:34:03 GMT
Accept-Ranges: bytes
Server: LiteSpeed
X-Powered-By: PHP/5.6.35
X-Frame-Options: SAMEORIGIN
Set-Cookie: CCS_569AEF99F1=67duu6cakceudp9bk9b7aeu7t0; expires=Sun, 22-Apr-2018 18:34:03 GMT; Max-Age=604800; path=/; domain=.private.com.au; secure; HttpOnly, CCS_569AEF99F1=67duu6cakceudp9bk9b7aeu7t0; expires=Sun, 22-Apr-2018 18:34:03 GMT; Max-Age=604800; path=/; domain=.private.com.au; secure; httponly
Content-Encoding: gzip
Vary: Accept-Encoding
alt-svc: quic=":443"; ma=2592000; v="35,37,38,39"

I'd be interested if anyone else can reproduce the problem or if it's just me.  Any advice on where to start or how to fix it?

Link to comment
Share on other sites

I set a product to have a stock level of 1, then added it to the cart - and viewed the Cart.

I then went back to the Homepage and clicked Add to cart on that same product.

I was taken to the product's page, with a banner that says: "Sorry but that product is out of stock." (Which is not exactly true, as "Can't buy more" is more appropriate.)

When viewing the Cart at checkout, changing the quantity, and clicking Update Basket, then I get "Sorry, you can't buy more than we have."

Disclaimer: I am using Firefox.

Link to comment
Share on other sites

I wasn't able to replicate the problem on Firefox, but Safari and Chrome both use webkit.

For copyright I show the year by the following code:

<noscript>2015</noscript><script type="text/javascript">document.write(new Date().getFullYear());</script>

So the javascript would write 2018 but the noscript is still 2015.  Weird that a script in the copyright would be the only part of the page to render.

Link to comment
Share on other sites

The other aspect of this problem is that the basket ends up empty.

I tested my 6.0.12 store and couldn't get it to fall over, only the 6.1.14 version.

Error Log is mostly full of:

"PHP Warning:  Invalid Security Token in /home2/myserver/public_html/cubecart/classes/sanitize.class.php on line 152"

and

"PHP Warning:  Stored session data did not match DB record. Session aborted as possible session hijack."

Old IP Address and New IP Address both the same, although the user agents are different:

Old User Agent: '&quot;Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1&quot;'
New User Agent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15'

"in /home2/myserver/public_html/cubecart/classes/session.class.php on line 702"

I've tried testing in the normal browser and in responsive view and the error will occur when testing in either mode and is not from switching modes during testing.

Edited by jasehead
Changed mycustomskin to cubecart, because classes were not in a skin.
Link to comment
Share on other sites

I was always puzzled by the disparity in User Agent strings in the same session. My initial guess was that somehow a user's phone and desktop were somehow sharing the same internet connection - and also sharing the same cookies.

But, I came to learn of a browser add-on or feature, or a computer security application like an anti-virus, that deals specifically with a method of foiling those who would try to fingerprint a user across an amalgamation of visited sites. A pro-active effort that routinely, at odd intervals of time, would choose a random User Agent string and send that in the request header.

So, in this regard, learn about the features of the browsers you use to discern if any use this "Don't even think about tracking me!" technique.

Invalid security token messages could be an older version of a skin running in CC6114. The latest version of Foundation is supposed to send the security token in the "Add to Basket" AJAX channel.

Any unique or one-off error messages?

So, some javascript runs -- what does the rest of the HTML source look like as attempted to be displayed by the browser? (Use the browser's View Source function.)

Link to comment
Share on other sites

There are no one-off error messages in the error log that coincide with the "2018" blank page.

In Chrome, view-source for the broken page shows the expected generated source as if the page was rendered properly.  The token in the URL is the same as the 15 occurrences of the token in the rendered page source.

But in Chrome's developer tools, the HTML from the blank page is pretty simple:

<html>
  <head></head>
  <body>2018</body>
</html>

This is an intermittent problem. Sometimes it's easy to make the cart fall over, other times it can take quite a lot of clicking around before it falls. If there was a code problem I'd expect it to break more reliably.  I'm leaning towards an issue with CubeCart's security token method and webkit browsers.

I've tested a vanilla Foundation skin and it still breaks, so the issue is not within the skin folder.  When transferring mods from 6.0.12, I took care to preserve as much of the 6.1.14 code as possible, so all the token calls are still there - most of my changes are cosmetic.  I have a heavily-modded catalogue.class.php (for search) but the vanilla version has the same result.  I've searched through all modded code and can't find anything that might cause an issue with security tokens.  Plugins and snippets are preorder plugin, partially shipped, and a tawk.to script at the end of main.php - but that's skin-specific.

I'm wondering if the Safari user agent changes are an attempt by Apple/webkit to address browser fingerprinting.  Would the same IP address but a user agent change upset Cubecart?

Although, I'm seeing errors with different user agents even from googlebot:

PHP Warning:  Stored session data did not match DB record. Session aborted as possible session hijack.
Old IP Address: '66.249.79.79'
New IP Address: '66.249.79.79'
Old User Agent: 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
New User Agent: 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
in /home2/myserver/public_html/cubecart/classes/session.class.php on line 702

Unless it's a reCaptcha issue.  When the page breaks, in Safari's debugger (sources) I see that api.js from Google says:
undefined is not an object (evaluating 's.parentNode') on line 22:7

^or maybe it breaks because the page didn't load properly, I don't know.

Edited by jasehead
reCaptcha
Link to comment
Share on other sites

A different User Agent string will destroy an existing session (based on the cookie). A destroyed session gets you logged out, and the shopping basket contents will be lost.

Google is wanting to validate your site is "mobile ready" - switching to Android and back.

You are saying there is a discrepancy between what Chrome shows as View Source versus what the Developer panels show as the current DOM?

There shouldn't be a token in any URL. A token gets POSTed (unless something has changed in CC6114).

Link to comment
Share on other sites

I don't know where the URL I copied was (probably in the developer tools), but I did copy a URL with a token the same as what was in the source.

All this is leading me towards not upgrading Cubecart to 6.1.14 and just sticking with 6.0.12 which seems more robust.  I can't be running a glitchy store that is going to serve up random blank pages and empty baskets - I'm just going to lose sales and customers as a result.  It's bad enough having a store that growls about CSRF - it's annoying to admin and it spooks customers.

OK, assume that a normal vanilla browser like Safari or Chrome may send different User Agent strings and that this is at least causing part of MY problem. (A mobile browser rotated during a session may also send different user agent strings.)

Is there anything I can change in Cubecart to make it less sensitive to user agent changes and still maintain some level of session cookie security.  Maybe change the user agent to a random number instead.

<?php

    $token = md5(uniqid(rand(), TRUE));
    $_SESSION['token'] = $token;

    ?>

Something like that, except still keep the IP address check and just replace the user agent with something random.

Link to comment
Share on other sites

We can render the user agent checker inoperative.

In /classes/session.class.php,

Near line 700, find:

if ($current[0]['useragent'] !== $this->_http_user_agent()) {

Change to:

if ( false && ($current[0]['useragent'] !== $this->_http_user_agent()) ) {

I still think there is a different reason why Chrome behaves differently than Firefox. But we will see.

Link to comment
Share on other sites

(sigh)

I made the change to session.class.php but I can still get the store to fall over with a blank "2018" page.

The error log* is no longer showing any user agent problems, but I am still getting lots of pairs of:

PHP Warning:  Invalid Security Token in /home2/myserver/public_html/cubecart/classes/sanitize.class.php on line 152
PHP Notice:  Undefined variable: _GET in /home2/myserver/public_html/cubecart/includes/functions.inc.php on line 196

It still seems to fall over at the Invalid Security Token.

I've found that I can make the "2018" error happen more reliably if I throw in a browser back button, but even if Cubecart doesn't like that it should still return a rendered page - and I've tested the javascript in my Copyright, the "2018" in the blank page is definitely generated from:

<script type="text/javascript">document.write(new Date().getFullYear());</script>

Of course, out of frustration, I could just change CSRF to false in the skin config.xml file - tempting.

This could all be avoided if the Add to Basket button wasn't available for items that were already consumed by the pending basket - it's clicking an unavailable item that causes the problem.  If a customer has your only copy already in their basket, then isn't it confusing for them to see the item as still available if they keep shopping?  An  Add to Basket button change (to Out of Stock?) should only apply for that customer during their session, not for everyone.  Although, it would make even more sense if the button or error messages said something like "Already in your basket" rather than "Out of stock" (Why out of stock? Has someone else already bought this while I was browsing?).  Better text and a disabled button would be the way to go because it would solve the problem, and also make the shopping experience better for the customer.

* by error log, I mean the error_log file in the base cubecart folder on the server, or the System Error Log in Admin if debugging has been enabled prior to testing the store.

Edited by jasehead
Link to comment
Share on other sites

I was able to replicate the problem with the latest Firefox 59.0.2 for Mac.

The only item in the error log this time was:

PHP Warning:  Invalid Security Token in /home2/myserver/public_html/cubecart/classes/sanitize.class.php on line 152

Edited by jasehead
Link to comment
Share on other sites

I tried deleting my copyright script and testing again with the Foundation skin.  Wow - what a mess.  The page did render, but my logo on the left was by itself (looked like its own column), then the rest of the page squeezed into the remaining space. AND THEN at the bottom of the footer the whole page was rendered AGAIN in proper dimensions - except my logo was missing because that was at the top of the page.  The search box was the right size.

Chrome, Safari and Firefox all show the same result - the page rendered twice as described.

My latest break-it-for-sure method is:

  • Add a single item to the basket
  • Click the Add to Basket item again
  • Get an out-of-stock error
  • Use a backwards button (or swipe) to return to the previous page
  • Click the Add to Basket item again for the same item
  • Repeat until it breaks

This problem also occurs without any back button at all, only clicks on store buttons or logos, but is more intermittent.  Maybe a result of locally cached pages, or pages in memory?

Edited by jasehead
Link to comment
Share on other sites

First, in admin, Store Settings, Stock tab, there is "Reduce Stock Levels" when the order goes to Pending. Keep in mind, the order must go to Pending. Stock levels are not reduced (as best I know) by merely placing the item in the shopping basket. (And this is where my comparison of CubeCart versus a b-n-m store fails.)

You can also "Hide Out-of-Stock" items and have "Allow Out-of-Stock Purchases" disabled.

CubeCart has become sensitive to CSRF security token. Not being in certain POSTs (AJAX requests and a couple others) wasn't an issue a few versions ago, but it now is. But, that would be CubeCart objecting and this objection would happen no matter what browser was being used.

Your browser's developer's tools must have a Network panel, where a time-lapse graph (a waterfall) of traffic being sent and content being received is shown.

There will be a POST entry. Examine the POST payload. Note if the security token is present. (It is strange, but disregard the token if it appears as part of the URL.)

The "page within a page" (actually, an entire page replacing just the Shopping Basket area - this is the AJAX Request and Response) is caused by CubeCart disagreeing with something that got POSTed.

Interestingly, an item that has a stock level of '1' was added to the basket. Then I clicked Add to Basket a second time. (The stock levels as reported in admin and on the Specs tab of the product storefront page are still at '1'.)

I get the red banner that says:

The following errors were detected:

Sorry but the chosen product option combination is out of stock or the last of the stock is already in your basket.

 

Link to comment
Share on other sites

Stock levels are not reduced when an item is added to the basket - that's right, and fine.  To everyone else, no stock has been purchased so no change has been made.  If someone else buys the item first then they win.

But imagine this...

You're in a very small but popular supermarket and it's late in the day so the shelves haven't been restocked.  You see the last lemon - you only need one for a recipe - so you put the last lemon in your basket.  It hasn't left the shop.  You could still decide not to buy the lemon and it would go back to where it was.  But for now YOU have the lemon in your basket.  Then you look again and there's still a lemon to buy that isn't in your basket.  You try and pick it up but the store attendant slaps your hand and tells you that you can't have that lemon because it's out of stock.  "But I can see the lemon on display," you say.  "But the lemon is already in your basket," the shop attendant replies.  "Then why is it still on display?" you ask, and try to pick it up again.  The store attendant replies by behaving in a bizarre manner by (a) showing you a distorted version of the store within the store, or (b) emptying your basket, saying nothing, and expecting you to start again.

What a bizarre supermarket.

So you leave without your lemon.

While the customer is shopping, if they add an item to their basket where there is only one in stock then the Add to Basket button for that item should temporarily change and be disabled (just for that customer) while that item is in their basket.

The trick would be to:

  • check if an item in the store matches an item in the basket, and
  • if the quantity in the basket is equal to the quantity originally available (considering product options), then
  • change the "Add to Basket" button to (out of stock/unavailable/already in basket) and disable the button

I see that product options complicates things, but the simple version would be that if ALL available product options for an item were exhausted by what was in the basket then the Add to Basket button for that item should change to disabled during that customer's session.

Not to say that avoiding the Add to Basket click will solve the problem - but I can't think of another button or element in Cubecart that would cause a similar state change.

BTW - I haven't noticed an error that says "the last of the stock is already in your basket", although I found it OK in my definitions file.

---

The waterfall.  With the "2018" version I only see a GET from Google Analytics and then nothing.  With the copyright javascript removed for the "page within a page" version, there is one POST statement.

Full URL has the store address followed by the query string ?_g=ajaxadd

REQUEST DATA / FORM DATA
Name: Value
quantity: 1
add: 3022
token: d560a6106068910c62413e50e96e4681

That all looks fine.  I don't really know what information you want to find.

---

I've tested another site running 6.1.14 and was able to reproduce the "page within a page" effect - so it's nothing to do with my personal mods at least.

Edited by jasehead
Link to comment
Share on other sites

"change the "Add to Basket" button to "You have the last one!" and disable the button"

I like that idea.

I see the POST payload includes the token. The next question is, is it the correct token?

To determine that, we would need to get serious about adding a bunch of diagnostic code to Cubecart.

Link to comment
Share on other sites

It's the same token that is used throughout the source code:

<input type="hidden" name="token" value="d560a6106068910c62413e50e96e4681">

...although probably not the token that cubecart is expecting, which is why it stumbles.

Making the Add to Basket button disabled if the basket contains the last item in stock may solve the problem - no click, no break.

Edited by jasehead
Link to comment
Share on other sites

Yes, of course.

Let's try logging what CubeCart expects the token to be and what the web form is sending back as the token.

In /classes/session.class.php, near line 255, find:

public function checkToken($token) {

On a new blank line after that, add:

trigger_error('CubeCart Token: ' . $this->get($this->_token_name), E_USER_WARNING);
trigger_error('POSTed Token: ' . $token, E_USER_WARNING);

Then, observe the error_log contents. Just above every "Invalid Security Token" line, there should be the values of tokens that were compared.

Edited by bsmither
Link to comment
Share on other sites

[18-Apr-2018 17:48:35 UTC] PHP Warning:  CubeCart Token: 4e525f3864a9f532c41e64c7cc0eb247 in /home2/myserver/public_html/cubecart/classes/session.class.php on line 256
[18-Apr-2018 17:48:35 UTC] PHP Warning:  POSTed Token:   4e525f3864a9f532c41e64c7cc0eb247 in /home2/myserver/public_html/cubecart/classes/session.class.php on line 257

[18-Apr-2018 17:48:38 UTC] PHP Warning:  CubeCart Token: 0f49326b2d1086695ad0b54f4fd0b11a in /home2/myserver/public_html/cubecart/classes/session.class.php on line 256
[18-Apr-2018 17:48:38 UTC] PHP Warning:  POSTed Token:   0f49326b2d1086695ad0b54f4fd0b11a in /home2/myserver/public_html/cubecart/classes/session.class.php on line 257

[18-Apr-2018 17:48:40 UTC] PHP Warning:  CubeCart Token: 3622c2819ff498ed5c72dc816d1644d4 in /home2/myserver/public_html/cubecart/classes/session.class.php on line 256
[18-Apr-2018 17:48:40 UTC] PHP Warning:  POSTed Token:   d560a6106068910c62413e50e96e4681 in /home2/myserver/public_html/cubecart/classes/session.class.php on line 257
[18-Apr-2018 17:48:40 UTC] PHP Warning:  Invalid Security Token in /home2/myserver/public_html/cubecart/classes/sanitize.class.php on line 152

Tidied for readability.  Vigorous use of the back button and Add to Basket click - took a few tries to get it to fall over but it's clear there's a token mismatch.  It's odd that cubecart doesn't render the page correctly when this occurs.

(updated to remove server/folder info)

Edited by jasehead
Link to comment
Share on other sites

Interesting.

The next question is: Where does the HTML (page or Shopping basket snippet) that has the wrong token come from? CubeCart or a cache somewhere?

For that, we need to look again at the waterfall. There should be a setting to "persist" network traffic. (Otherwise, the waterfall resets for every request made. Whether this includes XHR - aka AJAX - requests, I'm not sure.)

If you can persist the logs, then do it again. This time, if a POST fails, look at the POST /?_g=ajaxadd. The response of the failed POST will not be one of:

* a redirect to the product page because required options were not chosen
* a redirect to the current page because something went wrong with adding to the basket (stock levels, other availability issues)
* shopping basket snippet of HTML and a new token (the requester should update all other locations in the page that hold the token)

The current problem is that the response contains index.php because the token mismatch discards all GET and POST data, and sends back the Homepage. But with an ajaxadd, that is not supposed to happen.

Look at the previous POST /?_g=ajaxadd payloads and responses.

We need to find the condition where and when the rogue token appears.

Link to comment
Share on other sites

The Params of ?_g=ajaxadd had the token: 811754e0351d1b375363f980ce111513

The response payload can be the mini basket snippet of HTML (box.basket.php), but is more often the homepage with a "Security Alert: Possible Cross-Site Request Forgery (CSRF)" error.  The response payload (correct html) is different to the generated page source of the "page within a page" (in Firefox Quantum, select all, right click, View Selected Source) - so the response payload is not showing what is rendered on the screen.  I also see an error in developer tools that foundation.css could not be loaded.

The response of the failed POST session_token was: 8b956a4057839992990c75e4a9f4166f (also the same token in the generated page source)

IF I can make ?_g=ajaxadd appear in the waterfall WITH a token, THEN the main page appears broken and the tokens for ajaxadd and the response payload (or generated page source) are different - tested in all 3 browsers.

---

I'm testing a stock item with only one copy available and no product options.

In Firefox, if cubecart doesn't break it takes me to the product page with the error message "Sorry but that product is out of stock."  If it does break then I see the homepage as "page within a page", and the mini basket appears briefly (like a normal Add to Basket event).

In Safari, I have watched it stall on the "page within a page" render, or continue on to be replaced by "Your Shopping Basket" but with no error notices.

In Chrome it's harder to tell because it often scrolls to the bottom of the "page within a page" which looks rendered OK - have to scroll to top to see the drama.

Edited by jasehead
Link to comment
Share on other sites

Would this help with my security token issue?

#1921 - Calms down CSRF token generation
https://github.com/cubecart/v6/commit/79bf08ac1ec0cd9ca536d04c9b0f4d9964f3af63

In classes/sanitize.class.php around line 72:

 				$GLOBALS['session']->getToken(true);
 				self::_stopToken();
 			}
			//Make a new token // DELETE THIS LINE
			$GLOBALS['session']->getToken(true); // DELETE THIS LINE
 		}
 	}

 

So I tried the edit and it seems to be a fix for my problem - I can't get the store to fall over now.  It DID still manage to generate a couple of errors in the error log (one time the first token was different, another time it was missing) but this time the errors didn't spoil the customer experience.  I got one notice from Safari saying the page was reloaded because of a problem, but that was acceptable.  And in Chrome I had a suddenly empty basket, but that sorted itself by returning to the store.

Edited by jasehead
Link to comment
Share on other sites

It might. But this doesn't answer the question -- why (so far) is this affecting only you?

This, posted on the 15th:

<noscript>2015</noscript><script type="text/javascript">document.write(new Date().getFullYear());</script>

You say this is being received from your site. This is the ONLY thing (sometimes) being received from your site.

To solve this will require serious troubleshooting.

Link to comment
Share on other sites

It's probably only affecting me because I'm hammering 6.1.14 trying to make it break before going live with an update, and so I found a way to break it.

In my store, the document.write for the copyright year was responsible for the "2018" on a blank page when the security tokens didn't match and sanitise.class.php got upset.  When I removed my javascript from the copyright, the security token mismatch returned a "page within a page" render of the home/main page - the page rendered twice.  I tested other 6.1.14 Foundation stores (that were not mine) and they broke the same way.

I've tested the 6.1.15 fix for "#1921 - Calms down CSRF token generation" and it seems to be the desired outcome.  The security token issue happens less often and when it does, that doesn't cause the store to appear broken to the customer - the security token glitch happens less often and quieter, so the customer can continue shopping without noticing.

The fix was for "CSRF token regeneration on each page load damages usability #1921", back button etc. so I don't think the security token issue was just affecting me.
https://github.com/cubecart/v6/issues/1912
https://github.com/cubecart/v6/issues/1900

Edited by jasehead
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...