New Order email floods

[Moshe Nitzani]

Hello

 

This issue has been going on for a long while, and CC support say that they have never encountered such an issue.  From time to time we receive hundreds or thousands of New Order emails that have no data in most of the fields apart from the country.   

 

The country was always Serbia, and when I temporarily removed Serbia from the list of countries, it changed to Montenegro.  After deleting Montenegro, the country field is also empty.

 

• None of the CC logs show anything suspicious.
• The link in the email is pointing to a non existing product.
• There could be gaps of hours or days between each batch of email floods
• The email header shows that it does come from my server.
• Had a look at some of the server logs: auth.log and access.log  all looks ok to me.
• I have moved the CC installation from one server to another, issue still exists.

Below is one of the emails.  Any help where to look, and what to do is appreciated.

 

Moshe

 

 

Just placed order number on .

This order can be managed online by following the link below.

http://payments.lfs.org.uk/admin.php?_g=orders&action=edit&order_id=Part Payment to the LFS

Billing address:

 

 

 

 

 

 

 

 

Email:

Shipping address:

 

 

 

 

 

Item

Quantity

Cost

 

Shipping:

 

 

Discount:

 

 

Subtotal:

 

 

Order Total:

 

Kind regards,

The The London Film School Staff
http://payments.lfs.org.uk

 

[bsmither]

The order_id part of the URL is not a valid cart_order_id number. So, CubeCart wouldn't be able to pull anything out of the database to populate the Admin: Order Received email.

 

Is the word "Just" actually capitalized? The email template isn't.

 

Sending a thousand emails would take quite a bit of time. From the first of the batch to the last, what is the timestamp difference (the bottom most Received: line in the headers)?

 

Are the header Message-IDs the same or different?

 

Is your store settings using Email Sending Method: PHP mail() Function or SMTP?

 

And what is the exact version of CubeCart?

 

In your Store Settings, Features tab, Order status for admin email notification is set to what?

[Moshe Nitzani]

Hi bsmither

 

Thanks for the quick reply

 

I capitalized the word "Just" by mistake as I pasted the email without the "j" and added it manually

1325 emails were received today between 8:40am and 9:43am

Message IDs are different in each email I checked

I am using PHP mail()

I am using CC 5.2.2

 

regards

 

Moshe

[bsmither]

Comment: You have a lenient hosting provider as most would kick you off for sending that many emails in an hour. (Unless it's your own server.)

 

So, today, at 08:40, do you recall administering any orders? You can check by looking at the Admin Activity log (admin.php?_g=settings&node=logs#logs_activity) for about that time. (The date in this log may be UTC, so be sure to add/subtract the appropriate number of timezones. Also, using the first email of the batch, the email headers show UTC if the offset is zero, such as: Thu, 09 May 2013 13:40:21 +0000.)

 

I'm still looking.

 

Has this happened with any version of CubeCart prior to CC522?

 

In your Store Settings, Features tab, Order status for admin email notification is set to what?

[Moshe Nitzani]

The server is a VPS solution

 

No administering of orders at any time during the period the emails came flooding in.

 

This issue began after upgrading from 4.x to 5.x 

 

Order status for admin email notification is set to: Processing

 

Thanks

[bsmither]

VPS: meaning you can do almost anything you want. That's good.

 

Nothing suspicious in the Admin Activity log...

 

1325 in 63 minutes = 21 per minute = averaging 1 every 3 seconds.

 

PHP has a limit of a 30 second run time, so there isn't going to be an infinite loop happening.

 

Thinking way outside the box, it seems almost as if something was resting on your keyboard and holding down the Return key -- telling your browser to fetch the same page over and over and taking 3 seconds for the server to do it.

 

But your server's logs -- the web server (Apache?) access log doesn't show anything?

 

Was an actual order placed at 8:40 am? If so, what gateway was used?

[Moshe Nitzani]

Hi

 

There was a real order at 8:33am yesterday, 7 minutes before the flood of emails began.   Below is a link to the access.log showing all for the 9th of May. I can see loads of search sites accessing my server.  As of this morning I changed the robots.txt to stop indexing the site as we don't really need that. The CC is used by students of ours to pay fees, and we email instructions plus links on another site.

 

I am not convinced that this is some kind of an attack, not yet anyway.. 

 

http://goo.gl/kif31

 

Thanks

 

Moshe

[bsmither]

"There was a real order at 8:33am yesterday, 7 minutes before the flood of emails began."

 

And I presume you received the real Admin: Order Received email for that order?

 

So, this site, when an order is made, is considered a digital order? The order goes from Pending to Processing, then automatically to Completed within minutes?

 

I've looked at the file you've linked to and cannot find any POST lines - events that surely as one moves through the store and payment process, I would see log entries for. Nor am I seeing any GET lines for your store's javascript, css, and image files. I am somewhat confused and am not all that convinced this file is from your domain that runs your store.

 

I would expect to see this that would add an item to the shopping basket:

"POST /index.php?_a=product&product_id=2&_g=ajaxadd HTTP/1.1" 200 584

 

 

What is the time difference between where you are and UTC?

[Moshe Nitzani]

I can't find an admin email confirming the order done at 7:33.  CC is showing the order as Pending, and there is no record in Transaction Log of that purchase.  So I assume that something went wrong for that guy, and he abandoned paying.

 

The products are considered as Digital products, so the orders change their status from Pending to Processing when our payment gateway, Worldpay sends the callback to the cart.  I was told that by CC to manually change the status from Processing to Order Complete.  In version 4.x there was an issue I can't recall with making the products Digital, just the way it was presenting to the student, not sure what was wrong, but that made me decide to make all items as if they were physical ones.

 

Yes, I looked at this log : other_vhost_access.log  and that is the one I should have looked into.  I downloaded and extracted the lines for the 9th of May from midnight to 10:15 am for the domain CC is installed on.

 

http://goo.gl/6zaDk

 

 

The difference between London and UTC is +1

 

Kind Regards

 

Moshe

[Moshe Nitzani]

BTW: 81.178.173.97 in the logs is my IP at home, when I was looking at the CC site trying to figure out what to do...

 

Moshe

[bsmither]

The person who made an order at 8:33am did try to pay for it, but WorldPay sent back a response to your CubeCart and the web server (not CubeCart) really did not like it: 500 Internal Server Error

 

Have you received the email from WorldPay about the failed callback for the order placed at 8:33am on the 9th? If so, there is supposedly two attachments: what CubeCart sent and what WorldPay sent back.

 

I would like to see (in a Private Message) what WorldPay sent back.

[Moshe Nitzani]

Hi bsmither

 

I found out the reason for the email floods (99.99% sure anyway).  This has happened because we had a webpage on our site with a form allowing users to pay us fees, or part fees.  The payment gateway, Worldpay would do a callback to CubeCart, with a product that doesn't exists in CubeCart.  This caused CubeCart to go ballistic and start emailing blank emails in their hundreds and thousands.

 

I have disabled the page, but it might be interesting for CubeCart developers to know that this  can happen.

 

Thanks for your help

 

Kind Regards

 

Moshe

[bsmither]

Ok. Thanks.