Missing random order confirmation (merchant) emails

[skepticwebguy]

I've noticed, just lately, that I am missing random merchant order confirmation emails. There doesn't seem to be a pattern to it. Just some of them don't ever arrive. My mail queue is empty, so it's not as though they are stuck. Plus, I keep getting order confirmations. So, it's working. It's just not working on every order.

When I try to resend the merchant order confirmation email for orders that didn't arrive, the confirmation email still never arrives! I've also checked my spam folder and the emails aren't there.

Any ideas what might be causing this odd behaviour? Why am I getting some, but not all, order confirmations, and why can't I resend the ones I'm not receiving?

[ILoveHostasaurus]

Re: Missing random order confirmation (merchant) emails

Do you have the order notifications set to come 'from' the customer's address, and use an email service separate from the server where the website runs?

[skepticwebguy]

Re: Missing random order confirmation (merchant) emails

Yes. I have it set to come "from customer's email address" and the TO: address is an email address on Google's mail servers. My Mail Settings (under Domain Settings in Miva Admin) are set to use localhost on port 25.

When I click to resend the email merchant notification email, I can run "q" in a terminal window and see the mail in the queue for a second. If I catch the same email in my server's Mail Queue in Plesk Admin, I can see a "failure notice." Here are two examples:

Message 14/140691267 not found in the queue!
Message 19/140691272 not found in the queue!

I've tested about 5 orders that never arrived and I can reproduce a "failure notice" on all of them. When I resend orders for which I did receive a confirmation email, the confirmation email comes through just fine again.

[ILoveHostasaurus]

Re: Missing random order confirmation (merchant) emails

I recommend against having messages come from the customer's address primarily because it means the customer is going to get the bounce message when it's rejected or a delivery failure occurs for any reason, and that can be confusing or concerning from a security standpoint.

In any case, the most likely reason these particular messages are failing is because they are probably customers whose domains publish an explicit SPF (Sender Policy Framework) record in DNS to advise the world what mail servers are permitted to send email on behalf of their domain. It's an anti-forgery technique. What is occurring is customer with domainA.com publishes an SPF record that says mail claiming to be from domainA.com can ONLY come from mail.domainA.com. When they order from your site, the server gets the message from your store claiming to be from [email protected] and attempts to deliver it to Google. Google says woah, domainA.com publishes an SPF record that permits only mail.domainA.com to deliver email for that domain, so this message being attempted by randomwebsite.com must be a forgery and they then reject it.

There are two ways around this issue. One, configure your store to use Google's mail server as your store's outbound server. The downside to that is if someone checks out and Google's mail server is unreachable, non-functional, dns lookup error, etc. at the time, the checkout will likely time out and present an error to the customer which might cause them to order again, get confused, go elsewhere and later find out the order worked, etc.

Two, enable mail for your domain on the web server so the store can deliver those messages locally. Most servers won't do an SPF check for a local delivery, so the message that has the appearance of being forged will still be successfully delivered to the local mailbox. Then, have Google poll that mailbox for messages; that will get the message to them in a way that bypasses the SPF check. The downside to this option is you need to ensure you create email boxes on the local server for every possible address your website may generate emails too, and then poll them all from Google, or mail could go missing.

[skepticwebguy]

Re: Missing random order confirmation (merchant) emails

I hadn't thought of the SPF issue. That's a great explanation of what's likely happening for some customer addresses. Hence the disappearing order confirmation emails.

FYI, 'template-based' customer order confirmation emails are working fine. I did a test order to my gmail address, and the customer order confirmation email came through just fine to me (as the customer), and the email appears to be from our "orders" email address, as configured in our Domain Settings > Registration tab. That all works fine from the customer's perspective.

However, it's the merchant perspective where things are going wrong with some emails addresses. I currently don't use the template-based merchant order confirmation emails. Instead, I use Sebenza's "Additional Checkout Data Pro" module to collect extra info during checkout and then I use ACDPRO's "Email Merchant Notification" module to send me order confirmation emails (that appear to come from the customer) containing the data that customers entered into my "Additional Checkout Data" fields. I have this module set to send me the emails "from the customer's email address." I tried setting this to send me the emails from our "orders" email address, but that causes a gateway timeout sometimes, producing an error in the browser, even though the order goes through.

Now that I've narrowed this down to the Sebenza ACDPRO module, I'll write to them (unless someone from Sebenza chimes in here), before I go the route you suggested in your last paragraph, David.

Thank you for your help. Much appreciated!

[Rick Wilson]

Re: Missing random order confirmation (merchant) emails

You can use the new Custom Order fields to eliminate the need for things like Sebenza ACDPRO or Emporium's Addendum.

[skepticwebguy]

Re: Missing random order confirmation (merchant) emails

You can use the new Custom Order fields to eliminate the need for things like Sebenza ACDPRO or Emporium's Addendum.

Awesome. I would prefer to use fewer modules.

I've assigned the "Custom Fields" item to my OCST page and created a custom field of "type: order" in Utilities > Custom Fields. However, I must be missing some template code on OCST, because the field I created isn't showing up on the OCST page during checkout.

[Brennan]

Re: Missing random order confirmation (merchant) emails

Check out the custom field docs here:

http://www.mivamerchant.com/videos/a...-documentation

Here are the steps:

1. Create a custom order field in the admin.
2. Add a Write_Basket function call to checkout on the page after you collect the information to save the information to a custom basket field
3. The custom basket field should have the same code you used when you created the custom order field.
4. The custom basket field will automatically get converted to a custom Order field on INVC (as long as the codes match)

[skepticwebguy]

Re: Missing random order confirmation (merchant) emails

I'm still unclear as to how to get the new custom field to display on the OCST page to begin with. I want to set up a custom field on the OCST page that allows customer to enter comments regarding their order.

[Brennan]

Re: Missing random order confirmation (merchant) emails

There isn't an item/or variable to output the display code to the page. It needs to be setup manually.

On OCST create an input like this:

<label for="notes">Additional Order Information</label><input type="text" name="notes" value="" />

Then on OSEL you could save the value into a custom basket field

<mvt:item name="customfields" param="Write_Basket('order-notes', g.notes)" />

order-notes is the code you gave to your custom order field and g.notes is the global variable containing the value of the input from OCST.

If you already have addendum of ACD Pro you can copy the HTML output from their module on OCST and use that to control the font end display code for the fields you want to collect.

[skepticwebguy]

Re: Missing random order confirmation (merchant) emails

Okay. We're getting somewhere. I placed a text field on OCST and mvt:item with the Write_Basket parameter on my OSEL page in the "Basket Contents" tab (though, presumably, it could go anywhere on the OSEL Page template tab instead, right?). I did a test order and can confirm the the data is appearing on the Orders Admin screen for the test order, on the "Custom Field" tab.

Now, how do I get the contents of this custom field to show up on the merchant order confirmation (template based) emails? Will the same mvt:item with the Write_Basket parameter from the OSEL page work to output the data on the merchant notification email?

[Brennan]

Re: Missing random order confirmation (merchant) emails

The Read_Basket will work in the confirmation emails bu there is currently an issue where the emails are send before the data is available.

See this thread for the solution:

http://extranet.mivamerchant.com/for...802#post412802

[skepticwebguy]

Re: Missing random order confirmation (merchant) emails

…the most likely reason these particular messages are failing is because they are probably customers whose domains publish an explicit SPF (Sender Policy Framework) record in DNS to advise the world what mail servers are permitted to send email on behalf of their domain. It's an anti-forgery technique. What is occurring is customer with domainA.com publishes an SPF record that says mail claiming to be from domainA.com can ONLY come from mail.domainA.com. When they order from your site, the server gets the message from your store claiming to be from [email protected] and attempts to deliver it to Google. Google says woah, domainA.com publishes an SPF record that permits only mail.domainA.com to deliver email for that domain, so this message being attempted by randomwebsite.com must be a forgery and they then reject it.

This ended up being the problem. I confirmed it looking at my server's mail logs.

[skepticwebguy]

Re: Missing random order confirmation (merchant) emails

What am I missing here?

1) I created a custom "order field" type called "newsletter" of type text field
2) I checked "customfields" on the Items tab for OCST, OSEL, and my "Order Confirmation: Merchant" email template.
3) I added this code to the OCST Page:

<input type="text" name="newsletter" size="50" />

4) I added this code to my OSEL page template at the bottom of the code in "Basket Content" tab:

<mvt:item name="customfields" param="Write_Basket('newsletter', g.newsletter)" />

5) I added this code to the bottom of my "Order Confirmation: Merchant" email template:

<mvt:item name="customfields" param="Read_Basket( 'newsletter', g.newsletter)" />
<mvt:item name="customfields" param="Read_Order( 'newsletter', g.newsletter)" />

When I place a test order and view the order through Manage Orders, the custom field for "newsletter" shows the email address I entered on the test order. However, no custom field data shows up in my merchant order confirmation email.

[Brennan]

Re: Missing random order confirmation (merchant) emails

All your code looks correct.

The Read_Basket and Read_Order functions read in the custom field who code is newsletter and saves that value to the second parameter which you have defined as g.newsletter.

So you may just be missing the output of the variable to the page:

&mvte:global:newsletter;