Cloudflare timeouts

[ids]

Have a custom module that sends an API request, one per product code from the product catalog. Cloudflare times out after 100 requests. Doesn't seem like that many.

At them moment, the JSON request structure appears to only support one product code at a time. Until I get more info I have to make the MvCall per product. This is where I am thinking all the timing out is occurring. Is it possible to reset Cloudflare in between or after so many MvCalls? btw: ultimate goal is for this function to run on the product catalog in a scheduled task. There are many many more products that 100.

Thanks,

Scott

[Brennan]

You're likely hitting the Cloudflare timeout of 60 seconds which cannot be changed.

But I'm not totally following what is going on here. You have a module that is calling back into Miva to get product data?

You could also bypass Cloudflare by having the MvCALL hit the IP address, or setup a cname that goes direct to the server IP so it never gets routed through cloudflare.

[ids]

No, this particular module needs to call an external endpoint to grab updated product data. ATM inventory or pricing are examples.

MvCalling the IP address might be possible. The server is a black box for me. The spec has the URL to call. But, this server is acting like a back office machine in a warehouse running some form of enterprise software.

Pessimistically, even if I can bypass, an application timeout could still be possible with more and more products.

Scott

[ILoveHostasaurus]

I'm not aware of anything in Cloudflare that would care about the number of requests, unless the site in question (on the Cloudflare side) has a rate limit explicitly configured.

If you're talking about 100+ simultaneous requests, such as firing off async processes with mvcalls, well that I would expect to fail horribly because the store would be firing off requests back to itself at the rate it can iterate through the list, which would be effectively a DDoS against itself. I see no reason why those should not be handled synchronously, with appropriate timeout mitigation in the script so it can iterate through at the rate the store responds, and as the loop reaches the application timeout, continues on with a new loop and/or browser refresh/redirect to keep it going.

[ids]

A couple changes since...

Turns out the API has a couple of endpoints that will return a list of products instead making 100s of requests individually. One endpoint works nicely, but the other endpoints has an issue where I receive an error:

MvCALL: SSL Read Timeout

It comes after about 30 seconds, well within the overall timeouts as far as I can tell. 1st: why SSL READ Timeout? The timeout part is suppose to be an obvious conclusion?
Is there a data size limit by chance?

Scott

[ILoveHostasaurus]

That error typically occurs when the connection has been closed by the target abruptly. CF should allow the connection to run to 60 seconds, so it occurring at 30 is definitely weird. If the target is with us, we could check the web logs to see if the connection had errored earlier, as well as in CF to see if they're closing it for some reason. A site could potentially have its mvcall timeout set to 30 as well, which I think may produce a similar error. Data size I don't think should come into play unless we're talking gigabytes.

[ids]

Thanks David.

The current store is a mivatest dev store. Could the timeout be set to 30 by default? I'll ask the client to check or finally give me FTP and/or MyMiva creds.

If I understand "target," it's a 3rd party API and server that the URL is pointing to. It is a MS Windows/ASP.net server.

Scott

[ILoveHostasaurus]

Got it, in that case I don't think CF would be involved, as even though the site itself is behind CF, the mvcall would be going directly from the store on our network to the target ASP server, not through CF unless that target site also happens to use them. The fact that you're seeing the mvcall failure message visibly means the connection from you, through CF, to the site, was still active and remained active until the store page request completed and sent you the content with failure message present.

I found out the default mvcall timeout is 30 seconds, so I think you're just running into an issue where the target is taking longer than that to output content. You can increase the call timeout up to what you'd want to keep less than the application timeout, so my recommendation would be to check the application timeout (stored in s.miva_config:timeout_call) and then set a call timeout as part of mvcall using a variable that's s.miva_config:timeout_call minus a few seconds.

That being said, if the target is going to take longer than 60 seconds to respond, this is all going to fail because you'll hit Cloudflare's timeout. If that's occurring, you will need to implement it as an async task that can run behind the scenes with a long timeout. That would not be the ideal solution though; better that the script have error handling and work in chunks across timeouts than just hoping a super long connection always works.

[ids]

Seems the CF timeout of 60s will be in play. The benchmark from the 3rd party is 100 seconds in Postman. That is longer than the 90 second application timeout in this store obviously.
Would calling the IP address directly still bypass CF like you suggested earlier in this thread? Then for this store, also likely need to increase the application timeout to 120?

Scott

[ILoveHostasaurus]

The CF timeout only affects the request from your browser to the site; they are not playing a role between the mvcall and what it is hitting. But, if such a request triggers an mvcall that needs to run for 100 seconds, CF will have terminated YOUR connection to the site before that backend mvcall completes, and that will terminate the script that had made the mvcall, so it never completes what it was trying to do. You'll hit the 90 second default application timeout next, so that would also have to be increased. The mvcall timeout would have to be increased.

Alternatively, if you don't need to see the result of your request in real time, the mvcall and resulting work can be handled via async task which won't have the same timeout imposed. You can set the mvcall timeout to a long number, and let it run.

If you do need this all to work in real time with the human accessing the site seeing the results, then the site will not be compatible with cloudflare, our default application timeout, or our default mvcall timeout. CF would need to be removed, and both timeouts increased. This is not a reliable direction to head, so I'd advise finding alternatives to that 100 second request, whether that means multiple requests for less data or scheduled tasks, etc. Building around very long running http requests is just asking for trouble these days, and also prevents the use of very valuable web application firewalls like CF; the site may find itself in a situation of being attacked, and the only option is CF, so you're left with the choice of leave the site down from attack, or break the site by stopping the attack with CF.

[ids]

Thanks for explaining the distinctions. We actually have no desire to bypass CF? Async looks like it'll be the solution for this endpoint.

However, Can you clear up the previous suggestion (from Brennan) of the possibility of calling the IP address? Might be an option that needs to be explored.

You could also bypass Cloudflare by having the MvCALL hit the IP address, or setup a cname that goes direct to the server IP so it never gets routed through cloudflare.

Wouldn't the CF firewall need the exception even for the IP Address?

Scott

[ILoveHostasaurus]

What Brennan would have been referring to is when a request hitting a Miva Merchant site needs to run longer than 60 seconds, that is impossible with Cloudflare in the mix, so one would have to not use them, or go around them. That won't make your back end mvcall take less than the 100 seconds that provider requires, so you're already going to be facing an uphill battle with timeouts from CF, our web server default timeout, the Miva engine default application timeout, the default mvcall timeout. There have been times in the past where we'd raise some or all of those, but that is a very bad idea, and we won't guarantee we'll support it long term, so any fresh development should not be based around super long running web requests working reliably. If there's simply no way to break up what is being done, then the async task is the way to go, with the front end browser polling for when it completes. Then from the front end perspective, it's just making a quick request every ten seconds or however many you want, until the async tasks completes and the data shows up.

[ids]

Thanks again, David.