Do import modules have timeout protection?

[Kent Multer]

Hi folks,

I'm negotiating a for project to write an import module that will import very large files. The time required to import them will be longer than the store's global timeout. Does the Miva import-module API have timeout protection? Will the module be able to run all the way to the end of the file without timing out?

Thanks --

[dcarver]

Yes, the import system will automatically recycle if the import has been going for over 60 seconds or the process is within 3 seconds of timing out.

[ids]

I've asked this question in similar ways a few times. My conclusion is there isn't a way to prevent a timeout. You can only increase the timeout in the Empressa config. It generally doesn't solve this issue.

Scott

[ids]

Yes, the import system will automatically recycle if the import has been going for over 60 seconds or the process is within 3 seconds of timing out.

Please explain. If I've loaded a large file into a structured array, I see timeouts. Is your answer new somehow?

Scott

[dcarver]

The file is uploaded, parsed, and a "session" is created. If we hit the recycle time that session structure is saved and an import session ID is returned to the JS code which re-triggers the import process with the previous session ID. The import then picks up from where it left off. This is similar to how provisioning works, even copy product uses the same idea.

[ids]

Is this session "transfer" new in version 10? Is it also restricted to being part of the import module feature set?

Scott

[dcarver]

No this has been around forever and yes it's part of the "import" module API.

[ids]

Thanks, David.

Will you give an example of the function or sequence that triggers the JS to keep the import rolling? I've done custom imports based on the product import module and XML Provide import module and run into application timeouts. I've obviously missed a step or two. It would help to know what I'm looking for the next time I look at one of these projects -- in which one is coming up.

Scott

[dcarver]

I'm not sure why the import feature is not part of the LSK, but all the code that handles that is in there. Your module just needs to implement the import feature and all related API functions and it will work. Basically ImportModule_Delimited_Import_Begin is called to setup the import and ImportModule_Delimited_Import_Record is called for each record within the imported file. After every call ImportModule_Delimited_Import_Record the import system checks to see if a recycle / refresh is necessary. If one is, the session is saved and a refresh of the connection is done. You can look at the LSK at existing import modules to see how they work. An individual module should not have to deal with any timeouts on their own.

It should still be noted that if the processing of ONE record in an import / ONE tag within a provisioning import exceeds the global timeout you will have issues and there is nothing we can do about that. That is up to the module designer.

[ids]

Thanks, David.

[Kent Multer]

Hi David, thanks for the info.

I didn't realize that feature IMP was missing from the LSK. I assume that was an oversight? Can you please ask someone to add that folder to the LSK, and post a notice here when it's available?

Thanks --