How to import/export ASCII text which contains all manner of delimiters?

[Jonathan-Driftwood]

OK, because I am a pudgy old Miva scripter who still enjoys the heck out of old, uncompiled MivaScript I get approached by clients wanting me to create/edit/fix antique MivaScript apps. One came to me recently which has overwhelmed my obsolete brain matter.

The client wants to use both crypto payments and PGP stuff [no, he is not messing with illegal stuff he is just more paranoid than a QAnon leader on a good day], and wants to only have ASCII data. Problem is that he wants to export and potentially import dB fields containing those data sets, and each of them seem to contain just about every character in the ASCII set. And as anyone who's worked with crypto stuff knows, even one character out of place means everything fails. I must be missing something very fundamental here trying to eval all the potential characters used in crypto stuff.

So the problem is how to encapsulate that potentially full run ASCII data without the delimiter potentially appearing in the data? Anyone have any experience in this regard (my old friend Ivo has faded) or might be able to offer a perspective on this one?

Thanks!!!

[Kent Multer]

I believe that's what base-64 encoding was invented for: to convert binary data into a form consisting only of "printable" characters.

[Jonathan-Driftwood]

Thank you Kent, I have enjoyed your work ever since the 'Official Miva Web Scripting Book' which still sits in my stacks somewhere. Each time I've tried to actually understand base-64 encoding vs crypto I have fallen down a rabbit hole of binary which makes my brain hurt. I will brace for the pain and try again.

[Scot - ScotsScripts.com]

The basics of base64 and crypto are pretty much this: take whatever crypto function you're using (I use blowfish, just easiest for me) and get the encrypted data into a variable, then use the base64 functions to turn it into ascii for saving/retrieval. The data from the encrypt function won't be usable in any real way, can't evaluate it or do anything else, it's just step one before you use the base64 encrypt function.

[Jonathan-Driftwood]

Problem there Scot is that it has to be done in uncompiled (i.e. v3.97) MivaScript; which seems to predate the base64 encrypt/decrypt functions. Can capture the data, store the data, but exporting it to basic ASCII is where my delimiter logic seems to fail.

[Kent Multer]

Base-64 is really encryption. There's no key involved; anyone can decode your data. If you need security, you have to encrypt the data first, and then base-64-encode it for reliability.

Base-64 can be used for any type of binary data, such as compiled programs or ZIP files, not just encrypted data. IIRC, it was invented to allow the sending of binary data by email, or exchanging it between computers running different OSes, without worrying that some mail server or other program would corrupt it by stripping out control characters, putting in extra Newline characters to break up long lines, etc.

To encode data in base-64, a program takes 3 8-bit bytes, and slices and dices them into four 6-bit segments. Then there are only 64 possible values for each segment. That's a small enough number to be represented by letters, digits, and a few punctuation characters that most mail servers and other programs will accept.

Thanks for the kind words! I was just thinking earlier today that I wrote my first module in 2002. Happy 20th Miva-versary to me! :^) . If you remember the days of interpreted Miva Script, you must have been doing this work almost as long as me, or maybe longer.

[Kent Multer]

CORRECTION: somehow a word got deleted from my last post. I could swear I wrote that first sentence as "Base-64 is NOT really encryption."

[Kent Multer]

You can write a base-64 encoding function in Miva Script, using bit-shifting and bitwise AND and OR operations (BITSL, BITAND, BITOR). I did it myself; I still have a copy of the code in a file dated from 2008. There's a note in a comment saying that it may have a bug for strings of certain lengths. But I'll post it, or send it to you, if you want to have a look.

[Jonathan-Driftwood]

I would very much welcome that Kent. I have to redfaced admit that I have never used the BITxx operators. Just never had call for them I guess, or I saw "binary" and my brain shut down. I would sincerely thank you for that old code if you're willing to dig it up!

And yes, I believe we both got into scripting about the same time. Back in the days of Joe Austin, Ivo, Darren, Luray, Scott, Pamela and all the other OGs of MivaScript. To this day I still drool over Ivo's code and I learned more than I can say through his style of coding MivaScript and somewhere on my servers I think I still may host one of Darren's web sites. Still have all my pics and web pages from the first Miva Conference in 2004(?) and the sailing trip I sponsored that year, and the slideshow video that Darren made from that trip. I remember that it broke my heart to not have Rick W and John B on that boat; I had hoped that some of John's incredible scripting mojo would rub off on me.

[Kent Multer]

Hmmm, for some reason, the forum won't let me paste the code into this text box. And it won't let me attach a file either; I'm getting weird error messages. If you PM me your email address, I'll send the file to you.