Description
Currently, the program is using two separate file formats -- one for an encrypted database and another for encrypted bins. See additional context for a format description and comparison between the two. The only real difference between them is the bin_id, that is used as a human-readable identifier for the bin. This, too, can be changed by re-using the AES initial vectors and encoding them in base64 instead, or converting each character by limiting them to ASCII printable characters. So, instead of maintaining and interfacing with two different file formats, I can consolidate them into one and interface with it like that.
The new design would still need a little bit of rework, however, to ensure future updates are easier. Firstly, to differentiate between the different usages of the file (which I will be naming their 'formats') will need its own field, and so will an actual version, although it will most likely go unused. The global header can have a bitfield for flags, and so can each file. The global header bitfield can indicate if the archive is encrypted or not, which I plan on using for distributing a plaintext database for a better-controlled database delegation instead of a 'whoever has it' delegation, but that is another issue. The bitfield for files indicate if a file is compressed or not, as I would rather avoid wasting my huffmann implementation. Instead of compressing a high-entropy encrypted file, I can compress a low-entropy data, resulting in actual data reductions.
I don't have to worry about data corruption due to partial writes, as my pattern of creating a temporary copy, modifying the copy, moving the copy into the original directory under a different name, then doing an atomic rewrite to replace the actual save file. Apart from bad code corrupting the data, this pattern will make the save files protected from partial writes.
The iostream interface can still be used for interfacing with the file, but it would need some rework to support the new format, to make it more robust.
Additional Context
Bin format
[40-byte Global Header]
[8-byte VERSION]: "ARCHV-64"
[16-byte BIN_ID]: Like "abcd1234wxyz6789"
[16-byte AES_IV]
[8-byte Magic Block]
[8-byte MAGIC]: "UNLOCKED"
[24-byte File Header]
[8-byte MAGIC]: "ARCHVFLE"
[8-byte PATH_LEN]
[8-byte DATA_LEN]
[File Data]
[... FILE_PATH_DATA]
[... FILE_DATA]
[Footer]
[8-byte END]: "ARCHVEND"
DB format
[24-byte Global Header]
[8-byte VERSION]: "EDBASE64"
[16-byte AES_IV]
[8-byte Magic Block]
[8-byte MAGIC]: "UNLOCKED"
[24-byte Entry Header]
[8-byte MAGIC]: "DBASEFLE"
[8-byte KEY_LEN]
[8-byte VALUE_LEN]
[Entry Data]
[... KEY_DATA]
[... VALUE_DATA]
[Footer]
[8-byte END]: "DBASEEND"
Description
Currently, the program is using two separate file formats -- one for an encrypted database and another for encrypted bins. See additional context for a format description and comparison between the two. The only real difference between them is the
bin_id, that is used as a human-readable identifier for the bin. This, too, can be changed by re-using the AES initial vectors and encoding them inbase64instead, or converting each character by limiting them to ASCII printable characters. So, instead of maintaining and interfacing with two different file formats, I can consolidate them into one and interface with it like that.The new design would still need a little bit of rework, however, to ensure future updates are easier. Firstly, to differentiate between the different usages of the file (which I will be naming their 'formats') will need its own field, and so will an actual version, although it will most likely go unused. The global header can have a bitfield for flags, and so can each file. The global header bitfield can indicate if the archive is encrypted or not, which I plan on using for distributing a plaintext database for a better-controlled database delegation instead of a 'whoever has it' delegation, but that is another issue. The bitfield for files indicate if a file is compressed or not, as I would rather avoid wasting my huffmann implementation. Instead of compressing a high-entropy encrypted file, I can compress a low-entropy data, resulting in actual data reductions.
I don't have to worry about data corruption due to partial writes, as my pattern of creating a temporary copy, modifying the copy, moving the copy into the original directory under a different name, then doing an atomic rewrite to replace the actual save file. Apart from bad code corrupting the data, this pattern will make the save files protected from partial writes.
The
iostreaminterface can still be used for interfacing with the file, but it would need some rework to support the new format, to make it more robust.Additional Context
Bin format
DB format