Invalid file contents when downloading

[leetdev]

When downloading files with Blossom.fdpStorage.file.downloadData(), some binary files return invalid data. Specifically, the contents returned are longer than the original.

After doing some debugging, I suspect that this is due to the serialization process introduced in #114 - specifically, TextEncoder seems to escape certain characters, which then get decoded as separate bytes by TextDecoder. I don't think those interfaces are meant for encoding binary data as strings. You should consider using Base64 or Hex encoding instead.

To test this theory, I added the following console.log statement into src/services/fdp-storage/fdp-storage-access.ts:

The resulting output in the console when downloading a PDF file is as follows:

As you can see, the initial response from fdpStorage is 25573 bytes long (the actual size of the file). However, after serialization and deserialization, this turns into a 42344 byte array instead. Needless to say, this data is now unusable in the app using Blossom to download the file.

[tomicvladan]

This bug has been fixed, but only partly. The reason for that is architectural limitation. Maximum size of data that can be transferred between service worker script and web pages is about 50MB. So trying to upload or download any larger file will cause error. The plan to overcome this limitation is to update fdp-storage to support uploading/downloading of files in chunks. Until then nothing else can be done.

[crtahlin]

@tomicvladan can you open an issue in fdp-storage repo please and link to it here.

[tomicvladan]

@crtahlin It's already created: fairDataSociety/fdp-storage#227