Save local key-value store records with the correct extension (#91)

There was a bug that files in local key-value store were saved with the
wrong extension, e.g. `Actor.set_value('image.png', data)` would save
the record as `image.png.bin`.

This fixes it by making the filename generation more sensible:
- the `.bin` extension won't be added anymore if the content type is
`application/octet-stream`.
- the record metadata is no longer saved as
`{record_key}.__metadata__.json`, but as
`{record_filename}.__metadata__.json`
- this makes it a lot easier to find the right metadata for a given file
when opening the store
- the metadata no longer contains the `extension` attribute, just the
key
- the internal record representation now contains the `filename`
attribute pointing to the filename under which the record is saved.

I think the logic is a bit more readable this way, and it works better.

I also wrote a lot of tests to check if it works correctly, they're a
bit hacky, but good enough.

Fixes #85.

Author: fnesveda

CHANGELOG.md

@@ -11,6 +11,7 @@ Changelog
 ### Fixed
 
 - started enforcing local storage to always use the UTF-8 encoding
+- fixed saving key-value store values to local storage with the right extension for a given content type
 
 [1.0.0](../../releases/tag/v1.0.0) - 2022-03-13
 -----------------------------------------------

src/apify/_memory_storage/file_storage_utils.py

@@ -41,45 +41,6 @@ async def _update_dataset_items(
             await f.write(_json_dumps(item).encode('utf-8'))
 
 
-async def _set_or_delete_key_value_store_record(
-    *,
-    entity_directory: str,
-    persist_storage: bool,
-    record: Dict,
-    should_set: bool,
-    write_metadata: bool,
-) -> None:
-    # Skip writing files to the disk if the client has the option set to false
-    if not persist_storage:
-        return
-
-    # Ensure the directory for the entity exists
-    await makedirs(entity_directory, exist_ok=True)
-
-    # Create files for the record
-    record_path = os.path.join(entity_directory, f"""{record['key']}.{record['extension']}""")
-    record_metadata_path = os.path.join(entity_directory, f"""{record['key']}.__metadata__.json""")
-
-    await _force_remove(record_path)
-    await _force_remove(record_metadata_path)
-
-    if should_set:
-        if write_metadata:
-            async with aiofiles.open(record_metadata_path, mode='wb') as f:
-                await f.write(_json_dumps({
-                    'key': record['key'],
-                    'contentType': record.get('content_type') or 'unknown/no content type',
-                    'extension': record['extension'],
-                }).encode('utf-8'))
-
-        # Convert to bytes if string
-        if isinstance(record['value'], str):
-            record['value'] = record['value'].encode('utf-8')
-
-        async with aiofiles.open(record_path, mode='wb') as f:
-            await f.write(record['value'])
-
-
 async def _update_request_queue_item(
     *,
     request_id: str,