Package size exceeding lambda limits

[emileten]

I tried deploying this package as is to AWS lambda following the procedure in infrastructure/aws -- namely, deploying the package as a zip file in S3 which is then pulled by the lambda. I am getting an error due to the size of the unzipped package exceeding the limit of 262 MB -- I am at 263 MB.

One solution could be to switch to a docker-container based lambda but this is going to make the application slower. Instead, we could make the redis dependencies optional.

I am also suggesting that we pin more aggressively the package versions so as to prevent package size increase in the future -- which is what caused this issue (in the first place this package wasn't exceeding the size limit, and I guess that since then some of our dependencies increased in size and we hadn't pinned them).

What do you think of the latter?

[vincentsarago]

The current dev deployment is blocked by this.

There could be three solutions:

• remove Redis Cache
• use EFS to store the package (https://aws.amazon.com/blogs/compute/using-amazon-efs-for-aws-lambda-in-your-serverless-applications/)
• use diskcache instead of redis
  • use efs volume as storage
  • use lambda /tmp disk (512Mb) ⚠️ this will be per lambda and will be cleaned once the lambda gets deleted

[abarciauskas-bgse]

FYI @hrodmn since I know you are working on this. I'm going to assign it to you for now but we can close it and open a new issue if you prefer.

[hrodmn]

In #74 I was able to create a Lambda package that was ~230 MB, but that was without the --compile-bytecode flag in the uv pip install command:

When I do set --compile-bytecode the package size is inflated by the compiled .pyc files that are created. I do not know the exact impact pre-compiling has on Lambda application startup time, but I think it is probably worth the extra megabytes in the package size (as long as we are still under).

[vincentsarago]

I do not know the exact impact pre-compiling has on Lambda application startup time

I don't think the impact is huge, that's an nice option 👍

[hrodmn]

Another way to reduce the Lambda package size is to run strip *.so on the /asset directory:

RUN find /asset -type f -name '*.so' -exec strip {} \;

This command strips debug information from the compiled C/C++ extensions. With this change alone I can bring the Lambda package size down to 204 MB. If we strip and drop the --compile-bytecode flag the package size gets brought down to 180 MB.

I tested out the strip command on my local virtual environment and the pytest tests still pass.

[vincentsarago]

I think I had issue in the past with the strip command once running it in Lambda

[hrodmn]

I think I had issue in the past with the strip command once running it in Lambda

Thanks for the heads up! I did some testing with the titiler docker image and the container failed to start with an ImportError for numpy when I stripped all *.so* files.

After some testing it appears that if we leave the *.so* files from numpy.libs alone, everything* works.

*everything: the container can start, I can render tiles for a COG from the internet (url=https://github.com/GeoTIFF/test-data/raw/8506204783ff26a6c49ed1f721e7e1635b2e43ee/files/GA4886_VanderfordGlacier_2022_EGM2008_64m-epsg3031.cog)

[vincentsarago]

This is great 🙌

we should apply this on other projects 🙏

[abarciauskas-bgse]

Nice thank you @hrodmn