A lightweight caching component for KubeRocketCI pipeline artifacts, built in Go with Echo framework. Simplified for memory efficiency and stability in resource-constrained environments.
Note: This is a simplified version optimized for memory efficiency. Some advanced features have been removed to resolve OOM issues in Kubernetes environments with limited memory (512MB).
- Install
- Configuration
- Core Features
- Limitations
- Usage
- Container Deployment
- Setup
- API
- Use Cases
- LICENSE
go install github.com/KubeRocketCI/krci-cacheConfiguration is done via environment variables:
- UPLOADER_HOST -- hostname to bind to (default: localhost)
- UPLOADER_PORT -- port to bind to (default: 8080)
- UPLOADER_DIRECTORY -- Directory where to upload (default: ./pub)
- UPLOADER_UPLOAD_CREDENTIALS -- Protect upload/delete endpoints with username:password (e.g:
username:password)
export UPLOADER_HOST="0.0.0.0"
export UPLOADER_PORT="8080"
export UPLOADER_DIRECTORY="/var/cache/artifacts"
export UPLOADER_UPLOAD_CREDENTIALS="cache-user:secure-password"The service should be deployed behind proper authentication and access controls. Do not expose this directly to the internet without protection.
- Simplified Design: Reduced complexity to minimize memory footprint
- Essential Middleware Only: Only recovery and basic logging middleware
- Direct File Operations: Streamlined upload/download without complex buffering
- Built-in Tar.gz Limits: File extraction with safety limits (2GB per file, 8GB total)
- Path Traversal Protection: Prevents uploads outside designated directory
- Basic Authentication: Optional username/password protection for sensitive endpoints
- Tar.gz Safety: Built-in protection against zip bombs and malicious archives
- Directory Isolation: All operations confined to the configured upload directory
The service has been simplified for memory efficiency, with some trade-offs in functionality:
- Individual File Size: Maximum 2GB per file within tar.gz archives
- Archive Total Size: Maximum 8GB total uncompressed size for tar.gz uploads
- Regular File Uploads: No configurable size limits (limited by available disk space)
- Security: Built-in protection against zip bombs, path traversal, and malicious archives
- Basic file upload/download
- Tar.gz extraction with built-in size limits
- Basic authentication (
UPLOADER_UPLOAD_CREDENTIALS) - Health check endpoint (
/health) - File deletion (single and batch by age)
- Path traversal protection
- Static file serving
- method: GET
- path: /health
- description: Health check endpoint for load balancers and monitoring
- response: JSON with status, timestamp, and version
curl http://localhost:8080/healthThe service accepts HTTP form fields:
- file: The file stream of the upload
- path: The target path for the file
- targz: Set to extract tar.gz archives automatically on the filesystem
All endpoints return JSON responses for better integration:
{
"message": "File has been uploaded to example.txt",
"path": "example.txt",
"size": 1024
}The application is containerized using a multi-architecture approach with pre-built binaries. The container runs as a non-root user for security.
Use the provided Dockerfile which supports both amd64 and arm64 architectures:
docker build --build-arg TARGETARCH=amd64 -t krci-cache .# docker-compose.yml example
version: '3.8'
services:
krci-cache:
image: krci-cache:latest
ports:
- "8080:8080"
environment:
- UPLOADER_HOST=0.0.0.0
- UPLOADER_PORT=8080
- UPLOADER_DIRECTORY=/var/cache/artifacts
- UPLOADER_UPLOAD_CREDENTIALS=cache-user:secure-password
volumes:
- ./cache:/var/cache/artifacts
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3You can run the service directly or use containerization. The default upload credentials are username:password for the /upload and /delete endpoints.
For production deployment, set proper credentials via the UPLOADER_UPLOAD_CREDENTIALS environment variable.
Set up authentication credentials:
export UPLOADER_UPLOAD_CREDENTIALS="username:password"
./krci-cacheTest the deployment:
echo "HELLO WORLD" > /tmp/hello.txt
curl -u username:password -F path=hello-upload.txt -X POST -F file=@/tmp/hello.txt http://localhost:8080/upload
curl http://localhost:8080/hello-upload.txt-
method: POST
-
path: /upload
-
arguments:
- path: Target path for the file (relative to upload directory, directory traversal prevented)
- file: File post data (no size limits, limited by available disk space)
- targz: Boolean flag to extract tar.gz files on filesystem (tar.gz uploads subject to built-in size limits: max 2GB per file, 8GB total)
-
examples:
# Regular file upload (no size limits)
curl -u username:password -F path=hello-upload.txt -X POST -F file=@/tmp/hello.txt http://localhost:8080/upload# Large file upload (limited by disk space)
curl -u username:password -F path=large-database.sql -X POST -F file=@/path/to/large-database.sql http://localhost:8080/upload# Extract tar.gz automatically (max 2GB per file, 8GB total uncompressed)
tar czf - /path/to/directory|curl -u username:password -F path=hello-upload.txt -F targz=true -X POST -F file=@- http://localhost:8080/upload-
method: DELETE
-
path: /upload
-
arguments:
- path: Path to delete
-
example:
curl -u username:password -F path=hello-upload.txt -X DELETE http://localhost:8080/upload-
method: DELETE
-
path: /delete
-
arguments:
- path: Directory path to clean up
- days: Delete files older than X days
- recursive: Recursively delete in subdirectories (defaults to
false)
-
example:
curl -u username:password -F path=/path/to/directory -F days=1 -F recursive=true -X DELETE http://localhost:8080/deleteStore and manage build artifacts with basic cleanup functionality.
Upload build artifacts with extraction:
# Upload and extract build artifacts to versioned directory
curl -u cache-user:secure-password \
-X POST \
-F "[email protected]" \
-F "targz=true" \
-F "path=builds/v1.2.3/" \
http://cache-server:8080/uploadDownload specific artifacts:
# Access extracted binary
curl http://cache-server:8080/builds/v1.2.3/bin/application
# Check if artifact exists
curl -I http://cache-server:8080/builds/v1.2.3/config.jsonCleanup old build artifacts:
# Remove builds older than 30 days
curl -u cache-user:secure-password \
-X DELETE \
"http://cache-server:8080/delete?path=builds&days=30&recursive=true"Simple file upload and retrieval for small to medium-scale applications.
Upload files:
# Upload application logs
curl -u cache-user:secure-password \
-X POST \
-F "[email protected]" \
-F "path=logs/application-2024-01-15.log" \
http://cache-server:8080/uploadBasic retention:
# Remove old files after 7 days
curl -u cache-user:secure-password \
-X DELETE \
"http://cache-server:8080/delete?path=logs&days=7&recursive=true"Host simple static websites and documentation.
Deploy documentation:
# Upload and extract documentation site
tar czf docs.tar.gz -C /path/to/docs .
curl -u cache-user:secure-password \
-X POST \
-F "[email protected]" \
-F "targz=true" \
-F "path=docs/v2.1/" \
http://cache-server:8080/uploadAccess content:
# Browse documentation
curl http://cache-server:8080/docs/v2.1/index.html