🎯 SAM2 Segmentation Web App

A browser-based implementation of Meta's Segment Anything Model 2 (SAM2) for interactive image segmentation. This application runs entirely in the browser using ONNX Runtime Web, providing real-time AI-powered object segmentation with just a single click.

✨ Features

• 🧠 AI-Powered Segmentation: Uses Meta's SAM2 model for state-of-the-art object segmentation
• 🌐 Browser-Based: Runs entirely in the browser - no server or API calls required

🚀 Quick Start

Option 1: Direct Download

1. Clone or download this repository

   git clone https://github.com/krngd2/SamOnWeb.git
   cd sam2-web-segmentation

2. Sam Tiny Model is included

   • fp16.sam2_hiera_tiny.encoder.ort and fp16.sam2_hiera_tiny.decoder.ort are included.
   • If you want bigger models, you need to download onyx file and convert them to .ort file to use them

3. Serve the files locally

   # Using Python's built-in server
   python -m http.server 8000
   
   # Or using Node.js
   npx http-server
   
   # Or using any other local server

4. Open in browser Navigate to http://localhost:8000 in your web browser

Option 2: Web Server Deployment

Simply upload all files to any web server. The application is completely static and requires no backend processing.

📋 Requirements

None

Browser Requirements

• Modern web browser with WebAssembly support
• Recommended: Chrome 88+, Firefox 78+, Safari 14+, Edge 88+

Model Files (Required)

• fp16.sam2_hiera_tiny.encoder.ort - SAM2 encoder model (~40MB)
• fp16.sam2_hiera_tiny.decoder.ort - SAM2 decoder model (~2MB)

Note: The ONNX model files are not included in this repository due to their size. Please obtain them from the official SAM2 releases or convert them from the original PyTorch models.

🎮 How to Use

1. Upload an Image

   • Click "Choose Image" or drag & drop an image into the upload area
   • Supported formats: JPEG, PNG, WebP, and most common image formats

2. Wait for Processing

   • The app will automatically load the SAM2 models (first time only)
   • Image embeddings will be generated (takes a few seconds)

3. Segment Objects

   • Click anywhere on the uploaded image
   • The AI will automatically detect and highlight the object you clicked on
   • Click on different objects to segment multiple items

4. Manage Results

   • Use "Clear Masks" to remove all segmentation overlays
   • Use "Download Result" to save the segmented image

🏗️ Project Structure

sam2-web-segmentation/
├── index.html              # Main application file
├── style.css              # Styling and responsive design
├── README.md              # This file
├── fp16.sam2_hiera_tiny.encoder.ort  
└── fp16.sam2_hiera_tiny.decoder.ort  

🔧 Technical Details

Architecture

• Frontend: Vanilla HTML5, CSS3, and JavaScript
• AI Framework: ONNX Runtime Web for browser-based model inference
• Models: SAM2 (Segment Anything Model 2) with float16 precision
• Canvas API: For image rendering and mask visualization

Key Technologies

• ONNX Runtime Web: Enables running ONNX models in the browser
• Float16 Support: Optimized models for faster inference and smaller memory footprint
• WebAssembly: High-performance execution of AI models
• Canvas API: Real-time image manipulation and mask overlay

Performance Optimizations

• Float16 model precision for reduced memory usage
• Efficient image preprocessing with proper normalization
• Optimized mask rendering with composite operations
• Lazy loading of models to improve initial page load

🤝 Contributing

We welcome contributions! Here are some ways you can help:

Areas for Improvement

• Add support for polygon/brush selection modes
• Implement batch processing for multiple images
• Add more export formats (SVG, JSON masks)
• Improve mobile touch interaction
• Add undo/redo functionality
• Implement zoom and pan for large images

How to Contribute

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Meta AI Research for developing the Segment Anything Model 2 (SAM2)
• Microsoft for ONNX Runtime Web
• Open Source Community for various tools and libraries used

⚠️ Important Notes

Model Licensing

• The SAM2 models are subject to Meta's licensing terms
• Please ensure compliance with the original SAM2 license when using this application
• This repository contains only the web interface, not the actual model weights

Browser Compatibility

• Requires a modern browser with WebAssembly support
• Large images may require significant memory - recommended to use images under 2048x2048 pixels
• Performance varies based on device capabilities

Privacy

• All processing happens locally in your browser
• No data is sent to external servers
• Images and results remain on your device

🐛 Known Issues

• Very large images (>4K) may cause memory issues on some devices
• Initial model loading can take 10-30 seconds depending on connection speed
• Mobile devices may have slower inference times

📞 Support

If you encounter any issues or have questions:

1. Check the Issues page
2. Create a new issue with detailed information about your problem
3. Include browser version, image size, and error messages if applicable

🔮 Future Roadmap

• Video segmentation support (SAM2's video capabilities)
• Real-time webcam segmentation
• Integration with other AI models
• Collaborative editing features
• Advanced export options
• Performance analytics dashboard

---

Made with ❤️ by the open source community

Star ⭐ this repository if you find it useful!