​
SAM2 Model

SAM2 (Segment Anything Model 2) is Meta’s advanced image segmentation model. With InferX, you can run SAM2 on any device using the same API - from Jetson edge devices to powerful GPU servers.

​
Features

  • Universal Segmentation: Segment any object in any image
  • Point and Click: Interactive segmentation with point prompts
  • Box Prompts: Segment objects using bounding box inputs
  • Cross-Platform: Same code works on Jetson, GPU, or CPU
  • Real-time Performance: Optimized for fast inference

​
Installation

SAM2 is included with InferX. You’ll need to download the model weights:

# Install InferX
pip install git+https://github.com/exla-ai/InferX.git

# Download SAM2 model weights
mkdir -p ~/.cache/inferx/sam2/
wget -O ~/.cache/inferx/sam2/sam2_b.pth https://huggingface.co/facebook/sam2-base/resolve/main/sam2_b.pth

​
Basic Usage

from inferx.models.sam2 import sam2
import cv2

# Initialize the model
model = sam2()

# Load an image
image = cv2.imread("path/to/your/image.jpg")

# Segment with point prompts
results = model.inference(
    image=image,
    point_prompts=[(100, 150)],  # x, y coordinates
    point_labels=[1]  # 1 for foreground, 0 for background
)

# Get the segmentation mask
mask = results['masks'][0]

​
Advanced Usage

​
Box Prompts

from inferx.models.sam2 import sam2

model = sam2()
image = cv2.imread("image.jpg")

# Use bounding box as prompt
results = model.inference(
    image=image,
    box_prompts=[(50, 50, 200, 200)]  # x1, y1, x2, y2
)

​
Multiple Objects

# Segment multiple objects with different prompts
results = model.inference(
    image=image,
    point_prompts=[(100, 100), (300, 200)],
    point_labels=[1, 1],  # Both foreground points
    box_prompts=[(400, 300, 500, 400)]
)

# Access individual masks
for i, mask in enumerate(results['masks']):
    cv2.imwrite(f"mask_{i}.png", mask * 255)

​
Interactive Segmentation

import matplotlib.pyplot as plt

def on_click(event):
    if event.button == 1:  # Left click
        x, y = int(event.xdata), int(event.ydata)
        
        results = model.inference(
            image=image,
            point_prompts=[(x, y)],
            point_labels=[1]
        )
        
        # Display result
        plt.imshow(results['masks'][0], alpha=0.5)
        plt.draw()

# Set up interactive plot
fig, ax = plt.subplots()
ax.imshow(image)
fig.canvas.mpl_connect('button_press_event', on_click)
plt.show()

​
Performance

InferX optimizes SAM2 for your hardware:

HardwareInference TimeMemory Usage
Jetson AGX Orin~200ms~4GB
RTX 4090~80ms~6GB
Intel i7 CPU~2s~3GB

​
Response Format

{
    'masks': [numpy.ndarray],  # List of segmentation masks
    'scores': [float],         # Confidence scores
    'logits': [numpy.ndarray]  # Raw model outputs
}

​
Example Applications

​
Object Extraction

from inferx.models.sam2 import sam2
import cv2
import numpy as np

model = sam2()
image = cv2.imread("photo.jpg")

# Click on object to segment
results = model.inference(
    image=image,
    point_prompts=[(250, 200)],
    point_labels=[1]
)

# Extract object using mask
mask = results['masks'][0]
extracted = image * mask[:, :, np.newaxis]

cv2.imwrite("extracted_object.jpg", extracted)

​
Batch Processing

import os

model = sam2()

for filename in os.listdir("images/"):
    if filename.endswith(('.jpg', '.png')):
        image = cv2.imread(f"images/{filename}")
        
        # Auto-segment center point
        h, w = image.shape[:2]
        results = model.inference(
            image=image,
            point_prompts=[(w//2, h//2)],
            point_labels=[1]
        )
        
        # Save mask
        mask = results['masks'][0]
        cv2.imwrite(f"masks/{filename}", mask * 255)

​
Hardware Detection

✨ InferX - SAM2 Model ✨
🔍 Device Detected: AGX_ORIN
⠏ [1.2s] Loading SAM2 model weights
✓ [1.5s] Ready for segmentation

​
Next Steps