SAM2 Model

SAM2 (Segment Anything Model 2) is Meta’s advanced image segmentation model. With InferX, you can run SAM2 on any device using the same API - from Jetson edge devices to powerful GPU servers.

Features

  • Universal Segmentation: Segment any object in any image
  • Point and Click: Interactive segmentation with point prompts
  • Box Prompts: Segment objects using bounding box inputs
  • Cross-Platform: Same code works on Jetson, GPU, or CPU
  • Real-time Performance: Optimized for fast inference

Installation

SAM2 is included with InferX. You’ll need to download the model weights:

# Install InferX
pip install git+https://github.com/exla-ai/InferX.git

# Download SAM2 model weights
mkdir -p ~/.cache/inferx/sam2/
wget -O ~/.cache/inferx/sam2/sam2_b.pth https://huggingface.co/facebook/sam2-base/resolve/main/sam2_b.pth

Basic Usage

from inferx.models.sam2 import sam2
import cv2

# Initialize the model
model = sam2()

# Load an image
image = cv2.imread("path/to/your/image.jpg")

# Segment with point prompts
results = model.inference(
    image=image,
    point_prompts=[(100, 150)],  # x, y coordinates
    point_labels=[1]  # 1 for foreground, 0 for background
)

# Get the segmentation mask
mask = results['masks'][0]

Advanced Usage

Box Prompts

from inferx.models.sam2 import sam2

model = sam2()
image = cv2.imread("image.jpg")

# Use bounding box as prompt
results = model.inference(
    image=image,
    box_prompts=[(50, 50, 200, 200)]  # x1, y1, x2, y2
)

Multiple Objects

# Segment multiple objects with different prompts
results = model.inference(
    image=image,
    point_prompts=[(100, 100), (300, 200)],
    point_labels=[1, 1],  # Both foreground points
    box_prompts=[(400, 300, 500, 400)]
)

# Access individual masks
for i, mask in enumerate(results['masks']):
    cv2.imwrite(f"mask_{i}.png", mask * 255)

Interactive Segmentation

import matplotlib.pyplot as plt

def on_click(event):
    if event.button == 1:  # Left click
        x, y = int(event.xdata), int(event.ydata)
        
        results = model.inference(
            image=image,
            point_prompts=[(x, y)],
            point_labels=[1]
        )
        
        # Display result
        plt.imshow(results['masks'][0], alpha=0.5)
        plt.draw()

# Set up interactive plot
fig, ax = plt.subplots()
ax.imshow(image)
fig.canvas.mpl_connect('button_press_event', on_click)
plt.show()

Performance

InferX optimizes SAM2 for your hardware:

HardwareInference TimeMemory Usage
Jetson AGX Orin~200ms~4GB
RTX 4090~80ms~6GB
Intel i7 CPU~2s~3GB

Response Format

{
    'masks': [numpy.ndarray],  # List of segmentation masks
    'scores': [float],         # Confidence scores
    'logits': [numpy.ndarray]  # Raw model outputs
}

Example Applications

Object Extraction

from inferx.models.sam2 import sam2
import cv2
import numpy as np

model = sam2()
image = cv2.imread("photo.jpg")

# Click on object to segment
results = model.inference(
    image=image,
    point_prompts=[(250, 200)],
    point_labels=[1]
)

# Extract object using mask
mask = results['masks'][0]
extracted = image * mask[:, :, np.newaxis]

cv2.imwrite("extracted_object.jpg", extracted)

Batch Processing

import os

model = sam2()

for filename in os.listdir("images/"):
    if filename.endswith(('.jpg', '.png')):
        image = cv2.imread(f"images/{filename}")
        
        # Auto-segment center point
        h, w = image.shape[:2]
        results = model.inference(
            image=image,
            point_prompts=[(w//2, h//2)],
            point_labels=[1]
        )
        
        # Save mask
        mask = results['masks'][0]
        cv2.imwrite(f"masks/{filename}", mask * 255)

Hardware Detection

✨ InferX - SAM2 Model ✨
🔍 Device Detected: AGX_ORIN
⠏ [1.2s] Loading SAM2 model weights
✓ [1.5s] Ready for segmentation

Next Steps

SAM2 Model

SAM2 (Segment Anything Model 2) is Meta’s advanced image segmentation model. With InferX, you can run SAM2 on any device using the same API - from Jetson edge devices to powerful GPU servers.

Features

  • Universal Segmentation: Segment any object in any image
  • Point and Click: Interactive segmentation with point prompts
  • Box Prompts: Segment objects using bounding box inputs
  • Cross-Platform: Same code works on Jetson, GPU, or CPU
  • Real-time Performance: Optimized for fast inference

Installation

SAM2 is included with InferX. You’ll need to download the model weights:

# Install InferX
pip install git+https://github.com/exla-ai/InferX.git

# Download SAM2 model weights
mkdir -p ~/.cache/inferx/sam2/
wget -O ~/.cache/inferx/sam2/sam2_b.pth https://huggingface.co/facebook/sam2-base/resolve/main/sam2_b.pth

Basic Usage

from inferx.models.sam2 import sam2
import cv2

# Initialize the model
model = sam2()

# Load an image
image = cv2.imread("path/to/your/image.jpg")

# Segment with point prompts
results = model.inference(
    image=image,
    point_prompts=[(100, 150)],  # x, y coordinates
    point_labels=[1]  # 1 for foreground, 0 for background
)

# Get the segmentation mask
mask = results['masks'][0]

Advanced Usage

Box Prompts

from inferx.models.sam2 import sam2

model = sam2()
image = cv2.imread("image.jpg")

# Use bounding box as prompt
results = model.inference(
    image=image,
    box_prompts=[(50, 50, 200, 200)]  # x1, y1, x2, y2
)

Multiple Objects

# Segment multiple objects with different prompts
results = model.inference(
    image=image,
    point_prompts=[(100, 100), (300, 200)],
    point_labels=[1, 1],  # Both foreground points
    box_prompts=[(400, 300, 500, 400)]
)

# Access individual masks
for i, mask in enumerate(results['masks']):
    cv2.imwrite(f"mask_{i}.png", mask * 255)

Interactive Segmentation

import matplotlib.pyplot as plt

def on_click(event):
    if event.button == 1:  # Left click
        x, y = int(event.xdata), int(event.ydata)
        
        results = model.inference(
            image=image,
            point_prompts=[(x, y)],
            point_labels=[1]
        )
        
        # Display result
        plt.imshow(results['masks'][0], alpha=0.5)
        plt.draw()

# Set up interactive plot
fig, ax = plt.subplots()
ax.imshow(image)
fig.canvas.mpl_connect('button_press_event', on_click)
plt.show()

Performance

InferX optimizes SAM2 for your hardware:

HardwareInference TimeMemory Usage
Jetson AGX Orin~200ms~4GB
RTX 4090~80ms~6GB
Intel i7 CPU~2s~3GB

Response Format

{
    'masks': [numpy.ndarray],  # List of segmentation masks
    'scores': [float],         # Confidence scores
    'logits': [numpy.ndarray]  # Raw model outputs
}

Example Applications

Object Extraction

from inferx.models.sam2 import sam2
import cv2
import numpy as np

model = sam2()
image = cv2.imread("photo.jpg")

# Click on object to segment
results = model.inference(
    image=image,
    point_prompts=[(250, 200)],
    point_labels=[1]
)

# Extract object using mask
mask = results['masks'][0]
extracted = image * mask[:, :, np.newaxis]

cv2.imwrite("extracted_object.jpg", extracted)

Batch Processing

import os

model = sam2()

for filename in os.listdir("images/"):
    if filename.endswith(('.jpg', '.png')):
        image = cv2.imread(f"images/{filename}")
        
        # Auto-segment center point
        h, w = image.shape[:2]
        results = model.inference(
            image=image,
            point_prompts=[(w//2, h//2)],
            point_labels=[1]
        )
        
        # Save mask
        mask = results['masks'][0]
        cv2.imwrite(f"masks/{filename}", mask * 255)

Hardware Detection

✨ InferX - SAM2 Model ✨
🔍 Device Detected: AGX_ORIN
⠏ [1.2s] Loading SAM2 model weights
✓ [1.5s] Ready for segmentation

Next Steps