Tangledup-ai/sam3_local
6
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Include entire sam3 package instead of just sam3 and sam3.model (#327)

Browse Source 
Summary:
there are several imports within the `sam3.model` package that reference other packages within `sam3` other than `sam3` and `sam3.model` (for example [here](https://github.com/facebookresearch/sam3/blob/main/sam3/model/sam3_tracker_base.py#L15)).  This fixes the package structure so that you can `pip install` the package and `import sam3`

Pull Request resolved: https://github.com/facebookresearch/sam3/pull/327

Reviewed By: haithamkhedr

Differential Revision: D88950127

Pulled By: lematt1991

fbshipit-source-id: 3554512d304ccdf679a9af8606bbfe1f7f2a1cfb
This commit is contained in:
Matt Le
2025-12-11 09:23:19 -08:00
committed by meta-codesync[bot]
parent 757bbb0206
commit b26a5f330e
24 changed files with 271 additions and 264 deletions
Download Patch File Download Diff File Expand all files Collapse all files

476
examples/sam3_agent.ipynb
View File

@@ -1,242 +1,242 @@
{ {
"cells": [ "cells": [
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 1,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"# Copyright (c) Meta Platforms, Inc. and affiliates." "# Copyright (c) Meta Platforms, Inc. and affiliates."
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# SAM 3 Agent" "# SAM 3 Agent"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"This notebook shows an example of how an MLLM can use SAM 3 as a tool, i.e., \"SAM 3 Agent\", to segment more complex text queries such as \"the leftmost child wearing blue vest\"." "This notebook shows an example of how an MLLM can use SAM 3 as a tool, i.e., \"SAM 3 Agent\", to segment more complex text queries such as \"the leftmost child wearing blue vest\"."
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## Env Setup" "## Env Setup"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"First install `sam3` in your environment using the [installation instructions](https://github.com/facebookresearch/sam3?tab=readme-ov-file#installation) in the repository." "First install `sam3` in your environment using the [installation instructions](https://github.com/facebookresearch/sam3?tab=readme-ov-file#installation) in the repository."
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"import torch\n", "import torch\n",
"# turn on tfloat32 for Ampere GPUs\n", "# turn on tfloat32 for Ampere GPUs\n",
"# https://pytorch.org/docs/stable/notes/cuda.html#tensorfloat-32-tf32-on-ampere-devices\n", "# https://pytorch.org/docs/stable/notes/cuda.html#tensorfloat-32-tf32-on-ampere-devices\n",
"torch.backends.cuda.matmul.allow_tf32 = True\n", "torch.backends.cuda.matmul.allow_tf32 = True\n",
"torch.backends.cudnn.allow_tf32 = True\n", "torch.backends.cudnn.allow_tf32 = True\n",
"\n", "\n",
"# use bfloat16 for the entire notebook. If your card doesn't support it, try float16 instead\n", "# use bfloat16 for the entire notebook. If your card doesn't support it, try float16 instead\n",
"torch.autocast(\"cuda\", dtype=torch.bfloat16).__enter__()\n", "torch.autocast(\"cuda\", dtype=torch.bfloat16).__enter__()\n",
"\n", "\n",
"# inference mode for the whole notebook. Disable if you need gradients\n", "# inference mode for the whole notebook. Disable if you need gradients\n",
"torch.inference_mode().__enter__()" "torch.inference_mode().__enter__()"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"import os\n", "import os\n",
"\n", "\n",
"SAM3_ROOT = os.path.dirname(os.getcwd())\n", "SAM3_ROOT = os.path.dirname(os.getcwd())\n",
"os.chdir(SAM3_ROOT)\n", "os.chdir(SAM3_ROOT)\n",
"\n", "\n",
"# setup GPU to use - A single GPU is good with the purpose of this demo\n", "# setup GPU to use - A single GPU is good with the purpose of this demo\n",
"os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n", "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n",
"_ = os.system(\"nvidia-smi\")" "_ = os.system(\"nvidia-smi\")"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## Build SAM3 Model" "## Build SAM3 Model"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"import sam3\n", "import sam3\n",
"from sam3 import build_sam3_image_model\n", "from sam3 import build_sam3_image_model\n",
"from sam3.model.sam3_image_processor import Sam3Processor\n", "from sam3.model.sam3_image_processor import Sam3Processor\n",
"\n", "\n",
"sam3_root = os.path.join(os.path.dirname(sam3.__file__), \"..\")\n", "sam3_root = os.path.dirname(sam3.__file__)\n",
"bpe_path = f\"{sam3_root}/assets/bpe_simple_vocab_16e6.txt.gz\"\n", "bpe_path = f\"{sam3_root}/assets/bpe_simple_vocab_16e6.txt.gz\"\n",
"model = build_sam3_image_model(bpe_path=bpe_path)\n", "model = build_sam3_image_model(bpe_path=bpe_path)\n",
"processor = Sam3Processor(model, confidence_threshold=0.5)" "processor = Sam3Processor(model, confidence_threshold=0.5)"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## LLM Setup\n", "## LLM Setup\n",
"\n", "\n",
"Config which MLLM to use, it can either be a model served by vLLM that you launch from your own machine or a model is served via external API. If you want to using a vLLM model, we also provided insturctions below." "Config which MLLM to use, it can either be a model served by vLLM that you launch from your own machine or a model is served via external API. If you want to using a vLLM model, we also provided insturctions below."
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"LLM_CONFIGS = {\n", "LLM_CONFIGS = {\n",
" # vLLM-served models\n", " # vLLM-served models\n",
" \"qwen3_vl_8b_thinking\": {\n", " \"qwen3_vl_8b_thinking\": {\n",
" \"provider\": \"vllm\",\n", " \"provider\": \"vllm\",\n",
" \"model\": \"Qwen/Qwen3-VL-8B-Thinking\",\n", " \"model\": \"Qwen/Qwen3-VL-8B-Thinking\",\n",
" }, \n", " },\n",
" # models served via external APIs\n", " # models served via external APIs\n",
" # add your own\n", " # add your own\n",
"}\n", "}\n",
"\n", "\n",
"model = \"qwen3_vl_8b_thinking\"\n", "model = \"qwen3_vl_8b_thinking\"\n",
"LLM_API_KEY = \"DUMMY_API_KEY\"\n", "LLM_API_KEY = \"DUMMY_API_KEY\"\n",
"\n", "\n",
"llm_config = LLM_CONFIGS[model]\n", "llm_config = LLM_CONFIGS[model]\n",
"llm_config[\"api_key\"] = LLM_API_KEY\n", "llm_config[\"api_key\"] = LLM_API_KEY\n",
"llm_config[\"name\"] = model\n", "llm_config[\"name\"] = model\n",
"\n", "\n",
"# setup API endpoint\n", "# setup API endpoint\n",
"if llm_config[\"provider\"] == \"vllm\":\n", "if llm_config[\"provider\"] == \"vllm\":\n",
" LLM_SERVER_URL = \"http://0.0.0.0:8001/v1\" # replace this with your vLLM server address as needed\n", " LLM_SERVER_URL = \"http://0.0.0.0:8001/v1\" # replace this with your vLLM server address as needed\n",
"else:\n", "else:\n",
" LLM_SERVER_URL = llm_config[\"base_url\"]" " LLM_SERVER_URL = llm_config[\"base_url\"]"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"### Setup vLLM server \n", "### Setup vLLM server \n",
"This step is only required if you are using a model served by vLLM, skip this step if you are calling LLM using an API like Gemini and GPT.\n", "This step is only required if you are using a model served by vLLM, skip this step if you are calling LLM using an API like Gemini and GPT.\n",
"\n", "\n",
"* Install vLLM (in a separate conda env from SAM 3 to avoid dependency conflicts).\n", "* Install vLLM (in a separate conda env from SAM 3 to avoid dependency conflicts).\n",
" ```bash\n", " ```bash\n",
" conda create -n vllm python=3.12\n", " conda create -n vllm python=3.12\n",
" pip install vllm --extra-index-url https://download.pytorch.org/whl/cu128\n", " pip install vllm --extra-index-url https://download.pytorch.org/whl/cu128\n",
" ```\n", " ```\n",
"* Start vLLM server on the same machine of this notebook\n", "* Start vLLM server on the same machine of this notebook\n",
" ```bash\n", " ```bash\n",
" # qwen 3 VL 8B thinking\n", " # qwen 3 VL 8B thinking\n",
" vllm serve Qwen/Qwen3-VL-8B-Thinking --tensor-parallel-size 4 --allowed-local-media-path / --enforce-eager --port 8001\n", " vllm serve Qwen/Qwen3-VL-8B-Thinking --tensor-parallel-size 4 --allowed-local-media-path / --enforce-eager --port 8001\n",
" ```" " ```"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## Run SAM3 Agent Inference" "## Run SAM3 Agent Inference"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"from functools import partial\n", "from functools import partial\n",
"from IPython.display import display, Image\n", "from IPython.display import display, Image\n",
"from sam3.agent.client_llm import send_generate_request as send_generate_request_orig\n", "from sam3.agent.client_llm import send_generate_request as send_generate_request_orig\n",
"from sam3.agent.client_sam3 import call_sam_service as call_sam_service_orig\n", "from sam3.agent.client_sam3 import call_sam_service as call_sam_service_orig\n",
"from sam3.agent.inference import run_single_image_inference" "from sam3.agent.inference import run_single_image_inference"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,
"metadata": { "metadata": {
"output": { "output": {
"id": 689664053567678, "id": 689664053567678,
"loadingStatus": "loaded" "loadingStatus": "loaded"
}
},
"outputs": [],
"source": [
"# prepare input args and run single image inference\n",
"image = \"assets/images/test_image.jpg\"\n",
"prompt = \"the leftmost child wearing blue vest\"\n",
"image = os.path.abspath(image)\n",
"send_generate_request = partial(send_generate_request_orig, server_url=LLM_SERVER_URL, model=llm_config[\"model\"], api_key=llm_config[\"api_key\"])\n",
"call_sam_service = partial(call_sam_service_orig, sam3_processor=processor)\n",
"output_image_path = run_single_image_inference(\n",
" image, prompt, llm_config, send_generate_request, call_sam_service,\n",
" debug=True, output_dir=\"agent_output\"\n",
")\n",
"\n",
"# display output\n",
"if output_image_path is not None:\n",
" display(Image(filename=output_image_path))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"fileHeader": "",
"fileUid": "be59e249-6c09-4634-a9e7-1f06fd233c42",
"isAdHoc": false,
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.11"
} }
},
"outputs": [],
"source": [
"# prepare input args and run single image inference\n",
"image = \"assets/images/test_image.jpg\"\n",
"prompt = \"the leftmost child wearing blue vest\"\n",
"image = os.path.abspath(image)\n",
"send_generate_request = partial(send_generate_request_orig, server_url=LLM_SERVER_URL, model=llm_config[\"model\"], api_key=llm_config[\"api_key\"])\n",
"call_sam_service = partial(call_sam_service_orig, sam3_processor=processor)\n",
"output_image_path = run_single_image_inference(\n",
" image, prompt, llm_config, send_generate_request, call_sam_service, \n",
" debug=True, output_dir=\"agent_output\"\n",
")\n",
"\n",
"# display output\n",
"if output_image_path is not None:\n",
" display(Image(filename=output_image_path))"
]
}, },
{ "nbformat": 4,
"cell_type": "code", "nbformat_minor": 2
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"fileHeader": "",
"fileUid": "be59e249-6c09-4634-a9e7-1f06fd233c42",
"isAdHoc": false,
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
} }

8
pyproject.toml
View File

@@ -82,8 +82,12 @@ train = [
"Homepage" = "https://github.com/facebookresearch/sam3" "Homepage" = "https://github.com/facebookresearch/sam3"
"Bug Tracker" = "https://github.com/facebookresearch/sam3/issues" "Bug Tracker" = "https://github.com/facebookresearch/sam3/issues"
[tool.setuptools] [tool.setuptools.packages.find]
packages = ["sam3", "sam3.model"] include = ["sam3*"]
exclude = ["build*", "scripts*", "examples*"]
[tool.setuptools.package-data]
sam3 = ["assets/*.txt.gz"]
[tool.setuptools.dynamic] [tool.setuptools.dynamic]
version = {attr = "sam3.__version__"} version = {attr = "sam3.__version__"}

0
assets/bpe_simple_vocab_16e6.txt.gz → sam3/assets/bpe_simple_vocab_16e6.txt.gz
View File

11
sam3/model_builder.py
View File

@@ -3,6 +3,8 @@
import os import os
from typing import Optional from typing import Optional
import pkg_resources
import torch import torch
import torch.nn as nn import torch.nn as nn
from huggingface_hub import hf_hub_download from huggingface_hub import hf_hub_download
@@ -580,9 +582,10 @@ def build_sam3_image_model(
A SAM3 image model A SAM3 image model
""" """
if bpe_path is None: if bpe_path is None:
bpe_path = os.path.join( bpe_path = pkg_resources.resource_filename(
os.path.dirname(__file__), "..", "assets", "bpe_simple_vocab_16e6.txt.gz" "sam3", "assets/bpe_simple_vocab_16e6.txt.gz"
) )
# Create visual components # Create visual components
compile_mode = "default" if compile else None compile_mode = "default" if compile else None
vision_encoder = _create_vision_backbone( vision_encoder = _create_vision_backbone(
@@ -668,8 +671,8 @@ def build_sam3_video_model(
Sam3VideoInferenceWithInstanceInteractivity: The instantiated dense tracking model Sam3VideoInferenceWithInstanceInteractivity: The instantiated dense tracking model
""" """
if bpe_path is None: if bpe_path is None:
bpe_path = os.path.join( bpe_path = pkg_resources.resource_filename(
os.path.dirname(__file__), "..", "assets", "bpe_simple_vocab_16e6.txt.gz" "sam3", "assets/bpe_simple_vocab_16e6.txt.gz"
) )
# Build Tracker module # Build Tracker module

2
sam3/train/configs/eval_base.yaml
View File

@@ -39,7 +39,7 @@ paths:
# path to the SA-Co/silver images # path to the SA-Co/silver images
silver_img_path: <YOUR_SILVER_IMG_DIR> silver_img_path: <YOUR_SILVER_IMG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
# ============================================================================ # ============================================================================

2
sam3/train/configs/odinw13/odinw_text_and_visual.yaml
View File

@@ -10,7 +10,7 @@ defaults:
paths: paths:
odinw_data_root: <YOUR_DATA_DIR> odinw_data_root: <YOUR_DATA_DIR>
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}} supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}}
# Validation transforms pipeline # Validation transforms pipeline

2
sam3/train/configs/odinw13/odinw_text_only.yaml
View File

@@ -10,7 +10,7 @@ defaults:
paths: paths:
odinw_data_root: <YOUR_DATA_DIR> odinw_data_root: <YOUR_DATA_DIR>
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}} supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}}

2
sam3/train/configs/odinw13/odinw_text_only_positive.yaml
View File

@@ -10,7 +10,7 @@ defaults:
paths: paths:
odinw_data_root: <YOUR_DATA_DIR> odinw_data_root: <YOUR_DATA_DIR>
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}} supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}}

2
sam3/train/configs/odinw13/odinw_text_only_train.yaml
View File

@@ -10,7 +10,7 @@ defaults:
paths: paths:
odinw_data_root: <YOUR_DATA_DIR> odinw_data_root: <YOUR_DATA_DIR>
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
odinw_train: odinw_train:

2
sam3/train/configs/odinw13/odinw_visual_only.yaml
View File

@@ -10,7 +10,7 @@ defaults:
paths: paths:
odinw_data_root: <YOUR_DATA_DIR> odinw_data_root: <YOUR_DATA_DIR>
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}} supercategory_tuple: ${all_odinw_supercategories.${string:${submitit.job_array.task_index}}}

2
sam3/train/configs/roboflow_v100/roboflow_v100_eval.yaml
View File

@@ -8,7 +8,7 @@ defaults:
paths: paths:
roboflow_vl_100_root: <YOUR_DATASET_DIR> roboflow_vl_100_root: <YOUR_DATASET_DIR>
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
# Roboflow dataset configuration # Roboflow dataset configuration
roboflow_train: roboflow_train:

2
sam3/train/configs/roboflow_v100/roboflow_v100_full_ft_100_images.yaml
View File

@@ -8,7 +8,7 @@ defaults:
paths: paths:
roboflow_vl_100_root: <YOUR_DATASET_DIR> roboflow_vl_100_root: <YOUR_DATASET_DIR>
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
# Roboflow dataset configuration # Roboflow dataset configuration
roboflow_train: roboflow_train:

2
sam3/train/configs/saco_video_evals/saco_veval_sav_test.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_test.json ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_test.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_sav_test_noheur.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_test.json ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_test.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_sav_val.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_val.json ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_val.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_sav_val_noheur.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_val.json ytvis_json: <YOUR_GT_PATH>/saco_veval_sav_val.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_smartglasses_test.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_test.json ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_test.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_smartglasses_test_noheur.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_test.json ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_test.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_smartglasses_val.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_val.json ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_val.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_smartglasses_val_noheur.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_val.json ytvis_json: <YOUR_GT_PATH>/saco_veval_smartglasses_val.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_yt1b_test.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_test.json ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_test.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_yt1b_test_noheur.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_test.json ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_test.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_yt1b_val.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_val.json ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_val.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================

2
sam3/train/configs/saco_video_evals/saco_veval_yt1b_val_noheur.yaml
View File

@@ -11,7 +11,7 @@ paths:
experiment_log_dir: <YOUR EXPERIMENET LOG_DIR> experiment_log_dir: <YOUR EXPERIMENET LOG_DIR>
ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_val.json ytvis_json: <YOUR_GT_PATH>/saco_veval_yt1b_val.json
ytvis_dir : <YOUR_VIDEO_JPG_DIR> ytvis_dir : <YOUR_VIDEO_JPG_DIR>
bpe_path: <BPE_PATH> # This should be under assets/bpe_simple_vocab_16e6.txt.gz bpe_path: <BPE_PATH> # This should be under sam3/assets/bpe_simple_vocab_16e6.txt.gz
num_videos: null num_videos: null
# ============================================================================ # ============================================================================