Configure vision models and backends#

VisionModel provides one interface over transformers, timm, and thingsvision for layer-level activation extraction.

Backend	Install extra	Example model ID
`transformers`	`vneurotk[vision]`	`facebook/dinov2-base`
`timm`	`vneurotk[timm]`	`resnet50.a1_in1k`
`thingsvision`	`vneurotk[thingsvision]`	backend-specific

Model construction can access the network unless weights and processor/config files are already cached. The cells below require the selected optional extra and model assets; the documentation build does not execute them.

Load a model#

import torch

import vneurotk as vtk

device = "cuda" if torch.cuda.is_available() else "cpu"
model = vtk.VisionModel(
    "facebook/dinov2-base",
    backend="transformers",
    device=device,
)

VneuroTK passes model identifiers to the selected backend unchanged and does not silently fall back to another backend or model.

Inspect and select layers#

model.print_modules(max_depth=3)
model.set_selector(module_type="Dinov2Layer")
model.module_names

# Select by name, or take a union of type and name matches.
model.set_selector(module_type="Dinov2Layer", module_name="layernorm")

# A custom list can be assembled from model.module_list.
last_norm = [item for item in model.module_list if item.module_type == "LayerNorm"][-1]
model.set_selector([last_norm])

Use vneurotk.print_cached_models() when you need to inspect locally discoverable assets. Its result depends on the machine cache and is intentionally not stored in this notebook.

Next steps#