⚡ Zero OOM Guarantee
Calculates parameter count, precision bytes (FP32/FP16/INT8/INT4), and 25% activation headroom before suggesting or loading models.
📊 Empirical Benchmarking
Runs test evaluations across candidate models fitting hardware specs, ranking them by live latency (ms) and confidence score.
🚀 Fast Ensembles
Finds ultra-compact models fitting ≤ 35% memory headroom and aggregates predictions with weighted average or majority voting.
🔄 Dynamic Hot-Swapping
Clients consume an abstract gateway API (/predict). Swap models dynamically while purging VRAM to prevent memory leaks.
Interactive VRAM / Memory Fit Calculator
Estimated Memory Required: 2.33 GB (including 25% activation headroom)
Quickstart
# Install ModelFit-MCP
git clone https://github.com/DumboDhruvi/ModelFit-MCP.git
cd ModelFit-MCP
pip install -e .
# Inspect host hardware headroom
modelfit specs
# Search models fitting your machine
modelfit search "plant disease" --task image-classification
# Benchmark candidate models
modelfit benchmark "model-1,model-2" --samples "sample.jpg"
# Run high-speed ensemble
modelfit ensemble "model-1,model-2" --input "sample.jpg"