Coding LLMs & Agentic AI

In Lesson 04, we explored how LLMs work under the hood — from neural networks and transformers to quantization, inference engines, and GPU hardware. You now understand what these models are and how to run them.

This lesson narrows the focus to a single, high-impact domain: coding. Not all LLMs are equal at writing code. Some are general-purpose models that happen to know some Python. Others are purpose-built coding specialists, trained on trillions of code tokens, fine-tuned on instruction-following for development tasks, and optimized for the specific patterns that make code generation useful.

The difference matters. A coding-specialized 32B model can outperform a general-purpose 70B model on programming benchmarks — while using half the VRAM and running twice as fast. Choosing the right model for coding is not just a preference; it is a resource and quality decision.

How do you know if a model is good at coding? You cannot just ask it to write “Hello World” and declare victory. The industry uses standardized benchmarks that test increasingly realistic coding scenarios.

Why Multiple Benchmarks Matter

A model that scores 92% on HumanEval (isolated function generation) might score only 40% on SWE-bench (full-repo issue resolution). These benchmarks test fundamentally different skills: writing a function from a docstring is very different from navigating a 50-file codebase, understanding the bug, and producing a correct multi-file patch.

Score Landscape — How Top Models Compare

The table below shows approximate performance tiers for major open-weight coding models as of early 2026. Scores shift with every release — treat this as a landscape orientation, not a leaderboard.

The Complexity Ladder

Coding benchmarks form a complexity ladder. At the base sits HumanEval: generating an isolated function from a docstring. In the middle, Aider Polyglot tests multi-language code editing via natural language instructions. At the top, SWE-bench demands full-repo navigation, bug identification, and multi-file patching to resolve real GitHub issues.

A model’s position on this ladder determines what tasks you can trust it with. A model that excels at HumanEval but struggles on SWE-bench is suitable for code completion but not for autonomous issue resolution. Understanding where each benchmark sits tells you what the score actually measures.

Not every LLM is trained for code. The major coding model families are purpose-built: trained on curated code datasets, fine-tuned on coding instructions, and benchmarked against programming-specific evaluations. Understanding the landscape helps you pick the right tool.

What Makes a “Coding Model”?

A coding model differs from a general-purpose model in three ways: training data (heavy code corpus), fine-tuning targets (code completion, generation, editing instructions), and evaluation focus (HumanEval, SWE-bench, not just MMLU).

Coverage Matrix — What Each Family Does Best

Each coding model family has strengths and gaps. The matrix below maps families to common coding tasks, showing where each excels.

The Open-Weight Coding Landscape

The open-weight ecosystem for coding models has matured rapidly. Families like Qwen-Coder, DeepSeek-Coder, and StarCoder2 offer models across a range of parameter counts, making self-hosted coding assistance accessible from consumer GPUs to datacenter deployments.

The most powerful use of coding LLMs is not generating code in isolation — it is running them as agents that can read files, write code, run tests, and iterate on failures. This is the difference between a code completion tool and an AI pair programmer.

The Agent Loop

An agentic coding workflow follows a loop: the model receives a task, decides which tools to call (read file, write file, run command), executes the tool calls, observes the results, and decides what to do next. This continues until the task is complete or the model determines it cannot proceed.

Function Calling vs Tool Use

“Function calling” and “tool use” describe the same capability: the model outputs structured JSON indicating which function to call with what arguments, instead of generating plain text. The runtime (your agent framework) executes the function and feeds the result back to the model.

Simple Generation vs Agentic Workflow

The difference between a chatbot that generates code and an agent that writes software is the loop. Simple generation is one-shot: prompt in, code out. Agentic workflows iterate autonomously until the task is done.

For a model to call tools, it needs to output structured data in a specific format. Different model families use different formats. The inference engine must know how to parse the model’s output into actual function calls. This is where tool call parsers come in.

Why Format Matching Matters

If you serve a model that outputs Hermes-style tool calls through an engine configured for ChatML format, the tool calls will fail silently or produce garbage. The parser must match the model’s training format.

Format Comparison

Each model family uses a different syntax for tool calls. The inference engine must parse the model’s raw output and extract structured function call data. If the parser expects Hermes format but the model outputs Mistral format, the call fails silently.

vLLM Parser Configuration

vLLM supports multiple tool call parsers via the --tool-call-parser flag. Choosing the correct parser for your model is essential for reliable agentic workflows.

# Hermes-2 fine-tune (most community models)
vllm serve my-model --tool-call-parser hermes

# Llama 3.1+ Instruct (native Meta format)
vllm serve meta-llama/Llama-3.1-70B-Instruct \
  --tool-call-parser llama3_json

# Mistral / Codestral
vllm serve mistralai/Codestral-22B-v0.1 \
  --tool-call-parser mistral

# DeepSeek-V3 (dedicated parser)
vllm serve deepseek-ai/DeepSeek-V3 \
  --tool-call-parser deepseek_v3

# Enable tool calling on the OpenAI-compatible endpoint
# Then call with tools=[...] in your API request

Parser Matching Decision Table

Use this table to look up the correct --tool-call-parser flag for your model. When in doubt, check the model’s tokenizer_config.json for its chat template.

When you ask a coding model to write a penetration testing script, bypass a rate limiter, or implement a web scraper, some models will refuse. Others will comply without hesitation. This difference comes from alignment training — the safety fine-tuning applied after the model learns to code.

What Alignment Does to Coding Models

Alignment (RLHF, DPO, or constitutional AI methods) teaches models to refuse harmful requests. For general chat, this is straightforward. For coding, the boundary is murkier: security research, DevOps automation, and system administration often require code that looks like it could be misused.

Refusal Patterns in Practice

The impact of alignment on coding varies by task. Standard application development is unaffected. Security research, DevOps tooling, and system-level code trigger refusals more frequently in heavily aligned models.

Note that refusal behavior varies between model versions and even between quantization levels of the same model. The patterns above are generalizations — always test with your specific model and use case.

Uncensored Variants

Some open-weight models are released in “uncensored” or “abliterated” variants where safety fine-tuning has been partially removed. These models are more compliant for legitimate coding tasks but also remove guardrails entirely.

With dozens of coding-capable models available, how do you choose? The answer depends on your task type, hardware, latency requirements, and quality threshold.

Task-Based Selection Framework

Different coding tasks have different model requirements. Code completion (fill-in-the-middle) needs speed above all. Code generation needs quality and instruction following. Code review needs reasoning depth. Agentic workflows need reliable tool calling.

Quick Decision Flow

Hardware-Constrained Selection

Your GPU determines your ceiling. A consumer RTX 4090 (24 GB VRAM) runs different models than a dual-A6000 workstation (96 GB total). The model selection framework must account for what actually fits.

A coding model is only useful if it integrates into your actual development workflow. This section covers the tools that connect self-hosted models to your editor, terminal, and CI pipeline.

IDE Integration Options

The three dominant approaches to local AI coding assistance are: editor extensions (Continue.dev for VS Code/JetBrains), terminal agents (Aider for git-aware pair programming), and AI-native editors (Cursor, built from the ground up around AI interaction).

Self-Hosted vs Cloud-Backed

Every IDE integration tool can connect to either cloud APIs (OpenAI, Anthropic) or local inference endpoints (vLLM, Ollama). Running your own model gives you privacy, zero per-token cost, and the ability to use uncensored or fine-tuned models.

Quick Setup: Continue.dev + vLLM

The most common self-hosted coding setup pairs Continue.dev (IDE extension) with vLLM (inference engine). Here is the minimal configuration.

# Serve Qwen2.5-Coder-32B with tool calling enabled
vllm serve Qwen/Qwen2.5-Coder-32B-Instruct \
  --tool-call-parser internlm \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.90

{
  "models": [
    {
      "title": "Qwen2.5-Coder-32B (Local)",
      "provider": "openai",
      "model": "Qwen/Qwen2.5-Coder-32B-Instruct",
      "apiBase": "http://localhost:8000/v1",
      "apiKey": "not-needed"
    }
  ]
}

After both are running, open VS Code, trigger Continue.dev (Ctrl+L or Cmd+L), and start coding with your local model. Completions and chat are served from your GPU with zero cloud dependency.

Exercise 1: Benchmark Comparison

Pick two coding models from different families (for example, Qwen2.5-Coder-32B and DeepSeek-Coder-V2). Compare their HumanEval, SWE-bench, and Aider Polyglot scores. Which model wins at each benchmark? What does this tell you about their strengths?

Exercise 2: Tool Call Format Matching

For a model you want to serve with vLLM, identify the correct --tool-call-parser flag. Check the model’s documentation or chat template to determine which format it was trained on.

Exercise 3: Model Selection Decision

You have an RTX 4090 (24 GB VRAM) and need a coding assistant for daily Python development. Using the model selection framework from Section 7, determine which model and quantization level gives you the best quality within your VRAM budget.

Lesson Summary

Coding LLMs are not just general models that happen to write code — the best ones are purpose-built specialists trained on curated code corpora and evaluated against programming-specific benchmarks. The agentic revolution adds tool calling, multi-step reasoning, and IDE integration to turn these models into genuine development partners.

• Benchmarks (HumanEval, SWE-bench, Aider) measure different coding capabilities
• Coding model families offer specialization advantages at every parameter count
• Agentic workflows use tool calling to read, write, test, and iterate on code
• Tool call parser matching is critical for reliable function calling
• Alignment affects what coding tasks a model will and will not perform
• Model selection is task-dependent, hardware-constrained, and framework-driven
• Self-hosted coding assistants offer privacy, cost, and customization advantages

Further Reading

• Aider LLM Leaderboards — Live benchmark tracking
• SWE-bench — Real-world coding evaluation
• vLLM Documentation — Tool calling and parser configuration
• Continue.dev Documentation — IDE integration setup