Integration

LangSmith integration

LangSmith tracing and evaluation API. Query the runs an LLM application produced with filters and full text search, read one run in detail, compute latency, token and cost statistics over a project, manage tracing projects, build datasets and examples for evaluation, and write or read feedback used as evaluation scores. Naming trap worth stating up front: a tracing project is called a session throughout this API, so the project identifier is named session_id and the project resource lives under sessions.

Connect LangSmith How integrations work

Ask it in chat

Say what should happen.
The workflow builds itself.

When this happens, do that in LangSmith.Built from the 19 LangSmith tools below, on the connection you add once.

Common LangSmith use cases

exporting a dataset for an offline run
exporting every human review of a project
validating an API key before a batch export
finding the dataset id before adding examples
reading the run count and cost of one project
cleaning up projects created by a test harness
removing an example that turned out to be wrong
pulling every failed run of a project for triage
creating a regression set before wiring an evaluator
removing a dataset created by a throwaway experiment

What you can do with LangSmith

Each of these is one operation on one endpoint, available both as a workflow step and as a tool an AI agent can call on its own. Nothing bundles several operations into one action, so what a run did stays readable afterwards.

Most endpoints run on the free plan. A few, mainly the ones that publish on your behalf, need a paid plan.

Example, 5 operations

create_exampleCreate examplePOSTAdds an example to a dataset, optionally copied from a production run.
delete_exampleDelete exampleDELETEDeletes one example from its dataset.
get_exampleGet exampleGETReturns one example with its inputs, reference outputs and metadata.
list_examplesList examplesGETLists the examples of a dataset, optionally as of a past version.
update_exampleUpdate examplePATCHEdits the inputs, outputs, metadata or split of an example and returns it as it now stands.

Dataset, 4 operations

create_datasetCreate datasetPOSTCreates a dataset that examples and experiments attach to.
delete_datasetDelete datasetDELETEDeletes a dataset together with its examples.
get_datasetGet datasetGETReturns a dataset with its schemas, example count and experiment count.
list_datasetsList datasetsGETLists evaluation datasets with their example counts.

Project, 4 operations

create_projectCreate projectPOSTCreates a tracing project that runs can then be logged into.
delete_projectDelete projectDELETEDeletes a tracing project and the runs logged into it.
get_projectGet projectGETReturns one tracing project, optionally with its run statistics.
list_projectsList projectsGETLists tracing projects, which this API calls sessions, with optional statistics.

Feedback, 2 operations

create_feedbackCreate feedbackPOSTAttaches a score, value, comment or correction inside a tracing project, optionally aimed at one run.
list_feedbackList feedbackGETLists feedback attached to runs or projects, filtered by key, source or score.

Run, 2 operations

get_runGet runGETReturns a single run with its inputs, outputs, token usage and cost.
query_runsQuery runsPOSTQueries the runs of one or more tracing projects with filters, search and cursor paging.

Info, 1 operation

get_infoGet infoGETReturns the LangSmith version and the ingestion limits of the deployment.

Run stat, 1 operation

get_run_statsGet run statsPOSTReturns run counts, latency and token percentiles, error rate and cost for a filtered set of runs.

Use LangSmith from Claude Code, Cursor or Codex

Connect LangSmith in LiveContext, create an API key in Settings, then add the LiveContext MCP server to your client. Its LangSmith operations become tools the client can call, next to every other integration you connected.

Claude Code

claude mcp add --transport http livecontext https://livecontext.ai/mcp --header "X-API-Key: YOUR_API_KEY"

Codex (~/.codex/config.toml)

[mcp_servers.livecontext]
url = "https://livecontext.ai/mcp"

[mcp_servers.livecontext.http_headers]
"X-API-Key" = "YOUR_API_KEY"

Cursor, Claude Desktop and other JSON configs

{
  "mcpServers": {
    "livecontext": {
      "type": "http",
      "url": "https://livecontext.ai/mcp",
      "headers": {
        "X-API-Key": "YOUR_API_KEY"
      }
    }
  }
}

Set up the MCP server step by step

How to connect LangSmith

Create your workspace
Sign up free. Nothing to install.
Connect LangSmith once
Add your LangSmith connection in Settings. Every tool from this integration reuses it, and secrets are encrypted at rest and visible only to you.
Describe the job in chat
The workflow builds itself in front of you and picks the endpoints it needs. Drag the steps yourself on the canvas whenever you would rather be explicit.

LangSmith integration FAQ

What can you do with the LangSmith integration?

LiveContext exposes 19 operations on LangSmith, covering example, dataset, project and feedback. Common uses include exporting a dataset for an offline run, exporting every human review of a project and validating an API key before a batch export.