The Evaluation Framework Behind Infinize Agents:
From New Agent to Trusted Agent

An agent network only works when each agent owns a distinct kind of help.

“Fini” is the familiar front door, where Maya can ask the broad, messy questions students actually bring. “Universal Assistant” keeps that conversation connected to campus knowledge, services, and task routes without forcing every request into the same answer shape.

When Maya is still choosing a path, “Major Exploration” compares interests, program fit, cost, and outcomes. “Enrollment Companion” turns that exploration into admissions follow-up, reminders, summaries, and staff context when the next step needs a human.

Once she is enrolled, “Course Assistant” stays grounded in course material, not general campus knowledge. If friction appears, “Alerts & Nudges” catches the signal and “Academic Planning” keeps requirements, holds, credits, and term choices understandable.

By senior year, “Career Recommendations” connects Maya's courses and skills to roles she can pursue, while “Resume Builder” turns that record into language she can use. The evaluation problem is making sure each handoff preserves the right evidence, role, and boundary.

Infinize evaluates the agent network at two levels: a common operational standard for tracing, testing, review, and promotion, plus workflow-specific rubrics for each agent. Resume Builder, Course Assistant, Enrollment Companion, and Major Exploration all need different checks because they do different work.

A final response is the most visible part of an agent run, so it is tempting to evaluate only that. Read the answer, decide whether it is helpful, and move on. That works poorly for higher education because the most important decisions often happen before the answer is written.

Suppose Maya asks whether an assignment is due this Friday. A good answer depends on the active course, the current LMS state, the right student identity, the right read-only boundary, and the agent's ability to say when it does not have enough evidence. The wording can be friendly and still fail the workflow. It can cite a date from the wrong course. It can blend an old module with a current one. It can answer from memory when it should have used a tool.

The same pattern appears in staff workflows. Fini can produce a tidy summary while omitting the signal that matters. Enrollment Companion can route an inquiry to the wrong operational path. Course Assistant can draft feedback that sounds reasonable but does not follow the instructor's rubric. Career Recommendations can suggest a role that does not fit the student's actual record.

Infinize evaluates the full run behind the prose. The trace shows which tool was called, which source was retrieved, which route was chosen, which agent or sub-agent handled the request, where the model was involved, and what evidence was available when the answer was produced.

The rubric then asks whether those choices were acceptable for the workflow. If the agent had no evidence, did it say so clearly? If the user was not authorized, did the boundary hold before sensitive context reached the answer? If the task required a specialist, did the routing layer hand off correctly? If the output was an artifact, did it follow the expected format?

A polished answer can hide a weak run. Evaluation brings the run back into view.

Infinize evaluates agents through a mix of offline evals, online evals, rubrics, traces, experiments, and human review. Each layer answers a different question.

Offline evals are controlled scenarios. They run before broader usage and give the team a repeatable way to test known risks. The same input can be used against a new model, a revised prompt, a new retrieval path, a changed tool, or a newly onboarded agent. If the old behavior was safe and useful, the new behavior has to prove it still is.

Online evals are the learning loop from real usage. This pipeline is currently in progress, and it is intentionally connected to review rather than treated as passive analytics. Real interactions can reveal a question pattern, source gap, routing issue, or confusing edge case that should become tomorrow's offline scenario.

Rubrics are the vocabulary of quality. They make a product expectation reviewable. "Grounded" becomes a check against approved sources. "Safe" becomes a check against role boundaries and action limits. "Useful" becomes a check against whether the answer actually helps the student or staff member move forward. "Good artifact" becomes a check against the expected structure of a quiz, flashcard set, rubric draft, resume section, or advisor brief.

These dimensions are applied through scenario-scoped evaluators. A public knowledge question, a protected knowledge question, a denial case, an LMS read-only case, a course artifact case, and a staff summary case each need the evaluator set that fits the workflow.

For a course due-date case, the eval might check active-course scope, current LMS evidence, no cross-course leakage, and read-only behavior. For a Course Assistant explanation, the eval might check whether the answer is grounded in selected materials and does not invent outside course content. For a generated quiz, the eval might check question coverage, answer quality, difficulty fit, and whether the artifact follows the expected structure. For an Enrollment Companion summary, the eval might check completeness, sensitivity, trace evidence, and whether the recommendation belongs in a human review path.

Some of these checks are deterministic. Did the tool call happen? Was a forbidden fact present? Did the output include the required fields? Did the trace link to the dataset row? Other checks need a semantic judge: relevance, completeness, quality of explanation, usefulness, or hallucination control. The evaluation engine supports both, because agent quality is not one kind of signal.

Good evaluation asks whether the agent behaved correctly in the exact situation it was trusted to handle.

Infinize built the infinize-agent-evals skill for the evaluation engine so developers can onboard agents into a shared lifecycle instead of recreating tracing, datasets, auth assumptions, evaluator selection, and trace linkage one agent at a time.

The lifecycle is intentionally simple to describe: emit, become eval-able, become evaluated. Under that simple shape is the discipline that makes the system scale across Fini, Universal Assistant, Course Assistant, Enrollment Companion, Alerts & Nudges, Academic Planning, Career Recommendations, Resume Builder, and future product agents.

First, the agent emits traces. Different agents may run through web services, background jobs, serverless handlers, task runners, or orchestrated sub-agents. Each agent can keep its runtime shape while becoming observable in the same shared review workspace with the right service, environment, model, token, cost, and span context where those signals apply.

Next, the agent becomes eval-able. It receives a stable identity, an invocation contract, an authentication shape, fixtures, datasets, and evaluator definitions. Eval labels connect a dataset case to the trace produced by that case, so a reviewer can move from score to evidence without guessing which run created the result.

Then the agent becomes evaluated. The framework drives the agent through the dataset, scores each run, publishes the experiment, and links annotations back to trace evidence. A reviewer can inspect what was asked, what was retrieved, which agent handled it, which rubric applied, what the evaluator saw, and where a human should review.

The onboarding discipline also prevents evaluation drift. Every new agent must define its capability boundary, request shape, auth expectations, data visibility, trace contract, dataset rows, evaluator set, local dependencies, and handoff notes. A running agent becomes production-shaped when its behavior can be replayed, scored, reviewed, and explained.

The technical advantage is scale with specificity. Any agent can enter a common lifecycle while keeping its own workflow-specific rubric. The skill gives developers the onboarding path, product teams the review surface, and stakeholders the evidence trail.

The next agent Infinize adds will arrive as a capability with a boundary.

Before that agent earns more responsibility, the team needs to know what it answered, what it refused, what it retrieved, what it produced, and what a reviewer can verify. Promotion is where evaluation becomes operational.

The promotion pipeline, currently in progress, brings offline evals, trace review, rubric results, human checks, and online learning into the release path. An agent can start with a narrow capability, pass known scenarios, collect review evidence, expand to a controlled surface, and then move toward broader responsibility.

Tracing is what makes this promotion path explainable. If an eval fails, the team can inspect the case and the trace together. If an online pattern appears repeatedly, the team can turn it into a new offline row. If a reviewer disagrees with an evaluator, the rubric can improve. If a model or tool changes, the same scenario can be rerun. The evaluation engine becomes a durable memory for the platform.

This matters because campuses keep changing. Policies change. Course materials change. Catalog pages change. LMS data changes. Student contexts change. Staff workflows change. New agents will arrive as Infinize expands into more product surfaces. A shared evaluation lifecycle gives those agents the same operating model: define the boundary, run the scenarios, inspect the trace, review the rubric, promote carefully, and learn from usage.

When Maya's university adds the next agent, it should not feel like another tool dropped into her path. It should feel like the same journey extending one step further, across every student, staff, and institutional workflow Infinize supports.

Evaluations become part of how Infinize builds and scales that agent network, so every new agent has a reason to belong, a boundary to respect, and evidence behind the help it gives.