ai/agentic-ai-test-suite
Watch
1
Code Issues Pull requests Projects Releases Packages Activity Actions

chore: use javascript ast for evaluation #4

Merged
SethFalco merged 1 commit from js-ast into main 2026-02-21 22:49:32 +00:00
Conversation 1 Commits 1 Files changed 3 +165 -18
SethFalco commented 2026-02-21 22:49:26 +00:00 (Migrated from gitlab.com)
Copy link

Follow-up to https://gitlab.com/SethFalco/agentic-ai-test-suite/-/merge_requests/3.

I originally used prompt/context to enforce a line break between a JSDoc summary and block tag. This adds eslint-plugin-jsdoc and enforces this through ESLint instead. Later, I'll investigate a way to make Claude loop through and resolve ESLint errors as part of its evaluation.

Future

This means that code formatting will be part of the evaluation, but will not be done through being instructed in prompts, but rather by having it loop over ESLint errors post assignment. The final work must pass lint checks first, and only then it will be evaluated via node:test.

Follow-up to https://gitlab.com/SethFalco/agentic-ai-test-suite/-/merge_requests/3. I originally used prompt/context to enforce a line break between a JSDoc summary and block tag. This adds `eslint-plugin-jsdoc` and enforces this through ESLint instead. Later, I'll investigate a way to make Claude loop through and resolve ESLint errors as part of its evaluation. ## Future This means that code formatting will be part of the evaluation, but will not be done through being instructed in prompts, but rather by having it loop over ESLint errors post assignment. The final work must pass lint checks first, and only then it will be evaluated via `node:test`.
SethFalco commented 2026-02-21 22:49:32 +00:00 (Migrated from gitlab.com)
Copy link

mentioned in commit a21abf2efa

mentioned in commit a21abf2efa434e6468f64c820b433f25f2336e35
SethFalco (Migrated from gitlab.com) merged commit a21abf2efa into main 2026-02-21 22:49:32 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ai/agentic-ai-test-suite!4
No description provided.