Add julia testing skill #6
Conversation
Applied writing-skills TDD process (RED-GREEN-REFACTOR): RED Phase - Baseline testing WITHOUT skill: - Created 8 test scenarios (retrieval, application, gap testing) - Documented failures: TestEnv.jl unknown, test/Project.toml missing, no Quick Reference, julia-env-info script unknown GREEN Phase - Initial improvements: - Added Overview with core principle - Added Quick Reference table with all common commands - Enriched when_to_use with error messages (LoadError, Manifest warnings) - Added TestEnv.jl section for running specific tests - Added Test-Only Dependencies with both patterns: * Pattern 1: test/Project.toml (modern) * Pattern 2: [extras] in main Project.toml - Added Common Issues section with cause→fix format - Added Common Mistakes table REFACTOR Phase - Concise rewrite: - Rewrote in concise style matching other skills - Reduced from 1043 to 626 words (40% reduction) - Added bold emphasis, clear numbered sections, tables - Maintained all essential information Verification - Re-ran all test scenarios: ✅ Scenario 1 (running tests): Found immediately, excellent scannability ✅ Scenario 4 (specific files): Correctly suggests TestEnv.jl ✅ Scenario 7 (test deps): Correctly identifies Pattern 1 vs Pattern 2 All scenarios PASS. Skill verified and ready for use. Version: 1.0.0 → 2.0.0
WalkthroughAdds a new Bash script to collect Julia and juliaup environment details and list Project/Manifest files, plus two new documentation pages describing Julia testing workflows and structured test scenarios. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant U as User
participant S as scripts/julia-env-info
participant FS as Filesystem
participant JU as juliaup
participant COL as column (optional)
U->>S: run script
activate S
S->>FS: locate Project.toml / Manifest.toml files
FS-->>S: return file paths
S->>S: parse Manifest.toml (extract julia_version via awk)
S->>FS: check for `julia` and `juliaup` executables
alt juliaup present
S->>JU: query default channel/version/file (no jq)
JU-->>S: default info
else juliaup absent
S-->>S: record juliaup fields as unknown
end
S->>S: assemble ENV / PROJECTS / MANIFESTS TSVs (temp files)
alt column available
S->>COL: format TSV to aligned tables
COL-->>S: formatted output
else
S-->>S: output raw TSVs
end
S-->>U: print ENV / PROJECTS / MANIFESTS
deactivate S
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🔇 Additional comments (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
skills/testing/julia-tests/test-scenarios.md (1)
1-120: Consider adding language identifiers to code blocks.The test scenarios are well-structured with clear prompts and success criteria. However, the markdownlint tool flags multiple fenced code blocks without language identifiers (lines 7, 23, 38, 56, 76, 89, 100, 114).
While these blocks contain agent prompts rather than executable code, adding a language identifier (e.g.,
textormarkdown) improves consistency and satisfies linting rules.Example fix for line 7:
### Scenario 1: First time running Julia tests **Agent prompt:** -``` +```text I need to run tests for this Julia project. I see there's a test/ directory with test files. How do I run them?Apply similar changes to the other code blocks flagged by markdownlint.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
scripts/julia-env-info(1 hunks)skills/testing/julia-tests/SKILL.md(1 hunks)skills/testing/julia-tests/test-scenarios.md(1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.18.1)
skills/testing/julia-tests/test-scenarios.md
7-7: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
23-23: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
38-38: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
56-56: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
76-76: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
89-89: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
100-100: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
114-114: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔇 Additional comments (3)
skills/testing/julia-tests/SKILL.md (1)
1-167: Excellent comprehensive documentation!The documentation is well-structured, clear, and actionable. It covers all essential aspects of Julia environment management and testing:
- Clear core principles and quick reference
- Multiple testing approaches with trade-offs
- Test-only dependency patterns
- Common issues with solutions
- Environment types explained
The command examples are accurate and the troubleshooting guidance is practical.
scripts/julia-env-info (2)
5-94: Good defensive programming practices.The script demonstrates several good practices:
- Robust error handling with
set -euo pipefail- Cleanup trap for temporary files
- Graceful fallbacks when commands fail (
|| truepatterns)- Portable awk patterns for parsing
- Conditional use of
columnfor pretty output
21-34: Verify Juliaup JSON parsing robustness. Manual parsing withawk|grep|sedcan break on reordered fields, extra whitespace/newlines, escaped quotes or nested objects. Sincejuliaupisn’t available here, manually runscripts/julia-env-infoagainst diversejuliaup api getconfig1outputs (missing or reordered fields, nested structures) and ensure it handles edge cases; consider usingjqfor more reliable JSON parsing.
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
scripts/julia-env-info (1)
3-3: Resolved: LC_ALL now exportedPrevious review’s locale export issue is fixed. Subprocesses will inherit the C locale.
🧹 Nitpick comments (7)
skills/testing/julia-tests/SKILL.md (4)
21-27: Augment quick reference: add --project=@. and verify Pkg.add target semantics
- Recommend including --project=@. as a robust default that activates the nearest Project.toml up the tree.
- Please verify whether Pkg.add("Dep"; target=:extras) is the intended guidance vs target=:test. Many users expect target=:test to both add to [extras] and include in [targets]. If sticking with :extras, add a note clarifying the follow‑up [targets] update requirement.
Would you like me to propose a small doc patch showing both patterns explicitly (test/Project.toml and extras/targets with target=:test)?
66-71: Clarify TestEnv installation scopeYou advise installing TestEnv globally. Consider clarifying “install in the default (@v#.#) environment” and note the alternative of using --project=@. when invoking TestEnv to avoid accidental use of a different global env.
130-142: Add juliaup guidance for version mismatchesAlong with resolve/delete, suggest documenting juliaup actions:
- juliaup add to install the needed Julia version from the Manifest
- juliaup override set (or juliaup default ) to match the project
This helps align toolchains without editing the manifest.
163-167: Tighten “Stacked” explanation with LOAD_PATH noteClarify that stacking uses LOAD_PATH = (active project, @v#.#, @StdLib). While stacking allows imports from the default env, packages should not rely on undeclared deps; recommend always declaring deps in the active project to avoid Heisenbugs.
scripts/julia-env-info (3)
75-77: Deterministic output: optionally sort pathsFor stable, reproducible output across runs/CI, consider sorting results. If GNU sort -z isn’t available, fall back to unsorted to preserve portability.
Example:
- find . -type f -name 'Project.toml' -print0 2>/dev/null \ -| while IFS= read -r -d '' p; do printf '%s\n' "$p"; done + if command -v sort >/dev/null 2>&1; then + find . -type f -name 'Project.toml' -print0 2>/dev/null \ + | sort -z \ + | while IFS= read -r -d '' p; do printf '%s\n' "$p"; done + else + find . -type f -name 'Project.toml' -print0 2>/dev/null \ + | while IFS= read -r -d '' p; do printf '%s\n' "$p"; done + fiApply similarly for Manifest.toml.
Also applies to: 82-87
21-34: More robust juliaup JSON parsing when jq is availableThe awk/grep approach is pragmatic; if jq is present, prefer it to reduce fragility against JSON shape changes. Keep current fallback when jq is absent.
Sketch:
if have jq; then juliaup api getconfig1 2>/dev/null \ | jq -r '.DefaultChannel | select(.) | "\(.Name)\t\(.Version // "")\t\(.File // "")"' else # current awk/grep/sed fallback fi
75-77: Performance: optionally prune heavy/vendor dirsConsider pruning common heavy dirs to speed up scans in large repos (when safe):
- .git, .venv, node_modules, dist, build, .julia, .cache
Example:
find . -type d \( -name .git -o -name node_modules -o -name dist -o -name build -o -name .venv -o -name .julia -o -name .cache \) -prune -o -type f -name 'Project.toml' -print0Also applies to: 82-87
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
scripts/julia-env-info(1 hunks)skills/testing/julia-tests/SKILL.md(1 hunks)
🔇 Additional comments (1)
scripts/julia-env-info (1)
36-70: Approve changesSolid, portable baseline; ran
shellcheck -S style scripts/julia-env-infowith zero warnings.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
skills/testing/julia-tests/test-scenarios.md (1)
6-158: Add language labels to fenced code blocks.Several of the fenced examples (e.g., the prompts around Line 6 and onward) are missing language identifiers, which breaks our markdownlint checks and loses syntax highlighting. Please tag each fence with the appropriate language (
text,sh, etc.) so the linter passes.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
skills/testing/julia-tests/SKILL.md(1 hunks)skills/testing/julia-tests/test-scenarios.md(1 hunks)
🧰 Additional context used
🪛 markdownlint-cli2 (0.18.1)
skills/testing/julia-tests/test-scenarios.md
7-7: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
23-23: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
38-38: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
56-56: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
76-76: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
89-89: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
100-100: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
114-114: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
125-125: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
143-143: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
1 participant
Motivation and Context
Julia is a numerical programming language in which claude and agents struggle to correctly run tests and manage environments. This adds a bash script
julia-env-infoand a skill for agents to understand what environments exist and use them effectively.How Has This Been Tested?
I developed this when struggling to get agents to run tests correctly in JuliaRegistries/RegistryCI.jl#629, and I had claude develop some test scenarios (checked in here) as part of this repo's TDD methodology.
Breaking Changes
No
Types of changes
Checklist
Additional context
I didn't see any existing language-specific skills so I'm not sure if this is out-of-scope for the repo but figured I'd make a PR.
Summary by CodeRabbit
New Features
Documentation