Integrate Lighthouse CI #2079
Integrate Lighthouse CI #2079
Conversation
Summary by CodeRabbit
WalkthroughAdds Lighthouse CI configuration, a Makefile target, and .gitignore entry; inserts two identical Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changes
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
frontend/lighthouserc.json (1)
1-19: Stabilize runs and harden CI environment; consider gating severityCurrent config is OK to start. For more stable and CI-friendly audits:
- Use multiple runs (e.g., 3) to reduce flakiness, with the median result.
- Add chrome flags for GitHub-hosted runners.
- Optionally promote assertions from warn to error once baselines are stable, to gate regressions per the issue goals.
Proposed changes:
{ "ci": { "collect": { - "url": ["https://nest.owasp.dev"], - "numberOfRuns": 1 + "url": ["https://nest.owasp.dev"], + "numberOfRuns": 3, + "settings": { + "chromeFlags": "--no-sandbox --disable-dev-shm-usage" + } }, "upload": { "target": "temporary-public-storage" }, "assert": { "assertions": { - "categories:performance": ["warn", { "minScore": 0.9 }], - "categories:accessibility": ["warn", { "minScore": 0.9 }], - "categories:best-practices": ["warn", { "minScore": 0.9 }], - "categories:seo": ["warn", { "minScore": 0.9 }] + "categories:performance": ["warn", { "minScore": 0.9 }], + "categories:accessibility": ["warn", { "minScore": 0.9 }], + "categories:best-practices": ["warn", { "minScore": 0.9 }], + "categories:seo": ["warn", { "minScore": 0.9 }] } } } }Temporary public storage exposes a public link to results. Confirm this is acceptable for your data-sharing policy. If not, we can switch to a self-hosted LHCI server or artifact-only approach.
.github/workflows/run-ci-cd.yaml (2)
486-516: Streamline LHCI job: drop unnecessary installs, use npx, and add a timeoutInstalling pnpm + project dependencies isn’t needed to run LHCI against a remote URL; it adds time and cache noise. Use npx for the CLI and set a job timeout.
lighthouse_ci: name: Run Lighthouse CI needs: - deploy-staging-nest-proxy - runs-on: ubuntu-latest + runs-on: ubuntu-latest + timeout-minutes: 15 steps: - name: Check out repository uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 - - name: Install pnpm - uses: pnpm/action-setup@a7487c7e89a18df4991f7f222e4898a00d66ddda - with: - version: 10 - run_install: true - - name: Set up Node uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 with: node-version: 22 - cache: 'pnpm' - cache-dependency-path: frontend/pnpm-lock.yaml - - - name: run Lighthouse CI + - name: Run Lighthouse CI working-directory: frontend run: | - pnpm install -g @lhci/[email protected] - lhci autorun + npx -y @lhci/[email protected] autorun env: LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
509-516: PR status checks may not appear: job only runs after main-branch staging deployBecause this job depends on deploy-staging-nest-proxy (which runs only on main), it won’t execute on pull_request events, so the Lighthouse GitHub App may not post checks on PRs. If PR checks are desired, add a separate LHCI job for pull_request that targets a preview URL (if available) or the staging site with clear caveats.
I can sketch an additional PR-scoped job using the same steps, gated to pull_request, and optionally parameterize the target URL via an input or environment.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (3)
.github/workflows/run-ci-cd.yaml(1 hunks).gitignore(1 hunks)frontend/lighthouserc.json(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run frontend unit tests
- GitHub Check: Run backend tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (2)
.gitignore (1)
10-10: LGTM: Ignore rule correctly captures LHCI artifacts anywhere in the repo
.lighthouseci/will ignore the directory at any depth (e.g., frontend/.lighthouseci/), which aligns with running LHCI from the frontend working directory..github/workflows/run-ci-cd.yaml (1)
486-516: No duplicate lighthouse_ci job foundI searched
.github/workflows/run-ci-cd.yamland confirmed only onelighthouse_cijob definition at line 486. The PR summary’s mention of a duplicate appears to be incorrect—no further action needed here.Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
frontend/lighthouserc.json (3)
3-6: Reduce CI flakiness by adding Chrome flags.GitHub-hosted runners can occasionally hit Chrome sandbox/SHM limits. Add no-sandbox/dev-shm flags to stabilize runs.
Apply this diff:
"ci": { "collect": { "url": ["https://nest.owasp.dev/"], - "numberOfRuns": 3 + "numberOfRuns": 3, + "chromeFlags": "--no-sandbox --disable-dev-shm-usage" },
10-16: Expand assertions to cover PWA and Core Web Vitals explicitly.Issue #2037 mentions tracking PWA. Also, adding explicit metric-level assertions helps catch regressions earlier than category scores alone.
Apply this diff to extend the assertions:
"assert": { "assertions": { "categories:performance": ["warn", { "minScore": 0.9 }], "categories:accessibility": ["warn", { "minScore": 0.9 }], "categories:best-practices": ["warn", { "minScore": 0.9 }], - "categories:seo": ["warn", { "minScore": 0.9 }] + "categories:seo": ["warn", { "minScore": 0.9 }], + "categories:pwa": ["warn", { "minScore": 0.5 }], + "first-contentful-paint": ["warn", { "maxNumericValue": 2500, "aggregationMethod": "median" }], + "largest-contentful-paint": ["warn", { "maxNumericValue": 4000, "aggregationMethod": "median" }], + "total-blocking-time": ["warn", { "maxNumericValue": 300, "aggregationMethod": "median" }], + "cumulative-layout-shift": ["warn", { "maxNumericValue": 0.1, "aggregationMethod": "median" }] } }Notes:
- Thresholds are intentionally moderate to reduce noise; tune over time based on baseline results.
8-9: Be aware: temporary-public-storage is public and ephemeral.This is fine to start, but data expires and is public. For long-term trend visibility, consider a self-hosted LHCI server later, or rely on the GitHub App’s status checks for gating.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
.github/workflows/run-ci-cd.yaml(1 hunks)frontend/lighthouserc.json(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/run-ci-cd.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
- GitHub Check: Run backend tests
- GitHub Check: Run frontend unit tests
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run Lighthouse CI
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (2)
frontend/lighthouserc.json (2)
1-19: Solid baseline LHCI config aligned with the PR objectives.
- Targets the deployed site, runs multiple passes, and sets category thresholds (warn @ 0.9). This is a good starting point and low-risk to land.
4-6: No Lighthouse CI workflow detected—ignore gating suggestionI reviewed the
.github/workflowsdirectory and found nolighthouse_ciorlhcijobs to gate against deployment. You can safely disregard the original recommendation about ordering a non-existent LHCI job after your deploy step.Likely an incorrect or invalid review comment.
|
PR is ready for review, I'm not exactly sure what asserts to add. For now, I have added a warning for anything less than |
|
Okay, I just looked into CR's nitpick about adding Removing the sandbox sounds like a security issue but Github actions themselves are sandboxed so it shouldn't be an issue. I will wait for a review, will add if required. |
|
@arkid15r Yes, we can use it without the installed app, the report's url will be available via GitHub actions pipeline. |
|
@arkid15r |
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
.github/workflows/run-ci-cd.yaml (1)
486-504: Remove duplicate job definition.There appears to be a duplicate
lighthouse_cijob definition in the workflow. The AI summary indicates "inserts two identicallighthouse_cijob blocks," but only one is visible in the provided code. This suggests there may be another identical job definition elsewhere in the file that should be removed to avoid workflow conflicts.Please search for and remove any duplicate
lighthouse_cijob definitions:#!/bin/bash # Description: Search for duplicate lighthouse_ci job definitions in the workflow file. rg -n "lighthouse_ci:" .github/workflows/run-ci-cd.yaml
🧹 Nitpick comments (1)
.github/workflows/run-ci-cd.yaml (1)
501-504: Consider adding Chrome flags for CI environment.While the current implementation should work, you might want to add Chrome flags to handle potential sandbox issues in CI environments. The
--no-sandboxand--disable-dev-shm-usageflags are commonly used in containerized environments.If you encounter issues with Chrome in the CI environment, consider adding the following Chrome flags by creating a
LHCI_CHROME_FLAGSenvironment variable:- name: Run Lighthouse CI working-directory: frontend + env: + LHCI_CHROME_FLAGS: "--no-sandbox --disable-dev-shm-usage" run: | npx -y @lhci/[email protected] autorun
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
.github/workflows/run-ci-cd.yaml(1 hunks)
🔇 Additional comments (3)
.github/workflows/run-ci-cd.yaml (3)
486-490: LGTM! Proper job dependency structure.The job correctly depends on
deploy-staging-nest-proxy, ensuring Lighthouse CI runs after the staging deployment is complete. This aligns with the PR objective to run audits after each deployment.
496-500: LGTM! Proper Node.js setup for Lighthouse CI.The Node.js setup matches the version used in other frontend jobs (Node 22) and uses the same action version for consistency.
504-504: Lighthouse CI configuration file verified and properly configured.The verification confirms that the Lighthouse CI configuration file (
frontend/lighthouserc.json) exists and contains appropriate settings:
- URL Configuration: Points to the production site
https://nest.owasp.dev/- Test Runs: Configured for 3 runs to ensure reliable results
- Storage: Uses temporary public storage for results
- Assertions: Properly configured with warning thresholds (0.9 minimum score) for all key categories:
- Performance
- Accessibility
- Best Practices
- SEO
The
npx -y @lhci/[email protected] autoruncommand will successfully locate and use this configuration file.
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
frontend/Makefile (3)
30-32: Use pnpm dlx and avoid hardcoding the URL; let lighthouserc.json drive itKeeps tooling consistent (pnpm is used elsewhere), benefits from pnpm’s cache, and avoids overriding the repo config unintentionally. Allow overriding via a Make variable when needed.
Apply this diff:
-lighthouse-ci: - @cd frontend && npx -y @lhci/[email protected] autorun --collect.url=http://localhost:3000 +lighthouse-ci: + @cd frontend && pnpm dlx @lhci/[email protected] autorun $(LHCI_FLAGS)Add these (outside the selected lines) near the top of the file to make flags optional and explicit:
# Optional flags for LHCI (e.g., --collect.url=http://localhost:3000) LHCI_FLAGS ?=Example local run pointing to localhost only when desired:
make lighthouse-ci LHCI_FLAGS="--collect.url=http://localhost:3000"
30-32: Pin LHCI CLI version via a Make variable to prevent drift across CI and local runsCentralizing the version makes it trivial to keep the workflow and Makefile aligned.
Apply this diff:
-lighthouse-ci: - @cd frontend && pnpm dlx @lhci/[email protected] autorun $(LHCI_FLAGS) +lighthouse-ci: + @cd frontend && pnpm dlx @lhci/cli@$(LHCI_CLI_VERSION) autorun $(LHCI_FLAGS)Add this (outside the selected lines):
# LHCI CLI version (keep in sync with CI workflow) LHCI_CLI_VERSION ?= 0.15.1Optionally, if you prefer to override ad-hoc:
make lighthouse-ci LHCI_CLI_VERSION=0.15.2
30-32: Mark target as .PHONYPrevents Make from mistaking a file named lighthouse-ci for a successful build artifact.
Add this (outside the selected lines):
.PHONY: lighthouse-ci
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (4)
.github/workflows/run-ci-cd.yaml(1 hunks)cspell/custom-dict.txt(1 hunks)frontend/Makefile(1 hunks)frontend/lighthouserc.json(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- cspell/custom-dict.txt
🚧 Files skipped from review as they are similar to previous changes (2)
- frontend/lighthouserc.json
- .github/workflows/run-ci-cd.yaml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Run backend tests
- GitHub Check: Run frontend e2e tests
- GitHub Check: Run frontend unit tests
- GitHub Check: CodeQL (javascript-typescript)
🔇 Additional comments (1)
frontend/Makefile (1)
30-32: No version discrepancies found for @lhci/cliRipgrep shows only a single pin of
@lhci/[email protected]infrontend/Makefile. Version consistency across the repo is confirmed—no further changes needed.
* Integrate lighthouseci * use nest.owasp.dev * run lighthouse after staging deployment * Update code * Update code temporarily * run lighthouse after staging deployment * remove github app environment variable and test * run lighthouse after staging deployment * Update code --------- Co-authored-by: Arkadii Yakovets <[email protected]>
Proposed change
Resolves #2037
Integrate Lighthouse CI
Hi we can add Lighthouse CI GitHub App to our repository to add Github status checks.
Please add an environment variable
LHCI_GITHUB_APP_TOKENas well More info herePlease also change the asserts in
lighthouserc.jsonas requiredChecklist
make check-testlocally; all checks and tests passed.