Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion cli/translations/en.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,5 @@
"private_methods_count_option": "Private Methods Count",
"public_methods_count_option": "Public Methods Count",
"comment_ratio_option": "Comment to Code Ratio",
}
}

18 changes: 16 additions & 2 deletions docs/docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,16 @@ const config = {
// my site is not chinese. but thank you for the heads up
i18n: {
defaultLocale: 'en',
locales: ['en'],
},
locales: ['en', 'pt'],
// Optional: Add specific config for Portuguese if needed
localeConfigs: {
pt: {
htmlLang: 'pt-BR', // Or pt-PT
label: 'Portugues (Brasil)', // Label for the language dropdown
},
},
},


presets: [
[
Expand Down Expand Up @@ -88,6 +96,12 @@ const config = {
position: 'left',
label: 'Documentation',
},
{
type: 'localeDropdown',
sidebarId: 'docsSidebar', // Updated sidebar ID
position: 'left',

},
// {to: '/blog', label: 'Blog', position: 'left'}, // Blog disabled
{
href: 'https://github.com/spicecodecli/spicecode',
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules
.docusaurus
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
sidebar_position: 3
---

# API Reference: Analyzers Module (`spice/analyzers`)

The `spice/analyzers` module is where the core logic for extracting metrics and insights from the parsed source code resides. After the `lexers` module tokenizes the code and the `parser` module constructs an Abstract Syntax Tree (AST) or similar structured representation, the individual analyzer components within this module are invoked. Each analyzer is designed to traverse the AST (or work with the token stream directly for simpler metrics) and calculate specific characteristics of the code. They embody the specialized senses of SpiceCode, each focused on detecting a particular aspect of code quality, structure, or style.

## Purpose of Code Analysis

While the parser verifies the grammatical correctness of the code, the analyzers perform semantic analysis and metric calculation. They examine the structure represented by the AST to understand *what* the code does and *how* it is written. The goal is to produce objective measurements and qualitative assessments that help developers understand maintainability, complexity, adherence to conventions, and potential areas for improvement.

## Structure of the `spice/analyzers` Module

The `spice/analyzers` directory contains individual Python files, each typically implementing a specific analysis metric or a closely related group of metrics:

```
spice/
├── __init__.py
├── analyze.py # Main analysis orchestration logic?
└── analyzers/
├── __init__.py
├── count_comment_lines.py
├── count_comment_ratio.py
├── count_external_dependencies.py
├── count_functions.py
├── count_inline_comments.py
├── count_lines.py
├── count_method_type.py
└── indentation.py
```

### `analyze.py` (within `spice/`)

While not strictly inside the `analyzers` subdirectory, the `analyze.py` file likely plays a crucial role in orchestrating the analysis process. It might contain the main function or class that:

1. Takes the file path as input.
2. Invokes the appropriate lexer and parser based on the language.
3. Receives the resulting AST or token stream.
4. Instantiates and runs the relevant analyzers from the `spice/analyzers` directory, passing the AST/tokens to them.
5. Collects the results from each analyzer.
6. Formats and returns the final analysis report (either for display, JSON output, or export).

### Individual Analyzer Files (within `spice/analyzers/`)

Each `.py` file within the `spice/analyzers` subdirectory represents a distinct analysis capability. Examples include:

* **`count_lines.py`:** Implements the logic to count total, code, and blank lines, likely by processing the raw source text or token stream.
* **`count_comment_lines.py`, `count_inline_comments.py`, `count_comment_ratio.py`:** These implement the various comment-related metrics. They would typically traverse the AST or token stream, identifying comment nodes/tokens and comparing their counts/positions relative to code lines.
* **`count_functions.py`:** Traverses the AST to find and count nodes representing function or method definitions (e.g., `FunctionDefinition` nodes).
* **`count_method_type.py`:** Analyzes function/method definition nodes in the AST to categorize them based on language-specific features (e.g., class methods, static methods, constructors).
* **`count_external_dependencies.py`:** Scans the AST for import/require/use statements to identify and count external dependencies.
* **`indentation.py`:** Analyzes the token stream or AST to check for consistent use of whitespace for indentation, flagging inconsistencies based on language conventions.

Each analyzer file typically defines a function or class that takes the AST (or token stream/source text) as input and returns the calculated metric(s). They often rely on traversing the AST structure using patterns like the Visitor pattern to efficiently inspect relevant nodes.

## Usage within SpiceCode

When the `spice analyze` or `spice export` command is executed, the main analysis orchestrator (`spice/analyze.py`) identifies the language, parses the code, and then invokes the appropriate set of analyzers from this directory. If the user selected specific analyses interactively, only those corresponding analyzers are run. If `--all` is used or the command is `export`, all applicable analyzers for the language are executed.

The results from each analyzer are collected and aggregated into a final report, which is then presented to the user or saved to a file in the specified format.

This modular structure makes the analysis framework extensible. Adding a new analysis metric typically involves creating a new Python file within the `spice/analyzers` directory, implementing the logic to calculate the metric (usually by traversing the AST), and integrating it into the orchestration process in `spice/analyze.py`. This design promotes separation of concerns and makes it easier to maintain and enhance SpiceCode's analytical capabilities.
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
sidebar_position: 1
---

# API Reference: Lexers Module (`lexers`)

Central to SpiceCode's ability to understand source code across different languages is the `lexers` module. This package contains the fundamental components responsible for the first phase of code analysis: lexical analysis, or tokenization. Unlike tools that rely on external libraries or language-specific compilers, SpiceCode employs its own set of native lexers, meticulously crafted for each supported language. This approach ensures independence, consistency, and fine-grained control over how source code is interpreted at the most basic level, much like a Fremen relies on their own senses and knowledge rather than off-world instruments.

## Purpose of Lexical Analysis

Lexical analysis is the process of reading the sequence of characters that make up the source code and converting them into a sequence of meaningful symbols called tokens. These tokens represent the building blocks of the language, such as keywords (`def`, `class`, `func`), identifiers (variable names, function names), operators (`+`, `=`, `==`), literals (numbers, strings), punctuation (parentheses, commas), and comments. This stream of tokens is then passed to the parser for the next stage of analysis (syntactic analysis).

## Structure of the `lexers` Module

The `lexers` module is organized to support multiple languages in a modular fashion:

```
lexers/
├── __init__.py
├── token.py # Defines the base Token class
├── golang/
│ ├── __init__.py
│ └── golexer.py # Lexer implementation for Go
├── javascript/
│ ├── __init__.py
│ └── javascriptlexer.py # Lexer implementation for JavaScript
├── python/
│ ├── __init__.py
│ └── pythonlexer.py # Lexer implementation for Python
└── ruby/
├── __init__.py
└── rubylexer.py # Lexer implementation for Ruby
```

### `token.py`

This file defines the fundamental `Token` class (or a similar structure). Each token generated by a lexer is typically an instance of this class (or a named tuple/dataclass), containing information such as:

* **Type:** The category of the token (e.g., KEYWORD, IDENTIFIER, OPERATOR, STRING_LITERAL, COMMENT, WHITESPACE, EOF - End Of File).
* **Value:** The actual text sequence from the source code that constitutes the token (e.g., `"def"`, `"myVariable"`, `"+"`, `"# This is a comment"`).
* **Line Number:** The line in the source file where the token begins.
* **Column Number:** The column position on the line where the token begins.

This structured token information is essential for the parser and subsequent analyzers.

### Language-Specific Lexer Modules (`golang`, `javascript`, `python`, `ruby`)

Each subdirectory within `lexers` corresponds to a supported programming language and contains the specific lexer implementation for that language.

* **`golexer.py` (in `golang/`):** Implements the lexical analysis logic specific to the Go language syntax, handling its keywords, operators, comments (`//`, `/* */`), string literals, numeric literals, etc.
* **`javascriptlexer.py` (in `javascript/`):** Implements the lexer for JavaScript, recognizing its keywords, operators, comments (`//`, `/* */`), various literal types (including template literals and regex literals), and syntax features.
* **`pythonlexer.py` (in `python/`):** Implements the lexer for Python, paying close attention to its keywords, operators, comments (`#`), string formats (f-strings, raw strings), numeric literals, and crucially, handling indentation and dedentation as significant tokens.
* **`rubylexer.py` (in `ruby/`):** Implements the lexer for Ruby, identifying its keywords (including block terminators like `end`), operators, comments (`#`), symbols, string literals, heredocs, and other syntactic elements.

Each language-specific lexer class typically inherits from a base lexer class or follows a common interface, taking the source code string as input and providing an iterator or method to yield tokens one by one until the end of the file is reached.

## Usage within SpiceCode

The appropriate lexer is selected dynamically based on the detected language of the input file (usually determined by the file extension via the `utils.get_lang` and `utils.get_lexer` utilities). The chosen lexer processes the source code, and the resulting stream of tokens is fed into the `parser` module to build a structured representation (like an AST) for further analysis by the modules in `spice/analyzers`.

Understanding the role and structure of the `lexers` module is key for contributors looking to extend SpiceCode's language support or refine the analysis for existing languages. It represents the critical first step in SpiceCode's journey from raw source code to actionable insights.
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
sidebar_position: 2
---

# API Reference: Parser Module (`parser`)

Following the lexical analysis phase performed by the `lexers` module, the resulting stream of tokens is passed to the `parser` module. This module is responsible for syntactic analysis – the process of analyzing the sequence of tokens to determine the grammatical structure of the source code according to the rules of the specific programming language. The parser effectively checks if the token stream forms a valid program and typically constructs a data structure, often an Abstract Syntax Tree (AST), that represents the hierarchical structure of the code. This structured representation is crucial for the subsequent semantic analysis and metric calculation performed by the `spice/analyzers`.

Similar to the lexers, SpiceCode employs a native parsing approach, meaning it does not rely on external parsing libraries or language-specific compilers to generate ASTs. This ensures self-sufficiency and allows for tailored parsing logic suited to SpiceCode's specific analysis needs.

## Purpose of Syntactic Analysis

Syntactic analysis, or parsing, takes the flat sequence of tokens from the lexer and organizes them into a hierarchical structure that reflects the code's logical organization. It verifies that the code adheres to the language's grammar rules (e.g., ensuring parentheses are balanced, statements are correctly formed, keywords are used appropriately). The primary output of this phase in many compilers and analysis tools is an AST. An AST abstracts away much of the source code's superficial syntax (like parentheses or commas) and represents the essential structural elements (like function definitions, loops, conditional statements, expressions) and their relationships.

## Structure of the `parser` Module

The `parser` module within SpiceCode likely contains the core parsing logic and potentially definitions for the AST nodes:

```
parser/
├── __init__.py
├── ast.py # Defines the nodes for the Abstract Syntax Tree (AST)
└── parser.py # Contains the main Parser class and parsing logic
```

### `ast.py`

This file is expected to define the various types of nodes that make up the Abstract Syntax Tree. An AST is a tree representation of the code's structure. Each node in the tree represents a construct occurring in the source code. For example, there might be node classes for:

* **Program/Module:** The root node representing the entire file.
* **FunctionDefinition:** Representing a function or method definition, potentially containing nodes for parameters, return type, and the function body.
* **ClassDefinition:** Representing a class definition.
* **IfStatement:** Representing an if-else conditional structure.
* **ForLoop/WhileLoop:** Representing loop constructs.
* **Assignment:** Representing an assignment operation.
* **VariableDeclaration:** Representing variable declarations.
* **Expression Nodes:** Representing various kinds of expressions (e.g., BinaryOperation, FunctionCall, Literal, VariableReference).

Each AST node class would typically store relevant information about the construct it represents (e.g., a FunctionDefinition node might store the function name, parameter list, and a list of statements in its body).

### `parser.py`

This file likely contains the main `Parser` class (or equivalent logic). The parser takes the stream of tokens generated by a language-specific lexer as input. It implements parsing algorithms (e.g., recursive descent, or using a parser generator pattern) to consume the tokens and build the AST according to the grammar rules of the target language.

The parser needs to handle the specific syntax of each supported language (Go, JavaScript, Python, Ruby). This might involve:

* A single `Parser` class with methods that adapt based on the detected language.
* Or potentially separate parser implementations or configurations for each language, although the file structure suggests a more unified approach.

The parser interacts closely with the `ast.py` module, creating instances of the appropriate AST node classes as it recognizes grammatical structures in the token stream. It also performs error handling, reporting syntax errors if the token sequence violates the language's grammar rules.

## Usage within SpiceCode

After a lexer tokenizes the input source file, the `parser` module is invoked with this token stream. The parser constructs the AST. This AST is then passed to the various analyzers located in the `spice/analyzers` directory. The analyzers traverse this AST, extracting information and calculating metrics based on the structure and content represented by the tree nodes. For example, the `count_functions` analyzer would traverse the AST looking for `FunctionDefinition` nodes.

The `parser` module, along with the `ast.py` definitions, forms the bridge between the raw text of the source code and the structured understanding required for meaningful analysis. Its native implementation is key to SpiceCode's self-contained nature and its ability to provide consistent analysis across different programming languages.
Loading
Loading