PrimeCompress

A compression library featuring multiple adaptive strategies for different data types. Built on mathematical principles and targeted compression techniques to achieve efficient compression across various data patterns.

Try it Online

Visit our PrimeCompress Web Application to compress files directly in your browser using WebAssembly.

Features

• Strategy Selection: Intelligent selection of compression strategy based on data characteristics
• Block-Based Compression: Splits larger data into blocks, applying optimal compression for each block
• True Huffman Encoding: Dictionary compression with Huffman coding for text data
• Fast Path for High-Entropy Data: Optimized handling of random or incompressible data
• Pattern Detection: Recognition of constants, patterns, sequences, and spectral characteristics
• Perfect Reconstruction: Guaranteed exact data reconstruction with checksum validation

Compression Strategies

PrimeCompress uses multiple compression strategies:

• Zeros: For constant data (all bytes are the same)
• Pattern: For repeating patterns in binary data
• Sequential: For arithmetic sequences and i%N patterns
• Spectral: For sinusoidal and wave-like data
• Dictionary: For text-like data with frequency-optimized dictionary
• Statistical: Fallback for high-entropy data

Performance

PrimeCompress performance is measured through automated benchmarks. The results below are dynamically updated based on the latest tests.

Hutter Prize Benchmark

Our compression performance is measured against the Hutter Prize dataset (enwik8), a standard benchmark in the compression community:

The benchmark compares PrimeCompress against standard algorithms like gzip and brotli. Results are automatically updated with each code change.

You can run the benchmark yourself by triggering the "Hutter Prize Benchmark" workflow in the Actions tab.

Installation

From GitHub Packages:

# Configure npm to use GitHub Packages
echo "@uor-foundation:registry=https://npm.pkg.github.com" >> .npmrc

# Install the package
npm install @uor-foundation/prime-compress

From source:

npm install

Usage

// When installed from npm
const compression = require('@uor-foundation/prime-compress');

// OR when using from source
// const compression = require('./unified-compression.js');

// Compress data
const compressedData = compression.compress(data);

// Decompress data
const originalData = compression.decompress(compressedData);

// Compress with a specific strategy
const compressedWithStrategy = compression.compressWithStrategy(data, 'dictionary');

// Analyze data to determine optimal compression strategy
const analysis = compression.analyzeCompression(data);
console.log(`Recommended strategy: ${analysis.recommendedStrategy}`);

Advanced Features

Block-Based Compression

For larger datasets, PrimeCompress automatically applies block-based compression, dividing data into optimal blocks and compressing each with the best strategy.

// Enable block-based compression (enabled by default for data > 4KB)
const compressedBlocks = compression.compress(largeData, { useBlocks: true });

Enhanced Dictionary Compression

Text data benefits from frequency-optimized dictionary compression with Huffman coding for further space savings.

// Force dictionary strategy with Huffman coding
const compressedText = compression.compressWithStrategy(textData, 'enhanced-dictionary');

Analysis Tools

PrimeCompress includes tools for analyzing data characteristics:

const analysis = compression.analyzeCompression(data);
console.log(`Entropy: ${analysis.entropy}`);
console.log(`Recommended strategy: ${analysis.recommendedStrategy}`);
console.log(`Estimated compression ratio: ${analysis.theoreticalCompressionRatio}x`);

Mathematical Basis

The implementation is based on advanced mathematical principles including:

• Entropy-based optimization for strategy selection
• Spectral analysis for wave-like data patterns
• Frequency distribution analysis for dictionary optimization
• Huffman tree construction for optimal prefix coding

Testing

Run the comprehensive test suite:

node unified-compression-test.js

Web Application

The PrimeCompress project includes a React-based web application that allows users to compress files directly in their browser using WebAssembly. The web application:

• Runs completely client-side (no server required)
• Uses WebAssembly for high-performance compression
• Processes files in a Web Worker to prevent UI blocking
• Provides multiple compression strategies
• Shows detailed compression statistics

The frontend code is located in the ./frontend directory. See the Frontend README for more details.

Development

cd frontend
npm install
npm start

Deployment

The web application is automatically deployed to GitHub Pages when changes are pushed to the main branch. You can also deploy it manually:

cd frontend
npm run deploy

Publishing

This package is configured to publish to GitHub Packages.

Automatic Publishing (CI/CD)

The package will automatically be published to GitHub Packages when a new release is created on GitHub.

Manual Publishing

To publish manually:

1. Log in to the GitHub Packages registry:

   npm login --registry=https://npm.pkg.github.com

2. Update the version in package.json:

   npm version patch  # or minor, or major

3. Publish the package:

   npm publish

License

MIT