158 lines
4.6 KiB
Markdown
158 lines
4.6 KiB
Markdown
# Unified Test Harness for Terrace
|
|
|
|
This directory contains the unified test harness for all Terrace language implementations. The harness automatically discovers and runs tests across all supported languages: JavaScript, C, Python, Go, and Rust.
|
|
|
|
## Features
|
|
|
|
- **Cross-language testing**: Run the same tests across all language implementations
|
|
- **Automatic test discovery**: Finds all `.tce` test files in the project
|
|
- **Flexible test definitions**: Tests are defined in human-readable `.tce` files
|
|
- **Build integration**: Automatically builds required binaries (C test runner)
|
|
- **Comprehensive reporting**: Detailed test results with pass/fail status
|
|
|
|
## Test File Format
|
|
|
|
Test files use the Terrace format with the following structure:
|
|
|
|
```terrace
|
|
#schema test
|
|
|
|
describe Test Suite Name
|
|
it Test description
|
|
key test-key
|
|
packages js c python go rust
|
|
input
|
|
test input data
|
|
more input lines
|
|
output
|
|
expected output line 1
|
|
expected output line 2
|
|
```
|
|
|
|
### Fields
|
|
|
|
- `describe`: Defines a test suite
|
|
- `it`: Defines a test case
|
|
- `key`: The test identifier passed to language implementations
|
|
- `packages`: Space-separated list of packages to test (js, c, python, go, rust)
|
|
- `input`: Test input data (optional)
|
|
- `output`: Expected output (optional for Go/Rust which use different testing frameworks)
|
|
|
|
## Usage
|
|
|
|
### Run all tests
|
|
|
|
```bash
|
|
npm test
|
|
# or
|
|
npm run test:unified
|
|
```
|
|
|
|
### Run specific test
|
|
|
|
```bash
|
|
node test/test-runner.js <test-key>
|
|
```
|
|
|
|
### Run tests for specific packages
|
|
|
|
```bash
|
|
PACKAGES="js python" node test/test-runner.js <test-key>
|
|
```
|
|
|
|
## Supported Test Keys
|
|
|
|
### JavaScript
|
|
|
|
- `linedata:basic` - Basic line data parsing
|
|
- `linedata:tabs` - Tab-based indentation
|
|
- `linedata:head-tail` - Head/tail parsing
|
|
- `new-api:basic` - Basic new API functionality
|
|
- `new-api:hierarchical` - Hierarchical document parsing
|
|
- `new-api:functional` - Functional API features
|
|
- `new-api:node-methods` - Node method testing
|
|
- `new-api:inconsistent-indentation` - Arbitrary nesting
|
|
|
|
### C
|
|
|
|
- `linedata:basic` - Basic line data parsing
|
|
- `linedata:tabs` - Tab-based indentation
|
|
- `linedata:head-tail` - Head/tail parsing
|
|
- `new-api:basic` - Basic new API functionality
|
|
- `new-api:hierarchical` - Hierarchical document parsing
|
|
- `new-api:string-views` - String view functionality
|
|
- `new-api:legacy-compat` - Legacy API compatibility
|
|
|
|
### Python
|
|
|
|
- `linedata:basic` - Basic line data parsing
|
|
- `linedata:tabs` - Tab-based indentation
|
|
- `linedata:head-tail` - Head/tail parsing
|
|
- `new-api:basic` - Basic new API functionality
|
|
- `new-api:hierarchical` - Hierarchical document parsing
|
|
- `new-api:functional` - Functional API features
|
|
- `new-api:node-methods` - Node method testing
|
|
- `new-api:readers` - Reader utilities
|
|
|
|
### Go
|
|
|
|
- `TestTerraceDocument` - Document parsing tests
|
|
- `TestTerraceNode` - Node functionality tests
|
|
|
|
### Rust
|
|
|
|
- `test_basic_parsing` - Basic parsing functionality
|
|
- `test_navigation_methods` - Navigation and traversal
|
|
|
|
## Adding New Tests
|
|
|
|
1. Create a new `.tce` file in the `test/` directory or modify existing ones
|
|
2. Define test suites and cases using the Terrace format
|
|
3. Specify which packages should run the test
|
|
4. Add the test key to the appropriate language implementation's test runner
|
|
5. Update the `supports` array in `test-runner.js` if needed
|
|
|
|
## Test Output Format
|
|
|
|
Each language implementation should output test results in a consistent format for comparison. The expected format varies by test type:
|
|
|
|
- **LineData tests**: `| level X | indent Y | offsetHead Z | offsetTail W | line TEXT |`
|
|
- **New API tests**: `| level X | head "HEAD" | tail "TAIL" | content "CONTENT" |`
|
|
- **Node method tests**: `Node: head="HEAD", tail="TAIL", isEmpty=BOOL, is(NAME)=BOOL`
|
|
- **Go/Rust tests**: Simple "Test passed" message
|
|
|
|
## Integration with CI/CD
|
|
|
|
The test harness is designed to work with CI/CD systems:
|
|
|
|
- Returns exit code 0 on success, 1 on failure
|
|
- Provides detailed output for debugging
|
|
- Can be run in parallel across different environments
|
|
- Supports selective test execution
|
|
|
|
## Dependencies
|
|
|
|
The test harness requires:
|
|
|
|
- Node.js for the test runner
|
|
- Build tools for each language (make, go, cargo, etc.)
|
|
- All language implementations to be properly set up
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **C tests fail**: Ensure the C test runner is built with `make` in `packages/c/`
|
|
2. **Go tests fail**: Ensure Go modules are properly initialized
|
|
3. **Rust tests fail**: Ensure Cargo dependencies are installed
|
|
4. **Python tests fail**: Ensure Python dependencies are installed
|
|
5. **Test discovery fails**: Check that `.tce` files have correct syntax
|
|
|
|
### Debug Mode
|
|
|
|
Run with verbose output:
|
|
|
|
```bash
|
|
DEBUG=1 node test/test-runner.js
|
|
```
|