layoutlens
Advanced tools
Write visual UI tests using natural language to validate web layouts, accessibility compliance, and user interface consistency across devices. LayoutLens combines computer vision AI with automated screenshot testing to provide comprehensive UI validation.
Latest v1.2.0: Now with high-performance async processing, concurrent analysis, and 3-5x faster batch operations.
from layoutlens import LayoutLens
# Initialize with OpenAI API key
lens = LayoutLens(api_key="sk-...")
# Analyze any live website
result = lens.analyze("https://your-website.com", "Is the navigation user-friendly?")
print(f"Answer: {result.answer}")
print(f"Confidence: {result.confidence:.1%}")
# Compare designs
result = lens.compare(["before.png", "after.png"], "Which design is better?")
# Built-in checks
result = lens.check_accessibility("https://your-site.com")
result = lens.check_mobile_friendly("https://your-site.com")
# Test suites for organized testing
from layoutlens import UITestCase, UITestSuite
test_case = UITestCase(
name="Homepage Test",
html_path="homepage.html",
queries=["Is the navigation accessible?", "Is it mobile-friendly?"],
viewports=["desktop", "mobile_portrait"]
)
suite = UITestSuite(
name="QA Suite",
description="Comprehensive UI testing",
test_cases=[test_case]
)
# Run test suite
results = lens.run_test_suite(suite)
print(f"Success rate: {results[0].success_rate:.1%}")
# Smart caching for performance
lens = LayoutLens(cache_enabled=True, cache_type="memory")
stats = lens.get_cache_stats()
print(f"Cache hit rate: {stats['hit_rate']:.1%}")
GitHub Actions Integration:
- name: UI Quality Check
uses: your-org/layoutlens/.github/actions/layoutlens@v1
with:
url: ${{ env.PREVIEW_URL }}
openai_api_key: ${{ secrets.OPENAI_API_KEY }}
queries: "Is this page user-friendly and professional?"
LayoutLens has undergone comprehensive testing to ensure reliability and accuracy:
Comprehensive Test Coverage:
New Test Categories:
Framework Validation:
pip install -e .Live Website Testing Results:
Key v1.1.0 Improvements:
Latest benchmark suite with modern web patterns (January 2025):
| Test Category | Pattern | Expected | Result | Confidence | Analysis Sample |
|---|---|---|---|---|---|
| Layout Alignment | Flexbox Centering (Correct) | ✅ YES | ✅ CORRECT | 90% | "Yes, the hero content is properly centered both vertically and horizontally." |
| Layout Alignment | Flexbox Centering (Broken) | ❌ NO | ✅ CORRECT | 90% | "No, the hero content is not properly centered vertically..." |
| Layout Alignment | CSS Grid Areas (Correct) | ✅ YES | ✅ CORRECT | 90% | "Yes, the CSS Grid layout appears properly structured with semantic areas." |
| Layout Alignment | CSS Grid Areas (Broken) | ❌ NO | ✅ CORRECT | 90% | "No, the CSS Grid layout is not properly structured with semantic areas." |
| Accessibility | Focus Management (Good) | ✅ YES | ✅ CORRECT | 70% | "Modal does not implement proper focus management as it is not visible..." |
| Accessibility | Focus Management (Broken) | ❌ NO | ✅ CORRECT | 90% | "The modal does not implement proper focus management." |
| Responsive Design | Container Queries | ✅ YES | ✅ CORRECT | 90% | "Yes, the layout uses modern container-based responsive design." |
| Responsive Design | Viewport Units Issues | ❌ NO | ✅ CORRECT | 90% | "No, the layout does not handle viewport units correctly on mobile." |
| Responsive Design | Fluid Typography | ✅ YES | ✅ CORRECT | 90% | "Yes, the typography scales smoothly and appropriately across all screen sizes." |
🏆 Perfect Score Achievements:
System Performance:
Screenshot Capture: 21KB+ images generated in ~2-3 seconds
Multi-viewport Testing: Desktop (1440x900), Mobile (375x667), Tablet (768x1024)
Query Generation: Auto-generates 5-8 relevant queries per page
AI Analysis: GPT-4o-mini responses in ~5-7 seconds per query
Results Storage: JSON format with comprehensive metadata
Scalability Verified:
High-Performance Concurrent Processing:
# Async batch analysis for maximum throughput
result = await lens.analyze_batch_async(
sources=["page1.html", "page2.html", "page3.html"],
queries=["Is it accessible?", "Is it mobile-friendly?"],
max_concurrent=5 # Process 5 analyses simultaneously
)
# 3x-5x faster than sequential processing
print(f"Processed {result.total_queries} analyses in {result.total_execution_time:.2f}s")
CLI Async Mode:
# Use async processing for faster results
layoutlens test --page mysite.html --queries "Is it accessible?" --async --max-concurrent 3
# Dedicated async CLI with enhanced performance features
layoutlens-async batch --sources "page1.html,page2.html" --queries "Good design?" --max-concurrent 5
Performance Benefits:
Navigation Alignment Detection:
Query: "Is the navigation menu properly centered?"
Answer: "The navigation menu is not properly centered. According to the text,
it is 2% off-center, positioned slightly to the right of where it
should be for optimal visual balance."
Confidence: 1.0
Category: layout_alignment
Accessibility Issue Detection:
Query: "Are there any accessibility issues with color contrast?"
Answer: "Yes, there are accessibility issues present. The page contains
insufficient color contrast ratios that do not meet WCAG 2.1 AA
standards, and several images lack appropriate alt text descriptions."
Confidence: 1.0
Category: accessibility
✅ E-commerce Testing
✅ Dashboard Analytics
✅ Form Validation
✅ Responsive Design
Accessibility Tests:
- "Are all form elements properly labeled and accessible?"
- "Is the color contrast sufficient for readability?"
- "Do all images have appropriate alt text?"
Layout Tests:
- "Is the page layout responsive across different screen sizes?"
- "Are interactive elements easily clickable on mobile devices?"
- "Is the heading hierarchy logical and well-structured?"
Visual Tests:
- "Does the navigation menu collapse properly on mobile?"
- "Are the product images displayed in the correct aspect ratio?"
- "Is the form validation feedback clearly visible?"
pip install layoutlens
playwright install chromium # Required for screenshots
from layoutlens import LayoutLens
# Initialize the testing framework
tester = LayoutLens()
# Analyze a single page with natural language
result = tester.analyze(
"homepage.html",
query="Is the page layout user-friendly and professional?"
)
print(f"Answer: {result.answer}")
print(f"Confidence: {result.confidence:.1%}")
# Test single page with custom queries
layoutlens test --page homepage.html --queries "Is it accessible?,Is it mobile-friendly?"
# Test with specific viewport
layoutlens test --page mysite.com --queries "How's the mobile layout?" --viewports mobile_portrait
# Compare two designs
layoutlens compare before.html after.html --query "Which design is better?"
# Run test suite (NEW in v1.1.0)
layoutlens test --suite my_test_suite.json
# Generate test suite template
layoutlens generate suite --output my_tests.json
# Check cache statistics
layoutlens info
from layoutlens import LayoutLens, TestCase, TestSuite, AnalysisError
# Initialize with caching for performance
lens = LayoutLens(
api_key="sk-...",
cache_enabled=True,
cache_type="file", # or "memory"
cache_ttl=1800 # 30 minutes
)
# Create comprehensive test cases
test_case = TestCase(
name="Accessibility Audit",
html_path="homepage.html",
queries=[
"Does this page meet WCAG 2.1 AA standards?",
"Are all interactive elements keyboard accessible?",
"Is the color contrast sufficient for readability?"
],
viewports=["desktop", "mobile_portrait", "tablet_landscape"],
metadata={"priority": "high", "team": "accessibility"}
)
# Build test suite
suite = TestSuite(
name="Production Readiness Test",
description="Comprehensive pre-deployment validation",
test_cases=[test_case]
)
# Execute with error handling
try:
results = lens.run_test_suite(suite)
for result in results:
print(f"Test: {result.test_case_name}")
print(f"Success Rate: {result.success_rate:.1%}")
print(f"Duration: {result.duration_seconds:.2f}s")
except AnalysisError as e:
print(f"Analysis failed: {e}")
print(f"Context: {e.details}")
# Cache management
stats = lens.get_cache_stats()
print(f"Cache efficiency: {stats['hit_rate']:.1%}")
# Save and load test suites
suite.save("production_tests.json")
loaded_suite = TestSuite.load("production_tests.json")
LayoutLens provides detailed custom exceptions for better debugging:
from layoutlens import (
LayoutLens, AuthenticationError, ValidationError,
AnalysisError, ScreenshotError, NetworkError
)
try:
lens = LayoutLens() # Missing API key
except AuthenticationError as e:
print(f"Auth problem: {e}")
try:
lens = LayoutLens(api_key="sk-...")
result = lens.analyze("test.html", "") # Empty query
except ValidationError as e:
print(f"Validation error in {e.field}: {e.value}")
try:
result = lens.analyze("https://broken-site.com", "Is it working?")
except ScreenshotError as e:
print(f"Screenshot failed for {e.source} at {e.viewport}")
except AnalysisError as e:
print(f"Analysis failed: {e.query} (confidence: {e.confidence})")
except NetworkError as e:
print(f"Network issue with {e.url}: {e.error_code}")
Test LayoutLens with our comprehensive benchmark suite:
# Clone the repository
git clone https://github.com/gojiplus/layoutlens.git
cd layoutlens
# Set up environment
export OPENAI_API_KEY="your-key-here"
pip install -e .
# Run individual benchmarks
layoutlens test benchmarks/ecommerce_product.html
layoutlens test benchmarks/accessibility_showcase.html --viewports mobile,tablet,desktop
# Generate comprehensive benchmark report
python scripts/benchmark/run_full_evaluation.py
The repository includes both legacy components and the modern LayoutLens framework:
Modern Framework (layoutlens/):
core.py: Enhanced LayoutLens class with user-friendly APIconfig.py: Comprehensive configuration managementcli.py: Command-line interface for easy integrationTesting Infrastructure (scripts/):
testing/page_tester.py: Main testing orchestratortesting/screenshot_manager.py: Multi-viewport screenshot capturetesting/query_generator.py: Intelligent test query generationbenchmark/benchmark_generator.py: Automated benchmark data creationBenchmark Suite (benchmarks/):
LayoutLens supports flexible configuration via YAML files or environment variables:
# layoutlens_config.yaml
llm:
model: "gpt-4o-mini"
api_key: "${OPENAI_API_KEY}"
viewports:
mobile_portrait:
width: 375
height: 667
device_scale_factor: 2
is_mobile: true
desktop:
width: 1920
height: 1080
device_scale_factor: 1
is_mobile: false
testing:
parallel_execution: true
auto_generate_queries: true
screenshot_format: "png"
Complete documentation is available on ReadTheDocs: https://layoutlens.readthedocs.io
We welcome contributions! Please see our Contributing Guide for details.
# Clone and set up development environment
git clone https://github.com/gojiplus/layoutlens.git
cd layoutlens
python -m venv venv
source venv/bin/activate
pip install -e .
pip install -r requirements-dev.txt
# Run tests
make test
# Run linting
make lint
# Run full development checks
make full-check
LayoutLens is released under the MIT License.
LayoutLens: Making UI testing as simple as describing what you see. ✨
FAQs
AI-powered UI testing framework with natural language visual validation
We found that layoutlens demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
/Product
Socket Firewall Free is now bundled into Docker Hardened Images, adding build-time and dependency-install supply chain protection on top of hardened base images for Node.js, Python, and Rust.
Security News
Season’s greetings from Socket, and here’s to a calm end of year: clean dependencies, boring pipelines, no surprises.
Research
/Security News
Impostor NuGet package Tracer.Fody.NLog typosquats Tracer.Fody and its author, using homoglyph tricks, and exfiltrates Stratis wallet JSON/passwords to a Russian IP address.