hg-mpi

hg-mpi

Automated crawler for LCR-MPI-Challenge admin page.

Quick Start

# Fetch mode (download data)
npx hg-mpi fetch -u your_username -p your_password

# Update mode (update data)
npx hg-mpi update -u your_username -p your_password -d update_data.json

Usage

Fetch Mode

Downloads all submission data and files from the admin panel.

# Short format
npx hg-mpi fetch -u username -p password

# Long format
npx hg-mpi fetch --username=username --password=password

What it does:

• Logs into the admin panel
• Crawls all pages and extracts submission data
• Downloads all files to ./downloads/
• Saves metadata to ./results_data.json

Update Mode

Updates submission records on the admin panel from a JSON file.

npx hg-mpi update -u username -p password -d update_data.json

What it does:

• Phase 1 - Update: Reads update data from JSON file, matches records by ID, updates ssim, psnr, nrmse, and status fields, saves each page after filling
• Phase 2 - Verify: After all pages are updated, verifies all changes by searching through all pages (handles record reordering after save)

Status field tolerance:

• Supported values: pending, processing, done/completed, error

JSON format:

[
  {
    "id": "131",
    "ssim": "99.0001",
    "psnr": "99.0002",
    "nrmse": "99.0003",
    "status": "completed"
  },
  {
    "id": "130",
    "ssim": "",
    "psnr": "",
    "nrmse": "",
    "status": "error"
  }
]

Notes:

• Only include fields you want to update (id is required)
• Empty or missing fields will not be updated
• Status values are automatically normalized

Output

• downloads/ - All downloaded files
• results_data.json - Submission metadata (time, uploader, metrics, status)

Requirements

• Node.js >= 14.0.0
• Valid admin credentials

License

MIT