@monogrid/gainmap-js

gainmap-js

A Javascript (TypeScript) Encoder/Decoder Implementation of Adobe's Gain Map Technology for storing HDR Images using an SDR Image + a "Gain map"

:warning: This library is intended for encoding and decoding gain map images for the three.js 3D Library

It can be used for general encode/decode of gain maps but it depends on the three.js library which, in itself, is quite heavy if you only use it to encode/decode gain maps.

Share Your Project!

Have you built something awesome using this library? We’d love to see what you’ve created! Whether it’s a small demo or a full-scale application, your project can inspire others and help improve the library. Please share your work in one of these ways:

• GitHub Issues: Open a new issue in this repository with a brief description, screenshots, and links to your project.
• Email: Send us an email at [e-mail] with your project details. Your contributions and creative ideas make our community stronger. We can’t wait to see what you build!

Live Demo

• WebGPU Version
• Legacy WebGL Version

Both compares loading:

• a JPEG file with embedded gain map data
• a webp sdr file + a webp gain map + metadata JSON
• a comparable size .hdr file for comparison

Free online encoder

Use it to convert .hdr and .exr files into gain maps. It's free and the whole process happens in your browser.

Installing

$ npm install @monogrid/gainmap-js three

What is a Gain map?

See here for a detailed explanation, here are some relevant parts:

A gain map is a single file with a second pseudo-image embedded in it to create an optimized result for a specific monitor. It can be used to generate the HDR version (which looks dramatically better where supported), the SDR version (without tone mapping to ensures great quality), or anything in between (to better support less capable HDR displays).

Gain maps are not a new type of file, but rather a technology which can be embedded into a variety of image formats. There are reference specs already for the JPG, AVIF, JXL, and HEIF file formats. JPG is especially notable as it could not properly support HDR without gain maps and it offers a very useful bridge to the future (i.e. highly compatible with today’s software).

A gain map includes:

• A base (default) image. This can be an SDR or an HDR image (JPG gain maps are always encoded with SDR as the base). If the browser or viewing software does not understand gain maps, it will just the treat file as if it were just the base image.
• The gain map. This is a secondary “image” embedded in the file. It is not a real image, but rather contains data to convert each pixel from the base image into the other (SDR or HDR) version of the image.
• Gain map metadata. This tells the browser how the gain map is encoded as well as critical information to optimize rendering on any display.

Please note that Google is adopting the gain map technology in Android 14 but its naming of the technology refers to it as Ultra HDR Image Format and a JPEG file with embedded gain map is apparently called JPEGR in their terminology, we call it HDRJPEG for the moment.

API

Refer to the WIKI for detailed documentation about the API.

Examples

Decoding

The main use case of this library is to decode a JPEG file that contains gain map data and use it instead of a traditional .exr or .hdr image.

NOTE: Starting from v3.1.0 Decoding also works in a WebWorker, thanks to Alejandro Romano for the contribution

Using a single JPEG with embedded Gain map Metadata

This approach lets you load a single file with an embedded Gain Map.

The advantage is to have a single file to load.

The disadvantages are:

• No WEBP compression
• The JPEG cannot be manipulated in Photoshop, GIMP, or any other software that does not support the gain map technology (no photo editing software supports it at the time of writing 06-11-2023).
• Photo sharing websites and/or services (i.e. sharing with Slack) will likely strip the Gain map metadata and the HDR information will be lost, leaving you with only the SDR Representation.

import { HDRJPGLoader } from '@monogrid/gainmap-js'
import {
  EquirectangularReflectionMapping,
  Mesh,
  MeshBasicMaterial,
  PerspectiveCamera,
  PlaneGeometry,
  Scene,
  WebGLRenderer
} from 'three'

const renderer = new WebGLRenderer()

const loader = new HDRJPGLoader(renderer)
  .setRenderTargetOptions({ mapping: EquirectangularReflectionMapping })

const result = await loader.loadAsync('gainmap.jpeg')
// `result` can be used to populate a Texture

const scene = new Scene()
const mesh = new Mesh(
  new PlaneGeometry(),
  new MeshBasicMaterial({ map: result.renderTarget.texture })
)
scene.add(mesh)
renderer.render(scene, new PerspectiveCamera())

// Starting from three.js r159
// `result.renderTarget.texture` can
// also be used as Equirectangular scene background
//
// it was previously needed to convert it
// to a DataTexture with `result.toDataTexture()`
scene.background = result.renderTarget.texture

// result must be manually disposed
// when you are done using it
result.dispose()

Using separate files

Using separate files will get rid of the limitations of using a single JPEG file but it will force to use three separate files

• An SDR Representation file
• A Gainmap file
• A JSON containing the gainmap metadata used for decoding

import { GainMapLoader } from '@monogrid/gainmap-js'
import {
  EquirectangularReflectionMapping,
  Mesh,
  MeshBasicMaterial,
  PerspectiveCamera,
  PlaneGeometry,
  Scene,
  WebGLRenderer
} from 'three'

const renderer = new WebGLRenderer()

const loader = new GainMapLoader(renderer)
  .setRenderTargetOptions({ mapping: EquirectangularReflectionMapping })

const result = await loader.loadAsync(['sdr.jpeg', 'gainmap.jpeg', 'metadata.json'])
// `result` can be used to populate a Texture

const scene = new Scene()
const mesh = new Mesh(
  new PlaneGeometry(),
  new MeshBasicMaterial({ map: result.renderTarget.texture })
)
scene.add(mesh)
renderer.render(scene, new PerspectiveCamera())

// Starting from three.js r159
// `result.renderTarget.texture` can
// also be used as Equirectangular scene background
//
// it was previously needed to convert it
// to a DataTexture with `result.toDataTexture()`
scene.background = result.renderTarget.texture

// result must be manually disposed
// when you are done using it
result.dispose()

Decoding with WebGPU

NOTE: WebGPU decoding requires three.js r163 or higher and a browser that supports WebGPU

The library provides WebGPU-accelerated decoding through the @monogrid/gainmap-js/webgpu entry point. WebGPU decoding uses Three.js Shading Language (TSL) and offers improved performance and modern GPU architecture support.

Using a single JPEG with embedded Gain map Metadata (WebGPU)

import { HDRJPGLoader } from '@monogrid/gainmap-js/webgpu'
import {
  EquirectangularReflectionMapping,
  Mesh,
  MeshBasicMaterial,
  PerspectiveCamera,
  PlaneGeometry,
  Scene,
  WebGPURenderer
} from 'three/webgpu'

const renderer = new WebGPURenderer()
await renderer.init()

const loader = new HDRJPGLoader(renderer)
  .setRenderTargetOptions({ mapping: EquirectangularReflectionMapping })

const result = await loader.loadAsync('gainmap.jpeg')
// `result` can be used to populate a Texture

const scene = new Scene()
const mesh = new Mesh(
  new PlaneGeometry(),
  new MeshBasicMaterial({ map: result.renderTarget.texture })
)
scene.add(mesh)
renderer.render(scene, new PerspectiveCamera())

// Starting from three.js r159
// `result.renderTarget.texture` can
// also be used as Equirectangular scene background
scene.background = result.renderTarget.texture

// result must be manually disposed
// when you are done using it
result.dispose()

Using separate files (WebGPU)

import { GainMapLoader } from '@monogrid/gainmap-js/webgpu'
import {
  EquirectangularReflectionMapping,
  Mesh,
  MeshBasicMaterial,
  PerspectiveCamera,
  PlaneGeometry,
  Scene,
  WebGPURenderer
} from 'three/webgpu'

const renderer = new WebGPURenderer()
await renderer.init()

const loader = new GainMapLoader(renderer)
  .setRenderTargetOptions({ mapping: EquirectangularReflectionMapping })

const result = await loader.loadAsync(['sdr.jpeg', 'gainmap.jpeg', 'metadata.json'])
// `result` can be used to populate a Texture

const scene = new Scene()
const mesh = new Mesh(
  new PlaneGeometry(),
  new MeshBasicMaterial({ map: result.renderTarget.texture })
)
scene.add(mesh)
renderer.render(scene, new PerspectiveCamera())

// Starting from three.js r159
// `result.renderTarget.texture` can
// also be used as Equirectangular scene background
scene.background = result.renderTarget.texture

// result must be manually disposed
// when you are done using it
result.dispose()

Key differences between WebGL and WebGPU

• Import path: Use @monogrid/gainmap-js/webgpu instead of @monogrid/gainmap-js
• Three.js imports: Use three/webgpu instead of three
• Renderer: Use WebGPURenderer instead of WebGLRenderer
• Renderer initialization: Call await renderer.init() before using WebGPU renderer
• Async decoding: The decode() function is async in WebGPU

Encoding

Encoding a Gain map starting from an EXR file.

This is generally not useful in a three.js site but this library exposes methods that allow to encode an .exr or hdr file into a jpeg with an embedded gain map.

import { compress, encode, findTextureMinMax } from '@monogrid/gainmap-js/encode'
import { encodeJPEGMetadata } from '@monogrid/gainmap-js/libultrahdr'
import { EXRLoader } from 'three/examples/jsm/loaders/EXRLoader.js'

// load an HDR file
const loader = new EXRLoader()
const image = await loader.loadAsync('image.exr')

// find RAW RGB Max value of a texture
const textureMax = findTextureMinMax(image)

// Encode the gainmap
const encodingResult = encode({
  image,
  // this will encode the full HDR range
  maxContentBoost: Math.max.apply(this, textureMax)
})

// obtain the RAW RGBA SDR buffer and create an ImageData
const sdrImageData = new ImageData(encodingResult.sdr.toArray(), encodingResult.sdr.width, encodingResult.sdr.height)
// obtain the RAW RGBA Gain map buffer and create an ImageData
const gainMapImageData = new ImageData(encodingResult.gainMap.toArray(), encodingResult.gainMap.width, encodingResult.gainMap.height)

// parallel compress the RAW buffers into the specified mimeType
const mimeType = 'image/jpeg'
const quality = 0.9

const [sdr, gainMap] = await Promise.all([
  compress({
    source: sdrImageData,
    mimeType,
    quality,
    flipY: true // output needs to be flipped
  }),
  compress({
    source: gainMapImageData,
    mimeType,
    quality,
    flipY: true // output needs to be flipped
  })
])

// obtain the metadata which will be embedded into
// and XMP tag inside the final JPEG file
const metadata = encodingResult.getMetadata()

// embed the compressed images + metadata into a single
// JPEG file
const jpeg = encodeJPEGMetadata({
  ...encodingResult,
  ...metadata,
  sdr,
  gainMap
})

// `jpeg` will be an `Uint8Array` which can be saved somewhere

// encoder must be manually disposed
// when no longer needed
encodingResult.gainMap.dispose()
encodingResult.sdr.dispose()

Libultrahdr in Vite

If you import @monogrid/gainmap-js/libultrahdr You will need to exclude it from Vite optimizations.

// vite.config.js

module.exports = defineConfig({
  ...
  optimizeDeps: {
    exclude: ['@monogrid/gainmap-js/libultrahdr']
  },
  ...
})

Building

Clone the repository:

$ git clone [e-mail]:MONOGRID/gainmap-js.git
$ cd gainmap-js
$ npm i
$ npm run build

References

• Adobe Gainmap Specification
• Ultra HDR Image Format v1.0

License