How To Read Exif Hexadecimal Metadata from AVIF with JavaScript

How To Read Exif Hexadecimal Metadata from AVIF with JavaScript? Here is how, based off some random info gathered from Stackoverflow. Impossible to find any data online, the MIAB and ISO publications are not free. Go figure.

AVIF, HEIC, WebP, JXL, Késako?

As we look for alternatives to replace the aging JPEG image format, AVIF, HEIC, and JPEG XL have emerged as strong contenders. AVIF, developed by the Alliance for Open Media, offers excellent compression and is expected to become the dominant format for web delivery. HEIC, primarily supported in the Apple ecosystem, provides superior compression for full-color photos. JPEG XL, being developed by JPEG, Google, and Cloudinary, offers a balance between compression and visual fidelity along with advanced features.

Read this excellent in-depth comparison of these new image formats, where WebP is not even mentionned. WebP is pushed by Google, and it was a good idea to replace JPEG. Unfortunately, once you see what AVIF can do, WebP can be forgotten completely. There are many blogs comparing WebP vs AVIF, and even Adobe tried at some point, with this article of very poor quality (and many typos).

In AEM Dynamic Media, PNG image format is considered to be lossless. Hence all PNG images are always delivered at 100 quality. In this comparison we compared the images at quality setting of 90 for JPEG/WebP and 50 for AVIF. But this quality number is subjective to each format.

How do we ensure if the image has same visual quality? Answer is PSNR (Peak Signal to Noise Ratio). PSNR is a good measure for comparing restoration results for the same image. PSNR of JPEG/WebP/AVIF image is calculated with respect to its PNG image.

Here one can observe WebP at quality 90 and AVIF at quality 50 maintains the similar PSNR when compared to JPEG at quality 90:

The chart below shows the same images size’, for Webp quality=90 vs AVIF quality=50:

AVIF always beats WebP in size, for similar visual quality. Only problem is its CPU consumption: AVIF encoding requires 4x more CPU power then WebP.

Use Case Scenario to Read Exif from AVIF

ComfyUI currently (recently) added WebP support. Formerly it could only read prompts from PNG files. Now that @comfyanonymous fixed WebP support, let’s add AVIF!

Why AVIF? Not gonna enter a debate others already have discussed. It’s just better, open source, smaller, can handle transparency, etc. Based on heic and AV1 codecs already used by H.264, Youtube, Apple etc. The only downside is the CPU power needed to compress.

In our very specific use case scenarion, we are reading IFD Exif tags, that will represent ASCII JSON dumps from the workflow we want to load in ComfyUI. Tags can be any tags, but the best to use for this case are:

• 0x010e = ImageDescription = workflow
• 0x010f= Make = prompt

A ComfyUI custom node such as Save Image Extended or WAS Node Suite will always follow the below best practices when saving the prompt/workflow in Exif tags:

from PIL import Image, ExifTags
import numpy as np
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
metadata["prompt"] = prompt

## This method gives good results, as long as you save the prompt/workflow in 2 separate Exif tags
## Otherwise, [ExifTool] will issue Warning: Invalid EXIF text encoding for Make
## exif type is PIL.Image.Exif
exif = img.getexif()

## It seems better to separate the two
# 0x010e: ImageDescription
# 0x010f: Make
# interestingly, workflow will come first in the Exif blocks inside avif
exif[0x010f] = "Prompt: " + json.dumps(metadata['prompt'])     # Make
exif[0x010e] = "Workflow: " + json.dumps(metadata['workflow']) # ImageDescription

# Add more kwargs as needed: pnginfo=metadata, compress_level=self.png_compress_level etc
kwargs["exif"] = exif.tobytes()

img.save(image_path, **kwargs)

How To Read Exif From AVIF

AVIF File Signature

Very simple: we read from offest 4 and 8, and look for ftyp attached to avif or heic. HEIC is the licensed, unusable, concurrent codec from Apple:

In Javascript, that could look like this:

export function getAvifMetadata(file) {
  return new Promise((r) => {
    const reader = new FileReader();
    reader.onload = (event) => {
      const avif = new Uint8Array(event.target.result);
      console.log('event.target.result',event.target.result); // this is a decimal dump of the file: 0, 0, 0, 32, 102, == 00 00 00 20 ..
      console.log('avif',avif);
      const dataView = new DataView(avif.buffer);
      console.log('dataView',dataView);

      // Check that the AVIF signature is present: [4 byte offset] ftypheic or ftypavif
      // console.log('avif dataView.getUint32(4)',dataView.getUint32(4));  // 1718909296 = 0x66747970 = ftyp
      // console.log('avif dataView.getUint32(8)',dataView.getUint32(8));  // 1635150182 = 0x61766966 = avif
      // console.log('avif dataView.getUint32(8)',dataView.getUint32(8));  // 1751476579 = 0x68656963 = heic
      //                1718909296 = 0x66747970 = ftyp          1635150182 = 0x61766966 = avif         1751476579 = 0x68656963 = heic
      if (!(dataView.getUint32(4) == 0x66747970 && (dataView.getUint32(8) == 0x61766966 || dataView.getUint32(8) == 0x68656963))) {
        console.error("Not a valid AVIF file");
        r();
        return;
      }

Important Javascript functions to use here:

• dataView.getUint32() reads 4 bytes starting at the specified byte offset of DataView
• dataView.getUint16() reads 2 bytes starting at the specified byte offset of DataView
• dataView.getUint8() reads 1 bytes starting at the specified byte offset of DataView

AVIF Metadata Core Chunk Analysis

After the signature, we have some stuff defining the kind of avif it is, thecompatible Brands: [avif,mif1,miaf,MA1B,etc], the codecs etc, and then a 00 00 separator, and then 2-bytes defining the size of the core section:

The examples above are the exact same image generated with exact same workflow. Only difference is, the second one was a png converted to AVIF with imagick + exiftool to re-import the prompt:

PIL.Image AVIF

saved by Save Image Extended node output AVIF

PNG converted to AVIF

same image saved by Save Image Extended node as PNG, then converted to AVIF with imagick and workflow+prompt exported and re-imported by exiftool

So,  we have in this order:

• signature that starts at 0x4 and is 8-byte long: ftypavif most likely
• 4-byte 00 space
• multiple 4-byte words defining how the avif was build, ending with 00 00
  • examples above have different compatible brands: avifmiflmiaf (image magick) or avifmiflmiafMA1B (PIL.Image)
• Then the core image definition length on 2-bytes: examples above are 0x133 nd 0x12B long, ending just before the 8-byte mdat section
• mdat section is 8-byte long
• then we finally have the 6-byte Exif\0\0 followed by the IFD definitions, and then the Exif chunks

6-byte Exif IFD section offset position

Following the explanations from above, here is how to calculate the 6-byte Exif\0\0 position:

• we start at offset 4 + 8 + 4 = 0x10
• then we slice n 4-byte words until the last one starts with 00 00: 0x10 + 0x4<em>n = offset for core metadata size (meta)
• then we read core metadata size offset as a 4-byte word (meta)
• then we add 0x10 + 0x4<em>n</em> + 4 + meta + 8 = offset for Exif\0\0 section!

In Javascript, that gives this:

let metaOffset = 0x10
while (metaOffset < avif.length / 2) {
  let word = String.fromCharCode(...avif.slice(metaOffset, metaOffset+4));
  // console.log('word',avif.slice(metaOffset, metaOffset+4));
  // console.log('word',word);
  if (word.slice(0,2) != "\0\0") {
    metaOffset += 4
  } else break;
}
let metaSize = dataView.getUint32(metaOffset);
let offset = metaOffset + 4 + metaSize + 8;
//console.log('metaSize',metaSize);
//console.log('offset',offset);

AVIF Core Image Definitions Sections

This picture summarize it all: they are all of various length, chained to each other, with length of the next section at the end of each section:

The only value we are looking for in this section is the Exif chunk size:

Exif Chunk Size Position Method

The Exif chunk size is 4-byte long, defined by iloc, and succeeded by iinf.

• We start from metaOffset + 4 = offset for the whole meta section. As seen above, each section length in meta is defined by the last 1 or 2-byte, more or less.
• we set slice = 0xC
• as long as the current 4-byte word is not == iloc:
  • we slice and get the last 2 as length for the next section;
  • we offset + slice and
    • if next 4-byte word is not iloc, loop: slice = the last 2
    • if next 4-byte word is iloc, we offset + 0x2C and this is the offsetChunk_length!
    • read chunk_size and break loop

In JavaScript, that gives this:

let offsetChunk_length = metaOffset + 4;
let slice = 0xC;
while (offsetChunk_length < avif.length / 2) {
  let word = String.fromCharCode(...avif.slice(offsetChunk_length, offsetChunk_length+4));
  if (word != "iloc") {
    console.log('word',word);
    offsetChunk_length += slice; // next offset to read from
    slice = dataView.getUint16(offsetChunk_length - 2); // get new slice length
    console.log('next slice:',slice);
  } else break;
}
offsetChunk_length += 0x2C;
let chunk_length = dataView.getUint32(offsetChunk_length);
console.log('chunk_length',chunk_length);

How To read those Exif Metadata

Exif sections are pretty similar among AVIF, JPEG and WEBP, but placed at different positions in the file

The Exif APP1 Section is a 6-byte Exif\0\0 followed by TIFF Header of 10-bytes length. Therefore, after the 6-bytes Exif\0\0 we have 10-byte Tiff Header.

I generated many avif with PIL.Image and the structure is always the same. Only when created with Imagick, do we have slight differences in the meta section.

45 78 69 66 00 00             starts at offset = metaOffset + 4 + metaSize + 8: Exif + \0\0 = 6-bytes
E  x  i  f  00 00
Tiff header after is 0xA long (10 bytes) and contains information about byte-order of IFD sections and a pointer to the 0th IFD
Tiff header:
4D 4D 00 2A 00 00 00 08 00 02
-----|     |
49 49|     |                  (II for Intel) if the byte-order is little-endian
4D 4D|     |                  (MM for Motorola) for big-endian
     |00 2A|                  magic bytes 0x002A (=42, haha...)
           |         08|      following 4-byte will tell the offset to the 0th IFD from the start of the TIFF header.
                       |   02 last 2-byte seem like the number of extra Exif IFD, we have indeed only 2 in this example

IFD Fields are 12-byte subsections of IFD sections
For example, if we are looking for only 2 IFD fields with 0x010f/Make/prompt and 0x010e/ImageDescription/workflow:
4D 4D 00 2A 00 00 00 08 00 02         0th IFD, 10-byte long, last 2 bytes give how many IFD fields there are

01 0E 00 02 00 00 2F 15 00 00 00 26   010e/ImageDescription IFD1
01 0F 00 02 00 00 0F 7C 00 00 2F 3C   010f/Make             IFD2
-----|                                tag ID
     |-----|                          type of the field data. 1 for byte, 2 for ascii, 3 for short (uint16), 4 for long (uint32), etc
           |-----------|              length in 4 bytes, only for ascii, which we only care about; max is 4GB of data
                       |-----------|  4-byte field value, no idea what that's for
                                      
00 00 00 00                           then 4-byte offset from the end of the TIFF header to the start of the first IFD value
Workflow: {"...0.4} 00                W starts at offset 0x18B  (don't care) and is length 0x2F15 including 1 last \0
00                                    00 separator
Prompt: {"...     } 00                P starts at offset 0x30A1 (don't care) and is length 0x0F7C including 1 last \0

If that is not clear, don’t worry. Just open the file with HxD or whatever hexaddecimal tool you like and spend more time on it. All of this is corrobored by exiftool:

exiftool -struct image.avif

How to Code Exif Chunk Read Hexadecimal in Javascript

An example inside the reader.onload = (event) => {} loop would look like:

let txt_chunks = {}
// Loop through the chunks in the AVIF file
// console.log('chunk_length',chunk_length)
// avif clearly are different beasts than webp, there is only one chunck of Exif data at the beginning.
// If we ever come across one that is different, surely it's not been produced by a custom node and surely, the metadata is invalid
// while (offset < (offset + chunk_length)) { // no need to scan the whole avif file
  const chunk_type = String.fromCharCode(...avif.slice(offset, offset + 6));
  console.log('chunk_type',chunk_type)
  if (chunk_type === "Exif\0\0") {
    offset += 6;
    
    // parseExifData must start at the Tiff Header: 0x4949 or 0x4D4D for Big-Endian
    let data = parseExifData(avif.slice(offset, offset + chunk_length));
    for (var key in data) {
      if (data[key]) {
        var value = data[key];
        console.log('data[key]',value)
        let index = value.indexOf(':');
        txt_chunks[value.slice(0, index)] = value.slice(index + 1);
      }
    }
  }

  // offset += chunk_length;
// }

Then the function parseExifData will split the whole Exif chunk into each of its IFD components and return its content.

How Similar Exif Metadata look between AVIF JPEG and WEBP

Only WebP places the Exif IFD data at the bottom of the file, the others place it at the start or close.

Hope that helps anyone on earth… Thanks to Ozgur Ozcitak’s post on Stackoverflow for the help! So unfortunate that we have to reverse-engineer stuff that’s supposed to be open source… Any idea why the specs are not available?

ComfyUI AVIF support PR created, fingers crossed!

❌

◀️ ▶️