+144
-26
@@ -2,6 +2,7 @@
|
||||
<head>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta content="text/html;charset=utf-8" http-equiv="Content-Type"/>
|
||||
<link rel="stylesheet" href="colors-default.css" type="text/css">
|
||||
<link rel="stylesheet" href="colors.css" type="text/css">
|
||||
<link rel="stylesheet" href="style.css" type="text/css">
|
||||
<link rel="stylesheet" href="/highlight.css">
|
||||
@@ -12,18 +13,19 @@
|
||||
<article>
|
||||
<h1>Fucking around with perceptual hashes</h1>
|
||||
<h2>Introduction</h2>
|
||||
<p class="todo">TODO: why this project exists, the master's thesis that wasn't, discovering pHash after the fact.</p>
|
||||
<p class="todo">TODO: the rules of the game: 64 bits per image, hamming distance, nothing else.</p>
|
||||
</article>
|
||||
|
||||
<article>
|
||||
<h1>Perceptual hashing</h1>
|
||||
<h2>Locality-sensitive hashing</h2>
|
||||
<h3>Subjective image similarity</h3>
|
||||
</article>
|
||||
<article>
|
||||
<h1>Experimental setup</h1>
|
||||
<h2>Mutators</h2>
|
||||
<h2>Descriptor</h2>
|
||||
<h2>Experiments</h2>
|
||||
<p class="todo">TODO: cryptographic hash vs perceptual hash. One flipped pixel: md5 avalanches, a perceptual hash shrugs.</p>
|
||||
<h2>Terminology</h2>
|
||||
<p class="todo">TODO: descriptor / fingerprint / perceptual hash / signature all mean roughly the same thing depending on which corner of the literature you are standing in. Copy detection vs near-duplicate detection vs LSH.</p>
|
||||
<h2>Subjective image similarity</h2>
|
||||
<p class="todo">TODO: what "the same image" even means. Same pixels? Same scene? Same vibe?</p>
|
||||
</article>
|
||||
|
||||
<article id="demo">
|
||||
<h1>Demo</h1>
|
||||
<p>Select an image to run this demo with. Don't worry, nothing will be sent to any server! All calculations are done in the browser.</p>
|
||||
@@ -32,12 +34,15 @@
|
||||
type="file"
|
||||
id="dctimage"
|
||||
accept="image/*" />
|
||||
<span>or pick a sample:</span>
|
||||
<span id="sample-images"></span>
|
||||
<div id="base">
|
||||
<div class="image-original"></div>
|
||||
</div>
|
||||
</article>
|
||||
<article id="demo-resize">
|
||||
<h2>Resize</h2>
|
||||
|
||||
<article id="demo-resize" class="needs-image">
|
||||
<h2>Step 1: Resize</h2>
|
||||
<p>
|
||||
The first step is to size the image down, and remove all color information.
|
||||
Converting the image to grayscale is done by simply averaging the pixels.
|
||||
@@ -58,22 +63,135 @@
|
||||
<div id="image-resize"></div>
|
||||
</div>
|
||||
</article>
|
||||
<article id="demo-dct">
|
||||
<h2>DCT</h2>
|
||||
<p>What is DCT?</p>
|
||||
<p>Frequency domain. Plaatje. Bla bla.</p>
|
||||
<div id="image-dct"></div>
|
||||
<div class="image-original" id="dct-original"></div>
|
||||
<p>Mauris semper ipsum libero. Praesent laoreet massa sagittis enim consequat malesuada. Nullam viverra nibh sit amet lacus volutpat sollicitudin. Nullam lacus sem, commodo ut vulputate nec, consectetur in erat. Quisque eget nunc ac felis condimentum ultrices eu sit amet arcu. Praesent imperdiet faucibus aliquam. Curabitur sit amet faucibus erat. </p>
|
||||
<p>Nam pulvinar, mi id sagittis laoreet, risus dolor pellentesque metus, sed mattis nunc erat sed urna. Ut facilisis, velit vel condimentum euismod, ex ante dictum leo, lobortis feugiat eros sem nec velit. Aliquam quis eros lacus. Duis venenatis purus at luctus tempor. Praesent gravida euismod ante, tristique placerat tortor mattis molestie. Duis viverra ex eget lectus tempus consectetur. Aliquam semper, ligula at molestie dapibus, sapien turpis rutrum massa, ac tristique lorem ex eget neque. Donec vel turpis odio. </p>
|
||||
<p>Mauris semper ipsum libero. Praesent laoreet massa sagittis enim consequat malesuada. Nullam viverra nibh sit amet lacus volutpat sollicitudin. Nullam lacus sem, commodo ut vulputate nec, consectetur in erat. Quisque eget nunc ac felis condimentum ultrices eu sit amet arcu. Praesent imperdiet faucibus aliquam. Curabitur sit amet faucibus erat. </p>
|
||||
<p>Mauris semper ipsum libero. Praesent laoreet massa sagittis enim consequat malesuada. Nullam viverra nibh sit amet lacus volutpat sollicitudin. Nullam lacus sem, commodo ut vulputate nec, consectetur in erat. Quisque eget nunc ac felis condimentum ultrices eu sit amet arcu. Praesent imperdiet faucibus aliquam. Curabitur sit amet faucibus erat. </p>
|
||||
<p>Mauris semper ipsum libero. Praesent laoreet massa sagittis enim consequat malesuada. Nullam viverra nibh sit amet lacus volutpat sollicitudin. Nullam lacus sem, commodo ut vulputate nec, consectetur in erat. Quisque eget nunc ac felis condimentum ultrices eu sit amet arcu. Praesent imperdiet faucibus aliquam. Curabitur sit amet faucibus erat. </p>
|
||||
<p>Mauris semper ipsum libero. Praesent laoreet massa sagittis enim consequat malesuada. Nullam viverra nibh sit amet lacus volutpat sollicitudin. Nullam lacus sem, commodo ut vulputate nec, consectetur in erat. Quisque eget nunc ac felis condimentum ultrices eu sit amet arcu. Praesent imperdiet faucibus aliquam. Curabitur sit amet faucibus erat. </p>
|
||||
<p>Nam pulvinar, mi id sagittis laoreet, risus dolor pellentesque metus, sed mattis nunc erat sed urna. Ut facilisis, velit vel condimentum euismod, ex ante dictum leo, lobortis feugiat eros sem nec velit. Aliquam quis eros lacus. Duis venenatis purus at luctus tempor. Praesent gravida euismod ante, tristique placerat tortor mattis molestie. Duis viverra ex eget lectus tempus consectetur. Aliquam semper, ligula at molestie dapibus, sapien turpis rutrum massa, ac tristique lorem ex eget neque. Donec vel turpis odio. </p>
|
||||
<p>Nam pulvinar, mi id sagittis laoreet, risus dolor pellentesque metus, sed mattis nunc erat sed urna. Ut facilisis, velit vel condimentum euismod, ex ante dictum leo, lobortis feugiat eros sem nec velit. Aliquam quis eros lacus. Duis venenatis purus at luctus tempor. Praesent gravida euismod ante, tristique placerat tortor mattis molestie. Duis viverra ex eget lectus tempus consectetur. Aliquam semper, ligula at molestie dapibus, sapien turpis rutrum massa, ac tristique lorem ex eget neque. Donec vel turpis odio. </p>
|
||||
<p>Nam pulvinar, mi id sagittis laoreet, risus dolor pellentesque metus, sed mattis nunc erat sed urna. Ut facilisis, velit vel condimentum euismod, ex ante dictum leo, lobortis feugiat eros sem nec velit. Aliquam quis eros lacus. Duis venenatis purus at luctus tempor. Praesent gravida euismod ante, tristique placerat tortor mattis molestie. Duis viverra ex eget lectus tempus consectetur. Aliquam semper, ligula at molestie dapibus, sapien turpis rutrum massa, ac tristique lorem ex eget neque. Donec vel turpis odio. </p>
|
||||
<p>Nam pulvinar, mi id sagittis laoreet, risus dolor pellentesque metus, sed mattis nunc erat sed urna. Ut facilisis, velit vel condimentum euismod, ex ante dictum leo, lobortis feugiat eros sem nec velit. Aliquam quis eros lacus. Duis venenatis purus at luctus tempor. Praesent gravida euismod ante, tristique placerat tortor mattis molestie. Duis viverra ex eget lectus tempus consectetur. Aliquam semper, ligula at molestie dapibus, sapien turpis rutrum massa, ac tristique lorem ex eget neque. Donec vel turpis odio. </p>
|
||||
|
||||
<article id="demo-median" class="needs-image">
|
||||
<h2>Step 2: The simplest hash that could possibly work</h2>
|
||||
<p class="todo">TODO: median hash: one bit per pixel, brighter than the median or not. This is the "Median" baseline from Thomee et al.</p>
|
||||
<div class="hash-panel">
|
||||
<div>
|
||||
<h3>Bits</h3>
|
||||
<div id="median-bits" class="bit-grid"></div>
|
||||
</div>
|
||||
<div>
|
||||
<h3>Hash</h3>
|
||||
<code id="median-hash" class="hash-value"></code>
|
||||
</div>
|
||||
</div>
|
||||
<p class="todo">TODO: why this breaks: global median shifts, flips scramble everything. Foreshadow the moon/plate anecdote.</p>
|
||||
</article>
|
||||
|
||||
<article id="demo-dct" class="needs-image">
|
||||
<h2>Step 3: Frequency space</h2>
|
||||
<p class="todo">TODO: what the DCT does. JPEG uses the same trick. Low frequencies = global shape, high frequencies = detail we already threw away.</p>
|
||||
<div class="hash-panel">
|
||||
<div>
|
||||
<h3>DCT coefficients</h3>
|
||||
<div id="image-dct" class="heat-grid"></div>
|
||||
</div>
|
||||
</div>
|
||||
<p class="todo">TODO: reading the coefficient grid: top-left is the average, first row is horizontal waves, first column vertical waves.</p>
|
||||
<h3>From coefficients to bits</h3>
|
||||
<p class="todo">TODO: sign bits of the first 36 zigzag coefficients + 28 ordinal comparisons between neighbours in zigzag order. The odd-column signs get multiplied by the sign of coefficient (1,0), which buys horizontal flip invariance for one bit.</p>
|
||||
<div class="hash-panel">
|
||||
<div>
|
||||
<h3>Sign mask (36)</h3>
|
||||
<div id="dct-sign-bits" class="bit-grid wide"></div>
|
||||
</div>
|
||||
<div>
|
||||
<h3>Ordinal mask (28)</h3>
|
||||
<div id="dct-ordinal-bits" class="bit-grid wide"></div>
|
||||
</div>
|
||||
<div>
|
||||
<h3>Hash</h3>
|
||||
<code id="dct-hash" class="hash-value"></code>
|
||||
</div>
|
||||
</div>
|
||||
</article>
|
||||
|
||||
<article id="demo-phash" class="needs-image">
|
||||
<h2>Step 4: How pHash does it</h2>
|
||||
<p class="todo">TODO: same idea, different route: resize to 32x32 instead of 8x8, DCT, keep the top-left 8x8 block of low frequencies, threshold against the median coefficient.</p>
|
||||
<div class="hash-panel">
|
||||
<div>
|
||||
<h3>32×32</h3>
|
||||
<div id="phash-resize"></div>
|
||||
</div>
|
||||
<div>
|
||||
<h3>Low-frequency block</h3>
|
||||
<div id="phash-lowfreq" class="heat-grid"></div>
|
||||
</div>
|
||||
<div>
|
||||
<h3>Bits</h3>
|
||||
<div id="phash-bits" class="bit-grid"></div>
|
||||
</div>
|
||||
<div>
|
||||
<h3>Hash</h3>
|
||||
<code id="phash-hash" class="hash-value"></code>
|
||||
</div>
|
||||
</div>
|
||||
<p class="todo">TODO: what's genuinely different between our dct hash and phash (bit extraction, flip invariance) and what isn't (everything else).</p>
|
||||
</article>
|
||||
|
||||
<article id="demo-mutate" class="needs-image">
|
||||
<h2>Mutations</h2>
|
||||
<p class="todo">TODO: a copy is rarely byte-identical. Recoding, resampling, content processing, framing, inserted logos (the Thomee et al. taxonomy). Sweep the slider and watch which hash survives what.</p>
|
||||
<p>
|
||||
<select id="mutator-kind"></select>
|
||||
<input type="range" id="mutator-amount" />
|
||||
<span id="mutator-amount-value"></span>
|
||||
</p>
|
||||
<div id="mutate-compare" class="compare">
|
||||
<img id="compare-a" alt="original" />
|
||||
<img id="compare-b" alt="mutated" />
|
||||
</div>
|
||||
<p>
|
||||
<input type="range" id="compare-slider" min="0" max="100" value="50" />
|
||||
</p>
|
||||
<table class="distances">
|
||||
<tr><th></th><th>dct</th><th>median</th><th>phash</th></tr>
|
||||
<tr>
|
||||
<th>hamming distance</th>
|
||||
<td id="dist-dct"></td>
|
||||
<td id="dist-median"></td>
|
||||
<td id="dist-phash"></td>
|
||||
</tr>
|
||||
</table>
|
||||
<p class="todo">TODO: what counts as "the same" now? Thresholds. pHash uses 22 of 64 bits, we will measure our own.</p>
|
||||
</article>
|
||||
|
||||
<article id="demo-ranking">
|
||||
<h2>Find the copy</h2>
|
||||
<p class="todo">TODO: the fun part: a tiny search engine. Add a pile of images, click one, get the nearest neighbours per algorithm. Mention the white plate that matched the moon.</p>
|
||||
<p>
|
||||
<input type="file" id="ranking-files" accept="image/*" multiple />
|
||||
<select id="ranking-algo">
|
||||
<option value="0">dct</option>
|
||||
<option value="1">median</option>
|
||||
<option value="2">phash</option>
|
||||
</select>
|
||||
</p>
|
||||
<div id="ranking-pool" class="thumb-grid"></div>
|
||||
<h3>Results</h3>
|
||||
<div id="ranking-results" class="thumb-grid"></div>
|
||||
</article>
|
||||
|
||||
<article>
|
||||
<h2>At scale</h2>
|
||||
<p class="todo">TODO: the browser demo is anecdote, this section is data. 25k images, 16 mutations each, precision-recall over the hamming threshold.</p>
|
||||
<figure>
|
||||
<img src="" alt="PR curves for the dct hash" />
|
||||
<figcaption class="todo">TODO: dct-pr.png from the experiment run</figcaption>
|
||||
</figure>
|
||||
<figure>
|
||||
<img src="" alt="PR curves for median and phash" />
|
||||
<figcaption class="todo">TODO: median-pr.png and phash-pr.png</figcaption>
|
||||
</figure>
|
||||
<p class="todo">TODO: how the PR curves are computed, what a false positive means here, where the thresholds land per algorithm.</p>
|
||||
</article>
|
||||
|
||||
<article>
|
||||
<h2>Loose ends</h2>
|
||||
<p class="todo">TODO: vertical flips and 180 rotations (same trick, one more bit). Coefficient stability near zero. Video: keyframes vs temporally averaged frames vs 3D-DCT. The seen-images daemon idea. Sorting a folder by visual similarity.</p>
|
||||
</article>
|
||||
</div>
|
||||
<script type="module" src="script.js"></script>
|
||||
|
||||
Reference in New Issue
Block a user