201 lines
9.0 KiB
HTML
201 lines
9.0 KiB
HTML
<html>
|
|
<head>
|
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
<meta content="text/html;charset=utf-8" http-equiv="Content-Type"/>
|
|
<link rel="stylesheet" href="colors-default.css" type="text/css">
|
|
<link rel="stylesheet" href="colors.css" type="text/css">
|
|
<link rel="stylesheet" href="style.css" type="text/css">
|
|
<link rel="stylesheet" href="/highlight.css">
|
|
<script src="highlight/highlight.min.js"></script>
|
|
</head>
|
|
<body>
|
|
<div id="wrapper">
|
|
<article>
|
|
<h1>Fucking around with perceptual hashes</h1>
|
|
<h2>Introduction</h2>
|
|
<p class="todo">TODO: why this project exists, the master's thesis that wasn't, discovering pHash after the fact.</p>
|
|
<p class="todo">TODO: the rules of the game: 64 bits per image, hamming distance, nothing else.</p>
|
|
</article>
|
|
|
|
<article>
|
|
<h1>Perceptual hashing</h1>
|
|
<p class="todo">TODO: cryptographic hash vs perceptual hash. One flipped pixel: md5 avalanches, a perceptual hash shrugs.</p>
|
|
<h2>Terminology</h2>
|
|
<p class="todo">TODO: descriptor / fingerprint / perceptual hash / signature all mean roughly the same thing depending on which corner of the literature you are standing in. Copy detection vs near-duplicate detection vs LSH.</p>
|
|
<h2>Subjective image similarity</h2>
|
|
<p class="todo">TODO: what "the same image" even means. Same pixels? Same scene? Same vibe?</p>
|
|
</article>
|
|
|
|
<article id="demo">
|
|
<h1>Demo</h1>
|
|
<p>Select an image to run this demo with. Don't worry, nothing will be sent to any server! All calculations are done in the browser.</p>
|
|
<p>Unanimated images with no transparancy work best, but feel free to experiment.</p>
|
|
<input
|
|
type="file"
|
|
id="dctimage"
|
|
accept="image/*" />
|
|
<span>or pick a sample:</span>
|
|
<span id="sample-images"></span>
|
|
<div id="base">
|
|
<div class="image-original"></div>
|
|
</div>
|
|
</article>
|
|
|
|
<article id="demo-resize" class="needs-image">
|
|
<h2>Step 1: Resize</h2>
|
|
<p>
|
|
The first step is to size the image down, and remove all color information.
|
|
Converting the image to grayscale is done by simply averaging the pixels.
|
|
Since we are aiming for a 64-bit descriptor, the logical target size is 8 by 8 since that will give us 64 pixels to work with.
|
|
</p>
|
|
<p>
|
|
It is obvious, but worth noting, that this action is <em>destructive</em>.
|
|
We are throwing away a lot of information here, especially regarding the finer details.
|
|
For now, this is a good thing.
|
|
</p>
|
|
<p>
|
|
Here we use a nearest neighbour downscaling algorithm that does not preserve aspect ratio.
|
|
Press the button to resize your image: <br />
|
|
<button id="resize-button">Resize</button>
|
|
</p>
|
|
<div id="resize">
|
|
<div class="image-original" id="resize-original"></div>
|
|
<div id="image-resize"></div>
|
|
</div>
|
|
</article>
|
|
|
|
<article id="demo-median" class="needs-image">
|
|
<h2>Step 2: The simplest hash that could possibly work</h2>
|
|
<p class="todo">TODO: median hash: one bit per pixel, brighter than the median or not. This is the "Median" baseline from Thomee et al.</p>
|
|
<div class="hash-panel">
|
|
<div>
|
|
<h3>Bits</h3>
|
|
<div id="median-bits" class="bit-grid"></div>
|
|
</div>
|
|
<div>
|
|
<h3>Hash</h3>
|
|
<code id="median-hash" class="hash-value"></code>
|
|
</div>
|
|
</div>
|
|
<p class="todo">TODO: why this breaks: global median shifts, flips scramble everything. Foreshadow the moon/plate anecdote.</p>
|
|
</article>
|
|
|
|
<article id="demo-dct" class="needs-image">
|
|
<h2>Step 3: Frequency space</h2>
|
|
<p class="todo">TODO: what the DCT does. JPEG uses the same trick. Low frequencies = global shape, high frequencies = detail we already threw away.</p>
|
|
<div class="hash-panel">
|
|
<div>
|
|
<h3>DCT coefficients</h3>
|
|
<div id="image-dct" class="heat-grid"></div>
|
|
</div>
|
|
</div>
|
|
<p class="todo">TODO: reading the coefficient grid: top-left is the average, first row is horizontal waves, first column vertical waves.</p>
|
|
<h3>From coefficients to bits</h3>
|
|
<p class="todo">TODO: sign bits of the first 36 zigzag coefficients + 28 ordinal comparisons between neighbours in zigzag order. The odd-column signs get multiplied by the sign of coefficient (1,0), which buys horizontal flip invariance for one bit.</p>
|
|
<div class="hash-panel">
|
|
<div>
|
|
<h3>Sign mask (36)</h3>
|
|
<div id="dct-sign-bits" class="bit-grid wide"></div>
|
|
</div>
|
|
<div>
|
|
<h3>Ordinal mask (28)</h3>
|
|
<div id="dct-ordinal-bits" class="bit-grid wide"></div>
|
|
</div>
|
|
<div>
|
|
<h3>Hash</h3>
|
|
<code id="dct-hash" class="hash-value"></code>
|
|
</div>
|
|
</div>
|
|
</article>
|
|
|
|
<article id="demo-phash" class="needs-image">
|
|
<h2>Step 4: How pHash does it</h2>
|
|
<p class="todo">TODO: same idea, different route: resize to 32x32 instead of 8x8, DCT, keep the top-left 8x8 block of low frequencies, threshold against the median coefficient.</p>
|
|
<div class="hash-panel">
|
|
<div>
|
|
<h3>32×32</h3>
|
|
<div id="phash-resize"></div>
|
|
</div>
|
|
<div>
|
|
<h3>Low-frequency block</h3>
|
|
<div id="phash-lowfreq" class="heat-grid"></div>
|
|
</div>
|
|
<div>
|
|
<h3>Bits</h3>
|
|
<div id="phash-bits" class="bit-grid"></div>
|
|
</div>
|
|
<div>
|
|
<h3>Hash</h3>
|
|
<code id="phash-hash" class="hash-value"></code>
|
|
</div>
|
|
</div>
|
|
<p class="todo">TODO: what's genuinely different between our dct hash and phash (bit extraction, flip invariance) and what isn't (everything else).</p>
|
|
</article>
|
|
|
|
<article id="demo-mutate" class="needs-image">
|
|
<h2>Mutations</h2>
|
|
<p class="todo">TODO: a copy is rarely byte-identical. Recoding, resampling, content processing, framing, inserted logos (the Thomee et al. taxonomy). Sweep the slider and watch which hash survives what.</p>
|
|
<p>
|
|
<select id="mutator-kind"></select>
|
|
<input type="range" id="mutator-amount" />
|
|
<span id="mutator-amount-value"></span>
|
|
</p>
|
|
<div id="mutate-compare" class="compare">
|
|
<img id="compare-a" alt="original" />
|
|
<img id="compare-b" alt="mutated" />
|
|
</div>
|
|
<p>
|
|
<input type="range" id="compare-slider" min="0" max="100" value="50" />
|
|
</p>
|
|
<table class="distances">
|
|
<tr><th></th><th>dct</th><th>median</th><th>phash</th></tr>
|
|
<tr>
|
|
<th>hamming distance</th>
|
|
<td id="dist-dct"></td>
|
|
<td id="dist-median"></td>
|
|
<td id="dist-phash"></td>
|
|
</tr>
|
|
</table>
|
|
<p class="todo">TODO: what counts as "the same" now? Thresholds. pHash uses 22 of 64 bits, we will measure our own.</p>
|
|
</article>
|
|
|
|
<article id="demo-ranking">
|
|
<h2>Find the copy</h2>
|
|
<p class="todo">TODO: the fun part: a tiny search engine. Add a pile of images, click one, get the nearest neighbours per algorithm. Mention the white plate that matched the moon.</p>
|
|
<p>
|
|
<input type="file" id="ranking-files" accept="image/*" multiple />
|
|
<select id="ranking-algo">
|
|
<option value="0">dct</option>
|
|
<option value="1">median</option>
|
|
<option value="2">phash</option>
|
|
</select>
|
|
</p>
|
|
<div id="ranking-pool" class="thumb-grid"></div>
|
|
<h3>Results</h3>
|
|
<div id="ranking-results" class="thumb-grid"></div>
|
|
</article>
|
|
|
|
<article>
|
|
<h2>At scale</h2>
|
|
<p class="todo">TODO: the browser demo is anecdote, this section is data. 25k images, 16 mutations each, precision-recall over the hamming threshold.</p>
|
|
<figure>
|
|
<img src="" alt="PR curves for the dct hash" />
|
|
<figcaption class="todo">TODO: dct-pr.png from the experiment run</figcaption>
|
|
</figure>
|
|
<figure>
|
|
<img src="" alt="PR curves for median and phash" />
|
|
<figcaption class="todo">TODO: median-pr.png and phash-pr.png</figcaption>
|
|
</figure>
|
|
<p class="todo">TODO: how the PR curves are computed, what a false positive means here, where the thresholds land per algorithm.</p>
|
|
</article>
|
|
|
|
<article>
|
|
<h2>Loose ends</h2>
|
|
<p class="todo">TODO: vertical flips and 180 rotations (same trick, one more bit). Coefficient stability near zero. Video: keyframes vs temporally averaged frames vs 3D-DCT. The seen-images daemon idea. Sorting a folder by visual similarity.</p>
|
|
</article>
|
|
</div>
|
|
<script type="module" src="script.js"></script>
|
|
<script>hljs.highlightAll();</script>
|
|
</body>
|
|
</html>
|