Why does my GND density change when I change the step size?

What KAM actually measures

Kernel average misorientation is the simplest useful thing you can compute from an orientation map. For every pixel, take its neighbours, work out the misorientation to each one, throw away any pair further apart than a threshold (5° is the usual choice, so grain boundaries do not contaminate the number), and average what is left. One number per pixel, and it lights up exactly where the lattice is bent.

The specimen below is a synthetic polycrystal, the same one all three panels on this page use: grains with their own orientations and their own lattice curvature, sub-grain cells inside them, and a slice cut through it at a depth you choose. The pale pixels in the KAM map are where every neighbour pair was thrown away, which is to say the grain boundaries. That exclusion is the threshold doing its job, and it is also the first hint that the kernel has a size it must stay under. The default specimen is a fine-grained one, 150 grains in a 2.5 µm cube, so the grains are around 500 nm across and there is a great deal of boundary in the way.

The trouble starts at the next step, when that number is turned into a dislocation density. The standard route is ρ = αθ/(nδb), where θ is the KAM value, δ is the step size, n is how many neighbour shells the kernel spans, b is the Burgers vector, and α is 2 for a pure tilt arrangement, 4 for pure twist, 3 for mixed. Every symbol in there is something you chose except θ and b, and one of the things you chose is the step size.

Set the angular precision to perfect indexing, the sub-grain cells to none, and drag the step size. KAM itself climbs, doubling as the step doubles, because a bigger step spans more of the same bend. The dislocation density it implies does not move: 45 nm and 90 nm return the identical number. That cancellation is the whole reason the formula is allowed to exist.

Keep dragging and it falls apart, and not because of noise, because there is none. Watch the third number, the fraction of neighbour pairs thrown away by the 5° threshold. It runs 12%, 24%, 46%, 79%, and then 100%. The grains in this specimen are around 500 nm across, so by a 360 nm step most of what the kernel reaches for is on the other side of a boundary, and at 720 nm every single pair is. The density follows it down and then reaches zero: not a small number, nothing at all, because there is no pair left to average.

So on this material the step cancellation is exact over about one doubling and then stops being true. That is not a defect of the simulation, it is what a fine-grained material does to a kernel method. Make the grains bigger and the window widens in proportion; the ratio that matters is the kernel size against the grain size, not either one on its own.

One subtlety worth carrying: a four-neighbour kernel averages over two directions, and a bend that runs in only one of them contributes to half the pairs. The measured KAM comes out at half the true curvature times the step, so the implied density is half the density you would get from the curvature directly. That factor is a property of your kernel, not of your specimen. It is the first sign that this number is a relative measure.

The number that should not move, and does

Real orientation data carries measurement noise, and noise does not care about your step size. A bend contributes κδ to the misorientation between neighbours, which shrinks as the step shrinks. The noise contributes the same amount whatever the step. So as you refine the step, the ratio of signal to noise in every single neighbour pair gets worse, and because the formula divides by δ, the noise you just measured gets multiplied by 1/δ on the way out.

The floor line is not a model. It is the same kernel, the same formula and the same noise, run on the same grain structure with the curvature switched off: whatever it reports is noise by construction. It falls roughly as 1/δ and scales linearly with your angular precision. Only roughly, because the microstructure interferes: as the step coarsens, more of the kernel's pairs cross grain boundaries and get thrown away, so the fitted slope drifts above 1, and drifts further the finer the grain structure. On this specimen it runs from about 1.1 at 100 grains to 1.6 at 400. A real noise floor is not a clean power law either, for exactly this reason.

Put the two limits together and what you have is a window, not a direction. Too fine and you are measuring your indexing routine: at 45 nm with Hough-grade indexing this specimen reports around twenty times its real dislocation density. Too coarse and the kernel is eating your grains, until at 720 nm the map returns nothing at all. Between those, at 360 nm, the measurement finally clears twice its own noise floor, and by then it is reading about half the true density because the kernel is already discarding four pairs in five.

That is the uncomfortable part. On a material with 500 nm grains and half a degree of angular precision, there is no step size that is both quiet enough and small enough to be right. Every setting on that axis is wrong, in one direction or the other, and the map looks equally plausible at all of them.

The three precision presets are not invented. Conventional Hough-based indexing lands near half a degree; refined template or dictionary matching reaches about 0.1°, measured on a silicon single crystal against roughly 0.4° for Hough on the same patterns; and high angular resolution EBSD, which cross-correlates against a reference pattern, reaches 1×10⁻⁴ rad, about 0.006°. Two orders of magnitude between the ends of that list, and the noise floor moves with it one for one.

The practical rule this gives you: pick the step so that the real misorientation across one step comfortably exceeds your angular precision, then check that the kernel still fits inside your grains. With half a degree of noise and a lattice bending at 2° per micron, one step has to be a good fraction of a micron before the bend is even visible, which is far coarser than the step most people scan at. Scanning finer does not buy resolution here; it buys noise, amplified. Scanning coarser stops working as soon as the kernel starts spanning boundaries. When the two conditions do not overlap, and on a fine-grained material they often do not, KAM is the wrong tool for that specimen and no amount of tuning makes it the right one. Better angular precision is what opens the window, not a better choice of step: HR-EBSD moves the noise limit down by two orders of magnitude and leaves the grain-size limit exactly where it was.

What one map cannot see, whatever the step

Everything above is about precision. This section is about something that no amount of precision fixes. The dislocation content of a lattice is a tensor, Nye's α_ij, and it is built from the lattice curvature κ_ij = ∂ω_i/∂x_j by α_ij = κ_ji − δ_ijκ_kk. An EBSD map is a surface. It gives you orientation gradients in the two in-plane directions and nothing at all in the third.

Below is a synthetic polycrystal, built as a volume rather than a picture: Voronoi grains filling a 2 µm cube, each grain carrying its own orientation and its own lattice curvature, with sub-grain cell walls inside them. Because the volume exists, the Nye tensor can be computed voxel by voxel from all three gradient directions, and that is the ground truth. Then one slice is cut through it, the way a scan samples a polished surface, and the same tensor is computed again from that slice alone. Move the slice up and down and you are looking at maps a scan could actually have returned from the same specimen.

The tensor is only computed where the whole differencing stencil sits inside one grain. Across a boundary the orientation gradient is not a lattice curvature at all, it is a discontinuity, and the Nye tensor is undefined there. That is the pale border you can see around every grain in the third map, and it is a real limitation of GND mapping rather than a shortcut taken here.

Five of the nine components survive the slice: α₁₂, α₁₃, α₂₁, α₂₃ and α₃₃. The last of those is the interesting one, because κ₃₃ is not measurable but the trace lets you write α₃₃ = −κ₁₁ − κ₂₂, and both of those are in the plane. The four that are lost, α₁₁, α₂₂, α₃₁ and α₃₂, each need a gradient normal to the surface, and there is no such gradient in a surface map. Set the curvature to a pure out-of-plane bend and the slice reports nothing at all while the volume reports a real dislocation content.

Because every grain is bent in its own direction, the answer is not one number but a distribution. A typical slice recovers a median of 70 to 80% of the dislocation content, and the spread within a single map is enormous: some grains come back at 95% and others at a fifth of their true content, side by side, in the same scan, indexed identically. Which grains do badly is not a property of the material and not something better scanning fixes. It depends only on how that grain's curvature happens to sit relative to the surface you polished.

Drag the slice depth and watch the unlucky tenth move. On this volume it ranges from 8% to 67% depending on where the cut is taken: the same specimen, the same instrument, the same operator, polished a few hundred nanometres deeper. If you have ever wondered why two GND maps of the same material do not agree, this is one of the reasons, and it is invisible from inside either map.

So what is the GND density you are quoting? A lower bound, from five of nine components, under an assumption about dislocation character that fixes α, computed with a kernel whose geometry sets the prefactor, at a step size that sets the noise floor. That is a genuinely useful quantity for comparing one region of one map with another region of the same map, acquired the same way. It is not a dislocation count, and two GND maps from different laboratories are not comparable unless every one of those choices matches.

Sources & further reading

• J. F. Nye, “Some geometrical relations in dislocated crystals,” Acta Metallurgica 1, 153 (1953). The dislocation density tensor, and its relation to lattice curvature.
• W. Pantleon, “Resolving the geometrically necessary dislocation content by conventional electron backscattering diffraction,” Scripta Materialia 58, 994 (2008). Which components of the Nye tensor a surface map can and cannot reach.
• M. Calcagnotto, D. Ponge, E. Demir & D. Raabe, “Orientation gradients and geometrically necessary dislocations in ultrafine grained dual-phase steels studied by 2D and 3D EBSD,” Materials Science and Engineering A 527, 2738 (2010). The KAM route to a density, and the 2D against 3D comparison.
• T. B. Britton & A. J. Wilkinson, “Understanding deformation with high angular resolution electron backscatter diffraction,” IOP Conf. Ser. Mater. Sci. Eng. 304, 012003 (2018), arXiv:1710.00728. Relative misorientation to 1×10⁻⁴ rad, about two orders of magnitude finer than Hough-based indexing.
• S. Singh, F. Ram & M. De Graef, “Application of forward models to crystal orientation refinement,” and the refined template-matching literature more broadly: about 0.1° orientation precision after refinement, against roughly 0.4° for Hough indexing on the same silicon single-crystal patterns.
• The prefactor convention α = 2 for tilt, 4 for twist and 3 for mixed boundaries, and the normalised form ρ = (α/b)·dθ/dx, are set out in the KAM-to-density literature summarised by Baudin and co-workers.

What this page simulates and what it does not. The volume here is synthetic and its curvature is uniform within each cell, which makes the ground truth exact and the arithmetic checkable. A real specimen has curvature that varies on every length scale, dislocation cell walls that are not planes, and orientation noise that is correlated between neighbouring pixels rather than independent. The grains here are Voronoi cells and the sub-grain structure is drawn as rectangular blocks, neither of which is the shape real ones take. Correlated noise makes the floor shown here optimistic. The geometry of what a surface map can reach, which is the third section, does not depend on any of that.