edu.umich.eecs.april.image

Class AprilTagFilter

java.lang.Object

com.jhlabs.image.AbstractBufferedImageOp

edu.umich.eecs.april.image.AprilTagFilter

All Implemented Interfaces:

java.awt.image.BufferedImageOp, java.lang.Cloneable

public class AprilTagFilter
extends AbstractBufferedImageOp

An AprilTag detector image filter. This filter identifies April tags (https://april.eecs.umich.edu/wiki/index.php/AprilTags), which are two-dimensional images, similar to QR codes, but easier to detect. The options for the filter are the ones that have setter methods. For example, the TagFamily option as a setTagFamily() method. https://april.eecs.umich.edu/software/apriltag.html contains a set of pregenerated tags as png and PostScript files. However, these are of low resolution. To scale them, use linear interpolation to avoid blurring. For example, with ImageMagik, use: mogrify -scale 1000x1000 *.png Or, search the web for "tag 36H11". To create a pdf with all the images: convert *.png tag36h11.pdf To annotate an image with a string: convert tag36_11_00026.png label:'26' -gravity Center -append tag36_11_00026_labeled.png In the Ptolemy tree, a sample file may be found at $PTII/ptolemy/actor/lib/jjs/modules/aprilTags//demo/AprilTags/tag36_11_00586.pdf

Since:

Ptolemy II 11.0

Version:

$Id$

Author:

Edwin Bolson and Edward A. Lee

Pt.AcceptedRating:

Red (cxh)

Pt.ProposedRating:

Yellow (eal)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
(package private) static class	AprilTagFilter.GrayModel Fits a grayscale model over an area of the form: Ax + By + Cxy + D = value We use this model to compute spatially-varying thresholds for reading bits.
(package private) class	AprilTagFilter.Quad Represents four segments that form a loop, and might be a tag.
(package private) class	AprilTagFilter.Segment Represents a line fit to a set of pixels whose gradients are similar.

Field Summary

Fields

Modifier and Type	Field and Description
boolean	debug Produce debugging output.
java.awt.image.BufferedImage	debugInput
java.awt.image.BufferedImage	debugMag
java.awt.image.BufferedImage	debugSegInput
java.awt.image.BufferedImage	debugSegmentation
java.awt.image.BufferedImage	debugTheta
double	maxQuadAspectRatio Early pruning of quads which have insane aspect ratios.
double	minimumLineLength Minimum line length in pixels.
double	minimumSegmentSize Minimum number of pixels in a segment before we'll fit a line to it.
double	minimumTagSize Minimum size of tag (in pixels) as measured along edges and diagonals.
double	quadSearchRangeFraction
double	quadSearchRangePix
int	WEIGHT_SCALE During segmentation, the weight of an edge is related to the change in theta between the two pixels.

Constructor Summary

Constructors

Constructor and Description
AprilTagFilter()

Method Summary

Modifier and Type	Method and Description
java.awt.image.BufferedImage	filter(java.awt.image.BufferedImage source, java.awt.image.BufferedImage destination) Filter the source image, and if motion is detected, place a red circle at the center of gravity of the motion.
double	getMagThresh() When growing components, the intra component variation is allowed to grow when the component is small in size.
double	getMaxEdgeCost() Return the maximum angle range allowed for the gradient directions when connecting edges, in radians.
double	getMinMag() Return the gradient magnitude threshold for ignoring pixels.
double	getSegSigma() Return the Gaussian smoothing kernel applied to image (0 == no filter) used when detecting the outline of the box.
double	getSigma() Return the Gaussian smoothing kernel applied to image (0 == no filter, the default) used when sampling bits.
java.lang.String	getTagFamily() Get the name of the tag family being detected.
double	getThetaThresh() When growing components, the intra component variation is allowed to grow when the component is small in size.
boolean	isSegDecimate() Return whether decimating before segmenting is enabled.
void	setMagThresh(double magThresh) When growing components, the intra component variation is allowed to grow when the component is small in size.
void	setMaxEdgeCost(double maxEdgeCost) Set the maximum angle range allowed for the gradient directions when connecting edges, in radians.
void	setMinMag(double minMag) Set the gradient magnitude threshold for ignoring pixels.
void	setSegDecimate(boolean segDecimate) Set whether decimating before segmenting is enabled.
void	setSegSigma(double segSigma) Set the Gaussian smoothing kernel applied to image (0 == no filter) used when detecting the outline of the box.
void	setSigma(double sigma) Set the Gaussian smoothing kernel applied to image (0 == no filter, the default) used when sampling bits.
void	setTagFamily(java.lang.String name) Set the name of the tag family being detected.
void	setThetaThresh(double thetaThresh) When growing components, the intra component variation is allowed to grow when the component is small in size.
ArrayToken	tags() Return an array of records representing the most recently detected tags from an invocation of filter(BufferedImage, BufferedImage), or null if there has been no such invocation or if no tags were detected.

Methods inherited from class com.jhlabs.image.AbstractBufferedImageOp

clone, createCompatibleDestImage, getBounds2D, getPoint2D, getRenderingHints, getRGB, setRGB, stringToColor

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

maxQuadAspectRatio

public double maxQuadAspectRatio

Early pruning of quads which have insane aspect ratios.

minimumLineLength

public double minimumLineLength

Minimum line length in pixels. Set based on minimum plausible decoding size for Tag9 family.

minimumSegmentSize

public double minimumSegmentSize

Minimum number of pixels in a segment before we'll fit a line to it.

minimumTagSize

public double minimumTagSize

Minimum size of tag (in pixels) as measured along edges and diagonals.

quadSearchRangePix

public double quadSearchRangePix

quadSearchRangeFraction

public double quadSearchRangeFraction

debug

public boolean debug

Produce debugging output. If the debugging code annoys you (or makes porting harder) you can delete all of the code in an if (debug) block. NOTE: Most of the debugging code is disabled in this implementation.

debugInput

public java.awt.image.BufferedImage debugInput

debugSegInput

public java.awt.image.BufferedImage debugSegInput

debugSegmentation

public java.awt.image.BufferedImage debugSegmentation

debugTheta

public java.awt.image.BufferedImage debugTheta

debugMag

public java.awt.image.BufferedImage debugMag

WEIGHT_SCALE

public int WEIGHT_SCALE

During segmentation, the weight of an edge is related to the change in theta between the two pixels. This change is normalized via maxEdgeCost, resulting in a number[0,1]. We then convert this number to fixed-point by multiplying it by WEIGHT_SCALE. The resulting number must fit in the number of bits allocated to it, currently 16 (see Step 3). Large values of WEIGHT_SCALE are good because they lead to better fixed-point approximations. However, small numbers are better, because there are fewer discrete values of edge costs, which means that there is more spatial/temporal coherence when processing the sorted edges. This results in faster processing. Given that these orientations are pretty noisy to begin with, some quantization error is acceptable.

Constructor Detail

AprilTagFilter

public AprilTagFilter()

Method Detail

filter

public java.awt.image.BufferedImage filter(java.awt.image.BufferedImage source,
                                           java.awt.image.BufferedImage destination)

Filter the source image, and if motion is detected, place a red circle at the center of gravity of the motion. If the destination argument is null, then the red circle is added directly to the source, and the source is returned. Otherwise, the red circle is added to the destination image. In addition, this method records the tags discovered so that they can later by retrieved by invoking the tags() method.

Parameters:

source - The source image, on which motion is detected.

destination - The destination image, on which the red circle is added, or null to specify to add the circle to the source image.

getMagThresh

public double getMagThresh()

When growing components, the intra component variation is allowed to grow when the component is small in size. This threshold affects how much. The default is 1200.

Returns:

The mag threshold

See Also:

setMagThresh(double)

getMaxEdgeCost

public double getMaxEdgeCost()

Return the maximum angle range allowed for the gradient directions when connecting edges, in radians. This defaults to the radian equivalent of 30 degrees.

Returns:

The maximum angle range allowed.

See Also:

setMaxEdgeCost(double)

getMinMag

public double getMinMag()

Return the gradient magnitude threshold for ignoring pixels. Do not consider pixels whose gradient magnitude is less than minMag. Small values make the detector more sensitive, but also force us to consider many more edges resulting in slower computation time. A value of 0.001 is very sensitive. A value of 0.01 is quite fast. The default is 0.004.

See Also:

setMinMag(double)

getSegSigma

public double getSegSigma()

Return the Gaussian smoothing kernel applied to image (0 == no filter) used when detecting the outline of the box. It is almost always useful to have some filtering, since the loss of small details won't hurt. Recommended value = 0.8 (the default). The case where sigma == segsigma has been optimized to avoid a redundant filter operation.

See Also:

setSegSigma(double)

getSigma

public double getSigma()

Return the Gaussian smoothing kernel applied to image (0 == no filter, the default) used when sampling bits. Filtering is a good idea in cases where A) a cheap camera is introducing artifical sharpening, B) the bayer pattern is creating artifcats, C) the sensor is very noisy and/or has hot/cold pixels. However, filtering makes it harder to decode very small tags. Reasonable values are 0, or [0.8, 1.5].

See Also:

setSigma(double)

getTagFamily

public java.lang.String getTagFamily()

Get the name of the tag family being detected. This defaults to "Tag36h11". The supported families are "Tag16h5", "Tag25h7", "Tag25h9", "Tag36h10", and "Tag36h11". The default family seems least susceptible to false positives.

Returns:

The name of the tag family.

See Also:

setTagFamily(String)

getThetaThresh

public double getThetaThresh()

When growing components, the intra component variation is allowed to grow when the component is small in size. This threshold affects how much. The default is 100.

See Also:

setThetaThresh(double)

isSegDecimate

public boolean isSegDecimate()

Return whether decimating before segmenting is enabled. Instead of blurring the input image before segmentation, we can achieve similar effects by decimating the image by a factor of two. When enabled, this option applies a block LPF filter of width 2, then decimates the image. With this option, not only can we safely set segSigma = 0, but the slowest part of the algorithm (the segmentation) runs about 4 times faster. The downside is that the position of the targets is determined based on the segmentation: lower resolution will result in more localization error. However, the effect on quality is quite modest, and this optimization is generally recommended (along with segSigma = 0). If segSigma is non-zero, the filtering by segSigma occurs first, followed by the block LPF, and the decimation. This defaults to false, indicating that the option is not enabled.

See Also:

setSegDecimate(boolean)

setMagThresh

public void setMagThresh(double magThresh)

When growing components, the intra component variation is allowed to grow when the component is small in size. This threshold affects how much. The default is 1200.

See Also:

getMagThresh()

setMaxEdgeCost

public void setMaxEdgeCost(double maxEdgeCost)

Set the maximum angle range allowed for the gradient directions when connecting edges, in radians. This defaults to the radian equivalent of 30 degrees.

See Also:

getMaxEdgeCost()

setMinMag

public void setMinMag(double minMag)

Set the gradient magnitude threshold for ignoring pixels. Do not consider pixels whose gradient magnitude is less than minMag. Small values make the detector more sensitive, but also force us to consider many more edges resulting in slower computation time. A value of 0.001 is very sensitive. A value of 0.01 is quite fast. The default is 0.004.

See Also:

getMinMag()

setSegDecimate

public void setSegDecimate(boolean segDecimate)

Set whether decimating before segmenting is enabled. Instead of blurring the input image before segmentation, we can achieve similar effects by decimating the image by a factor of two. When enabled, this option applies a block LPF filter of width 2, then decimates the image. With this option, not only can we safely set segSigma = 0, but the slowest part of the algorithm (the segmentation) runs about 4 times faster. The downside is that the position of the targets is determined based on the segmentation: lower resolution will result in more localization error. However, the effect on quality is quite modest, and this optimization is generally recommended (along with segSigma = 0). If segSigma is non-zero, the filtering by segSigma occurs first, followed by the block LPF, and the decimation. This defaults to false, indicating that the option is not enabled.

setSegSigma

public void setSegSigma(double segSigma)

Set the Gaussian smoothing kernel applied to image (0 == no filter) used when detecting the outline of the box. It is almost always useful to have some filtering, since the loss of small details won't hurt. Recommended value = 0.8 (the default). The case where sigma == segsigma has been optimized to avoid a redundant filter operation.

See Also:

getSegSigma()

setSigma

public void setSigma(double sigma)

Set the Gaussian smoothing kernel applied to image (0 == no filter, the default) used when sampling bits. Filtering is a good idea in cases where A) a cheap camera is introducing artifical sharpening, B) the bayer pattern is creating artifcats, C) the sensor is very noisy and/or has hot/cold pixels. However, filtering makes it harder to decode very small tags. Reasonable values are 0, or [0.8, 1.5].

See Also:

getSigma()

setTagFamily

public void setTagFamily(java.lang.String name)

Set the name of the tag family being detected. This defaults to "Tag36h11". The supported families are "Tag16h5", "Tag25h7", "Tag25h9", "Tag36h10", and "Tag36h11". The default family seems least susceptible to false positives.

Parameters:

name - The name of the tag family.

See Also:

getTagFamily()

setThetaThresh

public void setThetaThresh(double thetaThresh)

When growing components, the intra component variation is allowed to grow when the component is small in size. This threshold affects how much. The default is 100.

See Also:

getThetaThresh()

tags

public ArrayToken tags()
                throws IllegalActionException

Return an array of records representing the most recently detected tags from an invocation of filter(BufferedImage, BufferedImage), or null if there has been no such invocation or if no tags were detected. Each record includes the following fields:

• id: The ID of the detected tag.
• center: An array with two doubles giving the center of the tag in pixel coordinates.
• perimeter: An array with four arrays, each of which gives the x and y coordinates of a corner of the AprilTag, listed in a counter-clockwise direction.

Throws:

IllegalActionException - If the array cannot be constructed.