Checkmate - DeDot - LUTDeCrawl - TComb

( Home - Spatial Smoothers - Temporal Smoothers - Spatio-Temporal Smoothers - Sharpeners - Dealing with Rainbows )

... Oh, who am I kidding? Compared to today's technologies, it sucked. Especially because it was prone to crosstalk artifacts: that is, dot crawl (in the luma) and rainbows (in the chroma). Sadly, some animé series (mostly older ones) have been transferred to DVD in such a way that these wonderful artifacts were introduced along the way and hardcoded into the video stream, so that you can't get rid of them on playback no matter how good your cables are. To make it even more annoying, even though we have AVISynth filters for dealing with these problems, they usually have to be applied BEFORE any inverse telecine (IVTC) operations in order to be effective. Those of you who edit directly with scripts will appreciate what a problem that is.

I'm sure you've all seen dot crawl at some point in your lives; for those unfamiliar with the term, it looks like this. It only shows up in the luma (Y) plane and consists of a checkerboard pattern that typically flickers back and forth every frame. Sometimes it may not be noticeable when you're just playing a DVD, but then you'll see it when you stop on a still frame. Thankfully, most modern animé releases do not have this problem; but if your source does, then this is the page for you.

I'm only going to bother with one comparison screenshot for this entire page, and I'm going to put it right up front, because all of these filters should have the same effect when things go right. It's from the opening sequence for X, which oddly enough was plagued with dot crawl despite the actual episodes themselves being pretty clean (hover mouse over image to see filtered version):

• thr (any nonnegative int) controls the spatial filter; higher values will (AFAICT) blend more.
• max (any positive int): actually, mf doesn't remember what this does. It probably controls the maximum amount by which a pixel's value may change, or something similar. In any case, increasing it will strengthen the filtering.
• tthr2 (any nonnegative int) controls the temporal blending; higher values will blend more. Setting this to 0 disables the temporal blending entirely.

Usage notes:

• Usually, dot crawl removal filters should be called BEFORE any inverse telecine (IVTC) operations. Try it this way first, and if every other frame or so still contains the dots, then try it after IVTC.
• Checkmate will NOT give you any warnings if you feed it video in anything other than the YV12 colorspace, and unexpected behaviour will result if you try. Thankfully, DVD video served by MPEG2Source() is virtually always YV12 anyway (unless you specify upconv=true), so this should not be an issue.

• luma2d (0 to 255) is the spatial threshold for the luma; what I believe this means is that this is the minimum difference between a pixel and its spatial neighbours for the pixel to be targeted. Lower values will detect more pixels as dot crawl, but may cause artifacts.
• lumaT (0 to 255) is the tolerance for temporal changes in the luma; I believe this refers to how close a pixel's two temporal neighbours can be to each other for the pixel to be targeted. Higher values will detect more pixels as dot crawl (but may cause artifacts).
• chromaT1 and chromaT2 (0 to 255) have to do with rainbowing, not dot crawl, and therefore will not be discussed on this page. If you intend to use DeDot to remove only dot crawl, then keep these at their recommended values (chromaT1=0, chromaT2=255).

• Usually, dot crawl removal filters should be called BEFORE any inverse telecine (IVTC) operations. Try it this way first, and if every other frame or so still contains the dots, then try it after IVTC.

• ythresh (0-255) determines how close the luma values of each pixel's two temporal neighbours have to be to each other for the pixel to be targeted. Higher values will detect more pixels as dot crawl, but may cause artifacts. I don't recommend higher than 20 or so.
• cthresh (0-255) determines how close the chroma values of each pixel's two temporal neighbours have to be to each other for the pixel to be targeted. Just as with ythresh.
• maxdiff (0-255) is the maximum difference allowed between the luma values of a pixel and each of its two temporal neighbours (so, the upper limit to what fluctuations are considered dot crawl). This is meant as an artifact-reducing measure; lower values will reduce artifacts but may cause the filter to miss some dot crawl. Obviously, this should never be lower than ythresh. If usemaxdiff=false, then this parameter is ignored.
• scnchg (0-255) is the scene change detection threshold; if the total luma difference between a frame and either of its temporal neighbours is greater than this value, then the frame will not be processed at all. (This is why the filter works very fast in high-motion clips: it doesn't take much processing to pass a frame through unchanged.)
• usemaxdiff (true/false) determines whether or not to cap luma fluctuations at maxdiff. Turning this off is not recommended, as it may introduce artifacts; but on the other hand, it produces a speed boost of about 30%. Test on your particular source; maybe you can get away with it.
• mask (true/false) -- When this is set true, the function will return the mask instead of the image: white pixels are targeted for correction, black pixels are not. Use this to find optimal values for the other parameters (but be aware that the scene change threshold, scnchg, is NOT reflected in the mask).

• Usually, dot crawl removal filters should be called BEFORE any inverse telecine (IVTC) operations. Try it this way first, and if every other frame or so still contains the dots, then try it after IVTC.
• This filter is more prone to artifacts at default settings than some others and may require tweaking. In particular, the parameter scnchg is your friend -- use it!

But, since there are no spatial considerations in this algorithm, this filter is more prone to artifacts than some others... so I added in the capping of fluctuations and the scene change detection in order to cut down on artifacts. However, that still doesn't change the fact that this filter will usually miss dot crawl on moving targets (like DeDot). Back to top

• mode (0 to 2) determines whether TComb should be used to remove dot crawl, rainbows, or both. 0 filters for dot crawl only, 1 filters for rainbows only, and 2 filters for both.
• fthreshl and fthreshc (0 to 255) are the filtered pixel correlation thresholds for luma and for chroma, respectively. What I believe this means is that if all pixels within the filtering window are within these values of each other after they have been filtered, then they are considered correlated. Only fthreshl matters for dot crawl. Higher values will detect more pixels as dot crawl, but may cause artifacts.
• othreshl and othreshc (0 to 255) are the original pixel correlation thresholds for luma and for chroma, respectively. If the original pixels in every other frame differ by less than these values, then they are considered correlated. Only othreshl matters for dot crawl. Higher values will detect more pixels as dot crawl, but may cause artifacts.
• map (true/false) -- When this is set true, the filter will return a map showing which pixels are being replaced with filtered values. A value of 0 in the given plane (Y/U/V) means the pixel is not being filtered in that plane; anything else means it is. (So, pixels that are not being filtered for dot crawl or rainbows will show up as dark green.) For details, see the documentation.
• scthresh (float 0.0-100.0) is the scene change detection threshold, as a percentage of maximum change on the luma plane. Higher values will allow filtering on more frames, but may cause artifacts. Based on the documentation, I can only assume that (just as with LUTDeCrawl) the scene change detection is not figured into the map that is displayed when map=true.

• Usually, dot crawl removal filters should be called BEFORE any inverse telecine (IVTC) operations. Try it this way first, and if every other frame or so still contains the dots, then try it after IVTC.
• Tritical includes in the documentation a handy guide to tweaking TComb for dot crawl removal, which I quote in its entirety here:
  "To find good values for fthreshL/othreshL, start with the following line:

  tcomb(mode=0,fthreshL=255,othreshL=255)

  Now, keep othreshL at 255 but set fthreshL down to 1. Keep increasing fthreshL in steps of 1 to 2 until you find the point at which all dot crawl is removed. Remember that value. Next, set fthreshL back to 255, and set othreshL to 1. Now, increase othreshL in steps of 1 or 2 until you find the point at which all dot crawl is removed. You've now got values for fthreshL/othreshL."