ReduceFlicker 0.5

An Avisynth 2.5x plugin for reducing temporal oscillations from video clips

By Rainer Wittmann gorw@gmx.de

Binary and source  are subject to the GNU General Public License. Last change September 9, 2005.

Installation

There are three versions of the plugin. Firstly there is ReduceFlicker.dll, which requires only an integer SSE capable cpu and there are the ReduceFlickerSSE2.dll ReduceFlickerSSE3.dll  for SSE2 and SSE3 capable CPUs. Because ReduceFlickerSSE3.dll had to be compiled with the Intel compiler, there are compatibility issues of this plugin with other filters, but I use it personally. If you are not sure about the properties of your cpu, the freeware utility CPU-Z will help you. Install only one version of the above plugin in the Avisynth plugin directory and make sure that there is no more any old version in the plugin directory. All three versions of the plugin are dynamically linked and require msvcr71.dll (for Microsoft specific stuff) and AvsRecursion.dll (for recursion within Avisynth). If none of the above dlls works, probably one of these two libraries is missing. They should usually be installed in C:\windows\system32. Inquiries, comments etc. should be posted to the support forum. Registration is not necessary for posting and browsing in this forum, but it is certainly appreciated. Only in exceptional cases you should send me email. If you do that you must put the word "Avisynth" in the email header, because all other email is discarded (I receive hundreds of spam emails per day to the above address). I also check this email address only every second week. Thus the forum is really the best way for getting support.

Overview

The plugin contains three filters, ReduceFlicker, ReduceFluctuation and LockClense. ReduceFluctuation was previously part of the SSETools plugin, but because it serves a similar purpose as ReduceFlicker, it was moved to this plugin. LockClense is essentially the implementation of an idea of Ralph Boecker. All three filters are purely temporal and can be used for all color spaces including the planar color spaces introduced in RemoveGrain. As purely temporal filters both filters can be used for all kind of material, but they are particularily useful for interlaced footage or progressive videos from digicams like my Canon Powershot S1 IS. They should be applied before deinterlacing. If done so, the motion detection of the deinterlacer is fooled less by flicker, whence more detail is preserved.

ReduceFlicker

ReduceFlicker should be used as follows 

ReduceFlicker(clip input, int "strength", bool "aggressive", bool "grey", bool "planar")

Here "input" is the clip to be processed. With the "strength" variable (default = 2) one can specify the strength of ReduceFlicker. Higher values mean more aggressive operation. Currently three values 1,2,3 are allowed for "strength". If grey = true (false is the default value), then only the luma plane is processed. The other planes contain random garbage. Thus the internal filter greyscale() has to be applied later. The variable "grey" is only used for planar color spaces. If you use planar YUY2, RGB24, RGB32, then you have to set "planar=true" (false is the default value). 

ReduceFlicker(strength=1) makes use of 4 frames, the current frame, the subsequent frame and the two predecessors. Consequently, the first two frames and the last frame are left unchanged, because not all three neighbours are available for these frames. ReduceFlicker(strength=2) uses 5 frames, the current frame, the two sucessors and the two predecessors. Finally ReduceFlicker(strength=3) uses 7 frames, the current frame, the three sucessors and the three predecessors. Increasing "strength" makes ReduceFlicker more aggressive and slower. Beginning with version 0.5 the boolean variable "aggressive" (default value = false) has been introduced. If aggressive= true, then a significantly more aggressive variant of the algorithm is selected (the speed is the same as with aggressive=false).

ReduceFlicker performs controlled averaging of a frame with its two adjacent temporal neighbours. Controlled means that averaging only takes place in the presence of oscillations. Without such a restraint the filter would creat massive ghosts. To explain the algorithm in more detail, let c be the luma value of a pixel, p1, p2,p3 its temporal predecessors and n1,n2,n3 its temporal successors. Thus the values are observed in the sequence p3,p2,p1,c,n1,n2,n3. Then ReduceFlicker(strength=1) calculates d = |c - p2|,  a = max(c, min(p1, n1) - d), b = min(c, max(p1, n1) + d) and clips the average (p1 + n1 +2*c)/4 at a,b, i.e. c is replaced by max(b, min((p1 + n1 +2*c)/4, a)). Note that we alwayshave a > = c > = b and that always either a = c or b= c. When  |n1 -n2| or d= |c - p2| is large then we even have a = b = c. To illustrate the above formulas, let us look at some examples. The input sequence 10, 20, 10, 20, 10, 20 is converted into 10,20,15,15,15,20. The first two and the last value remain unchanged, while in the middle the oscillations are averaged away. This was the ideal case. If we have the less regular 11,21, 9, 20, 12, 19 then ReduceFlicker yields 11,21,15,15,16,19. On the other hand 11,13,9, 20,12,19 yields 11,13,9,19,16,19. The third and the fourth member are changed very much, because there is not enough oscillation. If strength > 1 only the value of d is changed it may become smaller. More precisely we have d = min(|c - p2|,  |c - n2|) if strength = 2 and d = min(|c - p2|,  (|c - p3|,  |c - n2|,  |c - n3|) ). From the above definition of a and b it follows that the clipping band gets smaller as the value of d increases. Thus as d gets larger, averaging is reduced more and more. The clipping band spanned by a and b also gets smaller if |min(p1, n1) - max(p1, n1)| = |p1 - n1| gets larger, whence ReduceFlicker only generates significant averaging if d and |p1 - n1| are small. In other words the value of a pixel has to alternate three times within a sequence of four frames to get averaged, if strength = 1,2 and within a sequence of five frames if strength = 3. This is the key idea, which evolved from a long thinking process about flicker. So far we have described ReduceFlicker if aggressive=false. If aggressive=true, then the important correction term d in the definition of a and b, which is very important to prohibit artifacts, is replaced by two smaller terms. If strength=1 and aggressive=true, then we define d1 = max(0, p2 - c), d2 = max(0, c - p2), a = max(c, min(p1, n1) - d1), b = min(c, max(p1, n1) + d2). Note that d = d1 + d2 and that either d1=0 or d2=0. Similarily, if strength=2 and aggressive= true, then we define d1 = max(0, min(p2 - c, n2 - c)), d2 = max(0, min(c - p2, c - n2)), and if strength=3 and aggressive= true, then we define d1 = max(0, min(p2 - c, n2 - c, p3 - c, n3 - c)), d2 = max(0, min(c - p2, c - n2, c - p3, c - n3)). We do not recommend to combine strength=3 with aggressive=true.

The type of flicker, which can be removed or at least reduced with ReduceFlicker, is more prevalent in camcorder material and the chroma of analog clips, especially clips captured from VHS tapes. In other words ReduceFlicker is suited for removing or reducing certain kinds of electronic noise, but it has virtually no effect on dust and film grain.

Unlike most other purely temporal filters like the "cheap" ReduceFluctuation filter below, ReduceFlicker doesn't impose any limitation on the amount of change. Theoretically this bears a substantial artifact risk. The worst case scenario for ReduceFlicker is a lamp which is turned on and off 25 times per second within a PAL video. Obviously this is not very natural and quite rare. It is more natural if a regular pattern, like a fence, is moving at a constant speed within a video. If the spatial properties of this pattern fits well to its speed, then this pattern gets blurred by ReduceFlicker. However, the likelihood of such a "fit" is extremely low if strength = 1,2. It is far higher, but still tolerable,  if strength =3. It can be nicely compared with the physical phenomenon of resonance. We all learn at school, that if a large group of soldiers marches in step over a bridge, the bridge may crash. While theoretically possible, I never heart about such an event, because the step frequency and the resonance frequency of the bridge, which depends on its size, have to be very close.

ReduceFluctuations

ReduceFluctuations should be used as follows 

ReduceFluctuations(clip input, clip "previous", clip "next",  int "limit", int "limitU", int "limitV", int "limitA", int "recursive", bool "planar")

The role of the variables "input" and "planar" are the same as for ReduceFlicker. The role of the clip variables "previous", "next" is the same as for the filter "Clense" (contained in RemoveGrain ) .With the variable "limit" (default = 5) one can specifiy the maximum amount of change of the luma. Similarily, one can specify with "limitU", "limitV" the maximum amount of change of the U and the V values. "limitU" inherits the value of "limit" as the default value and "limitV" inherits the value of "limitU" as the default value. If limitV > 255, then the V channels is replaced by random values. If limitV > 255 and limitU > 255, then the entire chroma is replaced by random values. This is useful for b&w video. Of course, Greyscale() has to be applied later. For an RGB color space "limit"is for blue channel, "limitU" is for the green channel and "limitV" is for the red channel, but it may also be the other way round. "limitA" is the limit for the fourth channel of the RGB32 color space (it is only used, if planar = false). For interleaved RGB24 (i.e. planar = false)clips the values of  "limitU" and "limitV" are ignored and the value of "limit" is taken instead. This was necessary because an efficient SSE implementation is not possible with interleaved pixels of size 3.

ReduceFluctuations makes use of three frames, the current, the subsequent and the previous frame. Consequently, the first and the last frame are left unchanged, because only one of the neighbours is available for these frames. If recursive>=0 (-1 is the default), then instead of the previous frame, the previously processed frame from a recursion clip is taken, if the frames are requested sequentially, as it usually happens during an encoding process and if a recursion clip is assigned later with AssignRecursionClip to the recursion slot specified with the variable "recursive". AssignRecursionClip is contained in AvsTimer version 0.8 or later. For instance,
input=ReduceFluctuations(recursive = 7) 
AssignRecursionClip(input,7)
return input					
reserves the recursion slot 7 (values between 0 and 9 are allowed here) and assigns the output of ReduceFluctuation as the recursion clip for ReduceFluctuations. This was the default behaviour of ReduceFluctuations version 0.3. The following example is more sophisticated:
input=ReduceFluctuations(recursive = 7).RemoveGrain(mode = 17)
AssignRecursionClip(input7)
return input				
Now the output of ReduceFluctuations(recursive = 7).RemoveGrain(mode = 17) is used as recursion clip. For more information about recursion see section The delay filter and AssignRecursionClip of the AvsTimer documentation .

ReduceFluctuations is just a limited version of the very simple filter Clense (cf. RemoveGrain). In fact, it is a combination of Clense and the filter LimitChange from the SSETools plugin. It is slightly slower than Clense but significantly faster than the combination of Clense and LimitChange. Because Clense can be very destructive, limitation is absolutely necessary and the values for limit, limitU, limitV, limitA should never be > 10. Even the default value 5 may be too big.

LockClense

LockClense should be used as follows 

LockClense(clip input,  int "limit", int "limitU", int "limitV", int "limitA", bool "clense", int "recursive", bool "planar")

The boolean variable "clense" will be explained later. The role of the other variables of LockClense is very similar to the role of the corresponding variables in ReduceFluctuations.  To explain how LockClense works, let c be luma of a pixel in the current frame, p be the luma of the same pixel in the previous frame and n be the luma of the same pixel in the subsequent frame. If |c - p| <= limit, then LockClense replaces c by p. Otherwise, if |c - n| <= 2*limit, then LockClense replaces c by (c + n)/2 (actually we take a more sophisticated  unbiased average). Finally, if neither |c - p| <= limit nor |c - n| <= 2*limit, then LockClense does the same as ReduceFluctuations (with the same limit). The U channel and the V channel are handled the same way, but with limitU and limitV instead of limit. If "clense= false" (true is the default value of this variable), then ReduceFluctuations is not applied. Thus in this case c remains unchanged if neither |c - p| <= limit nor |c - n| <= 2*limit. If clense= false, then LockClense has much more discontinuities, which have a negative impact on compression.  Thus we do not recommend this mode, which is similar to the DNR1 mode of the DNR2 filter. The values for limit, limitU, limitV should be <= 5 to avoid artifacts (2 is the default value).