WHAT'S ON THAT DVD? (PART 2 of 2) by Tom Daniel
In Part 1 of this tutorial, we learned how to get some basic information about the DVDs we own. We can't always rely on such sources as the DVD case or insert, IMDB, Amazon.com, our favorite reviewers, or other places for this information. Often the reviewers don't discuss some of the most important things that we want and need to know before buying a DVD for ourselves. Often they don't know what to look for, or how to figure these things out. If you have a high end Home Theater setup, the flaws in the DVDs are much more obvious than when we used to watch on a smaller interlaced TV set. And for those fortunate few that have a really large projection screen setup, then only the very best quality DVDs will do to show off its advantages. That's why we visit the DVD Beaver. Gary and his crew scour the world searching out the best DVDs of the best movies. Have you ever read a review where the reviewer raved about the quality of a DVD; its sharpness, incredible detail, lack of edge enhancement, etc., and then when you viewed the DVD yourself, you wondered if the guy was watching the same DVD as you? You might wonder if he was watching on a laptop computer, or an old 19" TV set, or if he even bothered to watch the DVD at all. I can only chuckle sometimes when I read these things. Many reviewers are trying to do the right thing, but don't have the knowledge to pinpoint the problems. They might notice an unnatural softness in the picture, or think they've noticed some blending/ghosting during certain kinds of movement, but don't know how to account for it. Some people aren't real sensitive to these things. I am, and after dissecting so many DVDs, I can now spot these things a mile away. And now I'm going to teach you how to do it yourself, if you're willing to learn.
The main emphasis for this portion of the discussion will be one one of my pet peeves; bad standards conversions. There are right ways and there are wrong ways to convert a PAL source to NTSC, and an NTSC source to PAL. I know nothing about the economics involved here. Evidently it's expensive to prepare an NTSC master from a PAL source. If the DVD isn't expected to sell well, then it may not be feasible to do it. I'm thinking specifically of silent films here. Many of them, even American silent films, are being restored in Europe these days, and then a PAL master is used by such companies as Image Entertainment, Kino International, and Milestone Films (and even by Warner Home Video in the case of the Chaplin box set fiasco of 2003). Maybe the situation will be improving. The recently released Kino edition of the Fritz Lang film Woman In The Moon (Frau Im Mond) wasn't blended, and had a good NTSC transfer (although Spies (Spione) released on the same day was blended because of the use of a PAL master). I'll never forgive them for nearly ruining the earlier releases of the much better Lang films, Die Nibelungen Saga, and Metropolis, by allowing a field blended PAL to NTSC transfer for the R1 DVDs. The 2004 Image Entertainment DVD release of the Russian films Outskirts/The Girl With The Hatbox also used good NTSC transfers. I feel that part of a reviewer's responsibility is to point out shortcomings in the transfers, in order to alert us, the DVD viewing public, to any possible problems, and in the hopes that, by pointing out the shortcomings of a DVD, the companies involved will do a better job in the future. My feeling is that if so much money was invested in the restorations, that a bit more should be spent to prepare a proper master. But again, I don't know anything about the expenses involved. I'm thankful that they are being restored and released to DVD, but I'm unhappy because, by taking that extra step, the results often could be so much better.
For this portion we'll need to install some more software. I've decided that the easiest way is to get it from a package that contains most of what we need. But it contains much more that we don't need as well, and the download is large (almost 8 MB). If you aren't on a broadband connection, it'll take a while to get. Get Gordian.Knot.RipPack.0.32.beta.Setup.exe Then install it. Later on, when the time comes to open an AviSynth script (more on that soon) in VDubMod, and you get an error message about being unable to locate a decompressor, then you'll also have to install a codec. You have 2 choices here. You can either get the Gordian Knot Codec Pack (which at over 10 MB is big) and install either the DivX or the XviD codec (I recommend the XviD codec), or, alternatively, just get either the standalone decoder, or the full XviD Codec.
 |
You'll
be installing the Gordian Knot 0.32 beta (or whatever more recent version is
released after this is published-not the updates, though, the full package).
Check everything as in the picture, and uncheck the rest. Below what we can
see, also check AviSynth 2.5.4. If you already got some of it from the links
on the first page, then you can uncheck the appropriate boxes. This might include
VirtualDubMod, DVDDecrypter (the version linked in Part 1 is more recent than
this one, but for our purposes it won't make any difference), and DGMPGDec 1.0.12. Next and last is a real small download. Get KernelDeint and replace the version that came with GKnot in the AviSynth Plugins folder whereever you installed GKnot. This is a more up-to-date version of the one that comes with GKnot, and does one important thing that the older version can't; create full resolution fields from frames. It makes it possible to create double-rate frames, or in other words, convert 29.97 frames per second (for NTSC, and 25 frames per second for PAL) into 59.94 FIELDS per second (for NTSC, or 50 FIELDS per second for PAL). It's a so-called "Smart Bobber". This becomes important very soon. |
AviSynth is the important thing here. It's normally used as a frameserver. It serves up the frames to the encoder (VDubMod) by way of the Movie.d2v and the DGDecode.dll. In the process, you can also filter the frames in just about any way you want, similar to what you can do in such programs as Adobe Premiere or Sonic Foundry's Vegas. It doesn't have all the effects those programs do, but you don't need them for converting DVD to AVI. But it is extremely powerful. Among the things you can do is to crop and resize, IVTC and deinterlace, smooth and sharpen, remove grain, adjust the colors, the contrast, the brightness, the saturation, and a whole host of other things. We won't be using it for those things (except for creating fields from frames, and maybe cropping and resizing).
First, set up a folder for your projects. Then decrypt a chapter or a whole movie to that folder, as described in Part 1. Then create a Movie.d2v project file using DGIndex as described in Part 1. You can perform both of those steps through the Ripping Tab of GKnot. Now it's time to set up an AviSynth template to use for this and any other future projects you might create. Open Gordian Knot.
In the lower left, Open the Movie.d2v you created. Then you might see something like this:
This is from the movie Red Cherry, quite a wonderful, if very grim, film, I thought. It's sometimes called China's Schindler's List. Great movie with a very bad R1 DVD. Can you notice the interlacing on both sides of the Nazi soldiers, and the obvious blending/ghosting of the one on the right? Very bad signs, those. Next, in the upper left of the picture, go File->Save & Encode, and then "Edit". That will bring up this screen:
 |
In
your movie folder, go File->New->Text Document, and instead of naming
it Movie.txt, name it Movie.avs, and ignore the Windows warning. An .avs is
the script file (which you can open in Notepad) that we'll need to open the
movie in VDubMod or in GKnot. Grab the portion of the black and white .avs from
below "# PLUGINS" to just above "# TRIM". Hold down the
left mouse button, and circle it all, and then paste it into the Movie.avs.
Or use the Shift/Down Arrow to do the same thing. There, instead of being white
on black, it'll be black on white; a text file looking document. And now we're
going to edit the heck out of it, until it looks something like this:
LoadPlugin("F:\DIVXST~1\GORDIA~1\AviSynthPlugins\dgdecode.dll") Grab the two LoadPlugins and the mpeg2source line you need for your own specific setup, delete the rest, and you then cut and paste the 3 remaining lines from what's just above. Make sure that the first three lines don't have the "#" in the front. If it's there, then that line will be ignored. Save it, and then test it out by opening it in VDubMod (File->Open video file->Movie.avs). Do that, and there's your movie, 29.97fps if NTSC, or 25fps if PAL. You can go File->File Information and learn some things about the movie. If something's wrong, you'll usually get an error message that might help you to pinpoint why the movie didn't open in VDubMod (and again, if you get a decompressor error, you'll have to install a codec). Use the Scroll bar to advance quickly, and the left and right arrows on your keyboard to advance one frame at a time. |
Next, we'll try and learn what we can by finding a stretch of the movie with movement, and then advancing one frame at a time. If NTSC, if for every 5 frames, 3 are clean progressive, and 2 are interlaced, then you have a telecined movie. From Part 1, we now know how to tell if it was telecined during the encoding, and it's stored on the DVD as 24fps progressive frames with the flags set to output 29.97fps, or if it was telecined prior to encoding, and encoded as Interlaced/Video. DVD2AVIdg gives you that information. For most of Hollywood's latest and greatest, that's what you'll usually see, that 3/2 combination. If PAL, you hope to see every frame as progressive, without any interlacing/combing. And if you do see interlacing, it may still be basically progressive, and not truly interlaced. We'll pinpoint that shortly. But the problem comes when many of our favorite films aren't mastered from proper sources.
There are 2 main ways that PAL sources can be converted to NTSC. The better way is simply to use the 24fps film source, and then create a good NTSC master from that. And when going from NTSC to PAL, you speed the video from 24fps to 25fps. Either way, the video is clean, although PAL people are victimized by the 4% PAL speedup. A 100 minute NTSC movie runs about 96 minutes in PAL. The main objection to this is that the audio is then pitched about half a tone higher, and people with perfect pitch object violently to this, particularly when music is involved. I object violently when the other method is used to convert from PAL to NTSC. The PAL 25fps master is used, the fields that make up the frames are blended, and the number increased by 20%, to go from 25fps to 30fps. For PAL people, fields are sometimes blended to go from 24fps to 25fps. This keeps the movie the same length, the audio is unchanged, but it absolutely destroys the video.
There are 2 methods to use to determine if you have a bad PAL to NTSC or a bad NTSC to PAL transfer on your hands. One is almost guaranteed to tell you, and the other is absolutely guaranteed to tell you. Start counting frames again. For PAL to NTSC transfers, instead of having a 5 frame cycle, as you do with standard telecined NTSC material, here you have a 6 frame cycle. Within that 30fps DVD is a 25fps movie (30 x 5/6 = 25). For every 6 frames, sometimes 4, but usually 5 are interlaced. You'll usually have 1, and at most 2, clean progressive frame in each 6 frame cycle. In some stretches all the frames may be interlaced. If so, find another place to look. I did see for the first time recently an NTSC DVD from a PAL source (the R3 EDCO House Of Flying Daggers) where every frame in the entire movie was interlaced. For that one I had to go to the absolutely guaranteed method to make sure of what I was seeing. You may not really notice the ghosting doing it this way, as it's usually hidden by all the interlacing, but the absolutely guaranteed method will reveal it to you. So, for NTSC sources for NTSC DVDs, there's a 5 frame cycle. When there's a PAL master involved for an NTSC DVD, it's a 6 frame cycle.
And while on the subject of detecting bad conversions when using VDubMod, there is another source of blended NTSC DVDs. This can happen when you have an NTSC source, but rather than telecining to reach 30fps, you get the same blended fields being created, similar to the bad PAL to NTSC conversions. That's what happened to Red Cherry above. Underneath all the interlacing is a 24fps movie. In this case, when advancing through the frames, rather than having 2 out of every 5 frames being interlaced, as you would for standard telecined material, 4 of every 5 frames are interlaced. Typically 2 will have heavy interlacing, 2 will have light interlacing, and only 1 of the 5 will be a clean progressive frame. This doesn't just happen with "foreign" (or non-US) films. I've also seen quite a few US films mastered in this horrible way.
Now, lets see some ghosting. Many people can't actually see it when watching the DVD. You may have to train yourself to spot it. Then again, you may not want to, as once you do learn how to spot it, it may ruin your movie watching experience. It will reveal itself during certain kinds of movement. A person walking will often seem to have 4 legs. Movement at a certain medium speed will seem to be unnaturally blurry and out of focus. A hand waving in the air might seem to be 2 hands. Remember the Chinese Schindler's List, Red Cherry, from above? That's one frame from the movie. Let's break that frame into its 2 fields and then resize them to NTSC DVD resolution. As you know, a standard interlaced TV set displays fields. It scans all the even lines and displays them, followed by the odd lines. For an NTSC DVD, there are 480 rows of pixels. All of the even rows (0, 2, 4, etc.) make up the top field, and all of the odd numbered rows (1, 3, 5, etc.) make up the bottom field. Using this script:
LoadPlugin("F:\DIVXST~1\GORDIA~1\AviSynthPlugins\dgdecode.dll")
LoadPlugin("F:\DIVXST~1\GORDIA~1\AviSynthPlugins\KernelDeInt.dll")
mpeg2source("D:\Movie\Movie.d2v")
SeparateFields()
#KernelBob(Order=1)
#crop(0,0,720,480)
#LanczosResize(640,352)
We can open that AVS in GKnot (or in VDubMod) to get this:
 |
This is one of the fields from the DVD at 720x240. Note in the script above the new command, "SeparateFields()". It's one of the commands built into AviSynth. Note the heavy ghosting in this one field. When you see ghosting/blending/double images in a field, that's how you know that there's some funny business going on. Spotting ghosting like this by pausing the DVD might not tell you the truth. Blended frames may or may not be truly blended. It could be the deinterlacer that's creating it. HTPCs are especially prone to do this (blend the frames, when the fields are clean). The fields don't lie. If you were to take that one field, and interleave it with its companion field, you'd get the original frame further up. The fields themselves don't display interlacing. But the 2 interleaved fields will display interlacing, unless they fit together to form a progressive frame. It's those interlaced frames that might show ghosting when deinterlaced. |
What a "Smart Bobber" like KernelBob does is to resize that field back to its origianal resolution. Now, we'll put KernelBob to work showing both of the fields that make up the frame farther up.
Here's the script:
LoadPlugin("F:\DIVXST~1\GORDIA~1\AviSynthPlugins\dgdecode.dll")
LoadPlugin("F:\DIVXST~1\GORDIA~1\AviSynthPlugins\KernelDeInt.dll")
mpeg2source("D:\Movie\Movie.d2v")
KernelBob(Order=1)
#crop(0,0,720,480)
#LanczosResize(640,352)
Remove the "#" from the left of the KernelBob line to make it active,
save it, and open it.
Notice how badly the bottom field is blended. Showing them like this can tell you much about the DVD. Now, let's crop away the black and resize it to its correct aspect ratio. It's a 4:3 widescreen DVD (another strike against it), so we set it up for that in the Resolution Tab of GKnot:
We crop away the black, and then a couple of more pixels into the video, to achieve low Aspect Error, check View->Resized in the GKnot picture, and that gives us a 640x368 picture:
 |
I've
backed up several fields to get an unblended one. Remember, using KernelBob
gives double the framerate. In this case it's 59.94fps. Two fields make up one
frame. If we took the 2 pictures above of consecutive fields (but unresized,
at 720x240), and "slid" them back together again, they would recreate
the first Red Cherry frame farther up. Whereas the 4:3 DVD picture gets resized
by the player to correct aspect ratio, here, we've done the resizing ourselves
to show it at its correct AR. Now the PAR is 1:1. Resizing to the correct AR
is much more obvious when you start with a 16:9 DVD. You can also use this 59.94fps sequence of fields (or 50fps for PAL) to determine the true underlying framerate. It requires counting frames. For NTSC, for any sequence of 60 fields (like, xx00 to xx59), the number of original, unique, unblended, unduplicated fields will give you the true framerate. In this case, there will be 24 such fields in a 60 field sequence, because it's been field blended from 24fps to 30fps, rather than being telecined. For bad PAL to NTSC conversions, there will be 25 such fields. For bad NTSC to PAL DVDs, there will be 24 such fields in a sequence of 50 fields. This method can also be used to determine the framerate, as on the DVD, for silent films. I've seen them at framerates from 15fps to 26fps, and everything in between. So, you might count 18 such fields in a 60 field sequence. However, the next sequence of 60 frames might have 19, because sometimes silents are on the DVD at real screwy framerates, like 18.5fps. Remember, don't count the exact duplicate frames (of which there are often many for low framerate silent films), and don't count the blended frames (if, indeed, there are any). Just count the unblended unique frames. |
If you have a "normal" unblended movie, you'll see something slightly different. For telecined NTSC material, remember how, when looking at the frames (29.97fps), out of every 5 frames in the cycle, 3 were progressive, and 2 were interlaced? When looking at the fields you'll see a sequence of 3 identical fields followed by 2 identical fields in every 5 field sequence. 3/2 3/2, hence 3:2 Pulldown. You have 2 unique fields in every 5 fields, or 24 unique fields in a cycle of 60 fields. The original framerate is 24fps. For PAL people, you'll see a pattern of 2 identical fields, followed by 2 identical fields, etc. 25 sets of pairs in 50 fields, or 25fps.
When looking at the fields, if every single one is slightly different from
the one before, then you have true interlaced material.
There's other information available for PAL people by doing this. Say that the
frame appeared interlaced, and you want to determine if it's really interlaced,
or is basically progressive. When stepping through the fields, if every picture
is different, then it's truly interlaced. If you have every 2 fields being identical,
then it's progressive, but with shifted fields, as described in Part 1. These
will be straightened out by any player that can detect 2:2 pulldown (and most
can). Say that every pair of fields is the same, but every so often (every 25
fields) there's a third identical field. This is another of the ways that 24fps
is converted to 25fps. A duplicate frame is added every second, but as a field
every half second. This isn't a bad way to do it. There's no blending, and the
length is the same, so there's no PAL speedup. Actually, when stepping through
frames (not fields), there are ordinarily 13 that are progressive, and 12 that
are interlaced. That is, every time there's a new field added, there's a field
reversal. It will go from Top Field First to Bottom Field First and back again.
Two changes every second. Half the frames are progressive, and half are interlaced.
I think that players with 2:2 pulldown can straighten this out also. My PAL
BFI DVD of the silent film Piccadilly does this. It's a native PAL transfer,
but it's on the DVD at 24fps, and was encoded this way, so the principle is
the same as doing an NTSC 24fps tp PAL 25fps conversion. If you are PAL, and
the source for your PAL DVD is interlaced video, 30fps NTSC, then there's no
hope. Every field is different. Most fields are blended, and every frame is
interlaced. Converting interlaced NTSC 60 Fields Per Second to PAL interlaced
50 Fields Per Second results in a horrible mess, no matter how you do it.
 |
Here's a picture
of Harold Lloyd, the great silent film comedian, and he's blended. This is,
again, a picture of a field. It's an American film, restored by Lobster Films
in France, and released as part of the Harold Lloyd Collection on R1 DVD. Earlier
in 2004, Kino International released The Harold Lloyd Collection, The Charlie
Chase Collection, The Stan Laurel Collection, and the Laurel And Hardy film
The Flying Deuces. All were restored by Lobster Films in France. All are PAL
to NTSC, all are blended, and all received glowing reviews for the quality of
the restorations and of the DVDs. Remember, these are fields, so whether you have a progressive or an interlaced display, the blends will be there. At the very least, the picture will appear to be much less detailed, much more blurry than it should be. These blended fields aren't just every now and again. There are often close to half the fields that are this way. It can't help but affect how the DVD looks when viewed, either on an Interlaced, or on a Progressive display. You may not be able to spot the ghosts or blends when watching the movie, but if so many look like poor Harold when he's moving, can't you imagine what it does to the picture? In addition, because these are always encoded as Interlaced/Video, and the Progressive frames can't be returned, the players will also deinterlace them, messing up the picture even more.
|
Some people attribute this sort of thing, when silent films are being discussed,
to motion blurring, or some sort of artifact of the framerate being lower than
normal, but that's pure nonsense. They don't have to be blended. A proper NTSC
master (for NTSC DVDs) could have been prepared. PAL people are also victimized
by this when an NTSC master is used for the PAL DVD. It is true, though, that
although interlaced displays could then show the silent film without problems
(except for the blends), for a progressive display, it would still have to be
deinterlaced somehow. If it were up to me, I would insert duplicate frames to
get to 24fps, and then telecine off of that (for NTSC), so it could then be
displayed progressively. That is, I would be willing to put up with the slight
"judder" from the duplicate frames in order to view nice clear frames.
For example, say it's been determined that the silent should be displayed at
20fps. For every 5 frames, I would insert a duplicate frame to get it to 24fps,
and then telecine during encoding. Something like this, 1, 2, 3, 4, 5, 5, 6,
7, 8, 9, 10, 10... I've even seen a couple of silent film DVDs done this way.
But that's just me. A lot of people don't like the idea of purposely introducing
judder. Enough of the editorializing.
One more, and an interesting one, I think:
 |
I opened
the Chaplin.d2v in GKnot, went to the Resolution Tab, cropped away the black
from all 4 sides, resized to 512x384, and went back to the Save & Encode->Edit
white on black screen to cut and paste the mpeg2source, the crop, and the resize
lines into my AVS template, to get this:
LoadPlugin("F:\DIVXST~1\GORDIA~1\AviSynthPlugins\dgdecode.dll") Then you can open the AVS in VDubMod, or you can open the AVS in GKnot, or you can even open it in a player and "play" the AVS (no, not the Windows player, a real player). However, if opening it in GKnot, as I did to take the picture, instead of it being 4:3, as the DVD was, it's now 1:1, so you have to change that in the Resolution Tab. You also disable the Crop, as it's already been cropped. The R1 Image Entertainment Chaplin Mutuals DVDs are blended. I love the Chaplin Mutual films This is The Immigrant from the BFI UK PAL DVD of Volume 1 of the Chaplin Mutuals. I decided that since they were restored in Europe (by MK2, I believe), that the PAL versions were the ones to get. How wrong I was! They are on the DVD at 20fps. But did they telecine to 25fps? No, they field blended! I don't know where those red and green blotches came from. Someone forgot to greyscale it. I think this may be another sometime by-product of field blending, but I'm not really sure. I've seen PAL silent film DVDs where the base framerate was 20fps, and they were telecined to 25fps, just as you would telecine for NTSC to go from 24fps to 30fps. I know it can be done. In this case it wasn't. The restorers degraded their own restoration when creating the PAL master. Or someone did, at some point. Inexcusable. |
Well, that's it. Some people, even when they know that some kind of a bad standards conversion has taken place, take the position that it's not important, that the DVD still looks good, and that we should be thankful for it any way we can get it, specially when relatively obscure or not very popular films are involved. I hope the pictures convince you that these blended DVDs do not look good at all. I hope that a few of you out there, especially a few caring and responsible reviewers and others in a position to place pressure where pressure is required, will try out some of these tools and see for yourselves just how bad the situation is. A few still pictures don't begin to convey the seriousness of the problem, in my opinion. As for the being thankful part, maybe I'm cynical, but I sometimes think that these things are being foisted upon us because we're not supposed to know any better, that we can't tell the difference, that we won't call them on it, and that they can save themslves a few thousand dollars by not preparing a proper transfer. I think we should scream bloody murder over it, or in Peter Finch's immortal line from Network, "I'm mad as hell, and I'm not gonna take this anymore!" With much of the world moving to HDTV (sorry Europe, not just yet for you) and with the increasing number of progressive displays being sold, there is no longer any excuse at all for Interlaced encoding when Progressive encoding is called for. With HD-DVD just around the corner, in a year or 2, will you really want to watch a ghosted Hi-Def DVD? Sort of defeats the whole purpose, don't you think?
Although I know these tools pretty well, I'm far from an expert on how DVDs look on different displays, and in different regions. I use a Bravo D1 DVD/MPEG-4 player connected by DVI and upscaled to 1280x720 to a Samsung HLN4365W DLP HDTV. I don't have one of those real big projector setups, but what I have is fine for me. HTPC users have a whole set of problems of their own, because none of the software players can begin to match the quality of a good DVD player or HDTV featuring a Faroudja chipset. If your favorite films are those that come out of Hollywood, then these days just about any kind of a player and display will do. It's DVDs made from films from other places that sometimes give those of us with progressive setups problems.
So, if I've said anything wrong, or obviously stupid, I hope that someone will let me know so that it can be corrected. If any of you have any problems setting up or running the software, or think that I haven't explained its use well enough, just drop me a line and I'll walk you through it, and rewrite portions of this document so it's easier to understand. There's no need to correct me for any gross oversimplifications you might spot. Sometimes I simplify difficult concepts so that I can understand them myself. In other cases I simplify so that others new to this might understand better. Thanks for reading. Thanks to Gary for hosting this document. Keep visiting the DVDBeaver, and don't forget to get your DVDs through the links on this site so that Gary can continue doing what he does so well.
Contact info: man_ono@yahoo.com
LINKS:
http://www.avisynth.org/ Curious about
AviSynth? This is the place.
http://www.dvdscan.com/chaplin.htm
Here's a man that gets it. Actually, he gets it better than I do, since he also
understands how it was created. He comes at it from a different perspective,
and uses different language, but has the same sense of outrage. He calls it
frame overlap; Gary calls it PAL to NTSC ghosting; I call it field blending.
We're all talking about the same thing.