LawalMcCoy: Difference between revisions
imported>Psych2012 |
imported>Psych2012 |
||
| Line 25: | Line 25: | ||
===== Challenges===== | ===== Challenges===== | ||
We ran into several challenges using our program, the most of which was simply computing time. To add subtitles to even a clip of just roughly 30 seconds, it typically took our program over 4-10 minutes to accomplish this depending on how many subtitles were present in the clip and the resolution of the images. This was simply running our program to produce white subtitles with a shadow behind them, like most commercial subtitles. Furthermore, when trying to optimize for color, for each individual frame it would take four minutes for the program to give us the correct gray shade that maximized the deltaE difference between the text box and the pixels on the image. This clearly makes maximizing deltaE impractical for clips of reasonable length using our program. However, for the vast majority of frames we tried this on, we found that a completely white subtitle actually maximized deltaE so the base case assumption (which is used in commercial subtitles) was essentially sufficient for a movie thus cutting down computing time. For the interest of this presentation though, we will also | We ran into several challenges using our program, the most of which was simply computing time. To add subtitles to even a clip of just roughly 30 seconds, it typically took our program over 4-10 minutes to accomplish this depending on how many subtitles were present in the clip and the resolution of the images. This was simply running our program to produce white subtitles with a shadow behind them, like most commercial subtitles. Furthermore, when trying to optimize for color, for each individual frame it would take four minutes for the program to give us the correct gray shade that maximized the deltaE difference between the text box and the pixels on the image. This clearly makes maximizing deltaE impractical for clips of reasonable length using our program. However, for the vast majority of frames we tried this on, we found that a completely white subtitle actually maximized deltaE so the base case assumption (which is used in commercial subtitles) was essentially sufficient for a movie thus cutting down computing time. For the interest of this presentation though, we will also present a few images where the deltaE difference was not maximized with completely white subtitles. | ||
== Results == | == Results == | ||
Revision as of 05:55, 18 March 2012
Project Title
Improving subtitle visibility in low budget movies
Introduction
The main goal of our project was to develop a program in MATLAB that achieves visually pleasing viewing properties of a given subtitle and frame (picture) combination. Although the best solution is to have real people go through a movie, frame by frame, and decide what viewing properties for the subtitles makes them most visible, developing an automated solution may be the next best thing. In addition, many foreign films may not have the budget to have individual people review the movie as this is a tedious task.
The viewing properties of the subtitle that our program analyzed were the location of the subtitle, the shadowing around the text (if any), and the color of the text in the subtitle. For the color our program uses the CIELAB metric of deltaE as a proxy for the "correct" color. Color theory states that the larger the deltaE between two objects, the more discernible they are. In this situation we analyzed the average delta e difference between text and other pixels in the textbox.
One of our major goals was to answer the question: do larger deltaE differences appear more visually appealing to the viewer or do more subjective factors (such as shadowing and location) dominate? Furthermore, is there even really an advantage to larger deltaE differences (with all other factors held constant) in determining visually pleasing subtitles?
Method
How the program works
The program works through the use of a main wrapper function, place_captions. Before running the program though, preliminary work needs to be done. First the clips are converted into movie structs which are Matlab constructs that contain each frame in the movie as an image. Then, the subtitles must be transcribed into a text file in the same directory as place_captions. Now the program is ready to run. The program creates a black shadow around the text which research has shown increases discriminability between text and the image behind it. Next the location of the subtitle is so that it is in the lower left corner, and issues with this are discussed in the subsequent section. The program that we run finally selects the optimal shade of gray (equal amounts of RGB channel) that maximize the deltaE of the subtitles between the image the subtitle is overlaid on.
Shadow
Subtitle Location
The subtitle's upper left location was selected to be, in most cases, 10% above the bottom and 25% to the right from the left boundary of the image. This was done to replicate the the style in commercial subtitles. Issues came into play when the actual subtitle was either too long, the font needed to be adjusted, or if the images that were read in from the movie were small. This called for adjustments in the code to break up the subtitle into two lines if the width of the subtitle was greater than that in the image as well as allowing font to be an input to the function.
Subtitle Color
Challenges
We ran into several challenges using our program, the most of which was simply computing time. To add subtitles to even a clip of just roughly 30 seconds, it typically took our program over 4-10 minutes to accomplish this depending on how many subtitles were present in the clip and the resolution of the images. This was simply running our program to produce white subtitles with a shadow behind them, like most commercial subtitles. Furthermore, when trying to optimize for color, for each individual frame it would take four minutes for the program to give us the correct gray shade that maximized the deltaE difference between the text box and the pixels on the image. This clearly makes maximizing deltaE impractical for clips of reasonable length using our program. However, for the vast majority of frames we tried this on, we found that a completely white subtitle actually maximized deltaE so the base case assumption (which is used in commercial subtitles) was essentially sufficient for a movie thus cutting down computing time. For the interest of this presentation though, we will also present a few images where the deltaE difference was not maximized with completely white subtitles.
Results
The above three images represent from left to right, the bright white subtitle (not deltaE differential maximized) overlaid on an image, the deltaE differential maximized shade of gray for the subtitle with a shadow, and the deltaE maximized shade of gray with the subtitle without the shadow. Upon polling residents in our dorm, we found that consistently people found the deltaE differential maximized subtitle with a shadow as the least visually appealing. Generally, people found the deltaE maximized shade of gray without the subtitle to be the best and the white subtitle with the shadow to be middle best. However, there was not a massive preference between the latter two. This yields us to believe that the effect of the shadow works best with white subtitles, and perhaps can compensate for lack of deltaE differential.
Conclusions
test
References
test
Appendix I - Code and Data
test
Appendix II - Work Partition
test