ITK Release 4/A2D2 Projects/VideoGrabber

Goal

The purpose of this project is to develop a generic framework for real-time video capturing in ITK.

Team

• Patrick Cheng, Georgetown University
• Sebastian Ordas, Georgetown University

Design

Figure 1. Original system diagram for the generic itkVideoGrabber class

• This generic component will interface with different hardware devices through system media framework APIs, such as DirectShow on Windows, QuickTime on Mac, and Video4Linux on Linux
• We will carefully choose a couple of "Frame Grabber / Video Converter / Video Capture" card to support. The selection criteria are:
  1. Commercially available and easy to procure
  2. Low cost
  3. Cross-platform support (Windows, Mac, and Linux)
     • Georgetown has Windows 7 (32bit & 64bit), Mac Snow Leopard (OSX 10.6, 64bits), and Ubuntu 10.04 (32bit & 64bit) for testing
  4. Support system media frameworks
     • Windows Direct Show, Mac QuickTIme, and Linux V4L/V4L2
  5. Cover most video interface and standards
     • Interfaces: Composite, S-Video, VGA, and DVI
     • Standards: NTSC, PAL

Implementation

Work in progress. Follow these steps for the current state of the development (windows):

1. download Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP 1 v7.0 6.1.7600.16385 2009-07-24
   • http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c17ba869-9671-4330-a63e-1fd44e0e2505&displaylang=en

1. download Microsoft DirectX 9.0 SDK (October 2005)
   • http://www.microsoft.com/downloads/en/details.aspx?FamilyId=1C8DC451-2DBE-4ECC-8C57-C52EEA50C20A&displaylang=en

1. Build Visual Studio solution found in C:\Program Files\Microsoft SDKs\Windows\v7.0\Samples\multimedia\directshow\baseclasses

1. Run Cmake and make sure you have the following setup:
   • DIRECTSHOW_BASECLASSES_INCLUDE_DIR C:/Program Files/Microsoft SDKs/Windows/v7.0/Samples/multimedia/directshow/baseclasses
   • DIRECTSHOW_INCLUDE_DIR C:/Program Files/Microsoft DirectX 9.0 SDK (October 2005)/Include
   • DIRECTSHOW_QUARTZ_LIBRARY C:/Program Files/Microsoft SDKs/Windows/v7.0/Lib/quartz.lib
   • DIRECTSHOW_STRMIIDS_LIBRARY C:/Program Files/Microsoft SDKs/Windows/v7.0/Lib/strmiids.lib
   • DIRECTX_INCLUDE_DIR C:/Program Files/Microsoft DirectX 9.0 SDK (October 2005)/Include

Hardware

Comparison Chart

Last update: July 16, 2010	Epiphan VGA2USB frame grabber	EasyCap USB 2.0 video capture adapter	Imaging Source, DFG/USB-It video converter	Imaging Source, DFG/1394-1e video converter
Pictures	Figure 2A	Figure 2B	Figure 2C	Figure 2D
Cost	$299	<$10	$190	$330
Vendor link	Epiphan	Amazon	Imaging Source	Imaging Source
Interface
Input	VGA (DVI with DVI-to-VGA adapter)	Composite + S-Video	Composite + S-Video	Composite X2 + S-Video
Output	USB	USB	USB	IEEE 1394 / Firewire
Platform Support (Driver & SDK)
Windows	YES Driver SDK	YES Comes with driver CD For Windows 7, referring to this Youtube Video (hacked version)	YES Driver, App, and SDK	YES Driver, App, and SDK
Mac OSX	YES Driver V3.24.8 WhackedTV - App & Sample Code	YES 1. EasyCap Viewer, open source driver & app (GPL/BSD License) 2 . VideoGlide, commercial driver ($30)	NO	YES QuickTime driver from ASC, commercial software ($50)
Linux	YES Pre-compiled drivers SDK	YES Non-official open source drivers (GNU License)	NO	YES 1. V4L2 driver on sourceforge, (GPL License) 2 unicap, (GPL License)
Media Framework Compatibility
Direct Show	YES Tested with VLC Capture SDK includes DirectShow integration	YES Tested with VLC Capture	[MAYBE] Test failed with VLC Capture	[MAYBE], Not tested Quote,"The Imaging Source offers the driver IC WDM V2F for Windows® 2000 and XP that supports all parameters of the Video-to-FireWire converter. Following Microsofts® specification, it is a WDM Stream Class driver."
Quick Time	YES Tested with WhackedTV using Sequence Grabber	[MAYBE] Video capture works with EasyCap Viewer Looks like EasyCapViewer is using QTKit	??? Not tested	[MAYBE], Not tested QuickTime driver from ASC, commercial software ($50)
V4L/V4L2	[YES] Not tested Driver supports V4L	[YES] Not tested Driver includes VLC testing program, which uses V4L2	??? Not tested	[MAYBE], Not tested V4L2 driver on sourceforge, (GPL License)

Summary Notes

1. Epiphan VGA2USB (Figure 2A)
   • Pros
     • Good cross-platform support and supports all media framework
     • SDK freely available on all platforms
     • Well established company and good support
     • VGA/DVI connections are widely used in modern monitors
   • Cons
     • High-end model can be pricey
2. EasyCap DC 60 (Figure 2B)
   • Pros
     • Cheap!!
     • Wide user base, mostly gamers who use it to capture game video
     • Cross-platform support (with user developed drivers)
     • Looks like it should support all media framework
   • Cons
     • Official support = zero
     • No SDKs, official driver only available for Windows
3. Imaging Source, DFG/USB2-It (Figure 2C)
   • Pros
   • Cons
     • No driver for Mac and Linux
4. Imaging Source, DFG/1394-1e (Figure 2D)
   • Pros
     • Compare to the USB model, it does have third-party driver for Mac and Linux, and they are suppose to be compatible with media frameworks
   • Cons

Georgetown Resource

Video Grabbers

1. VGA2USB
2. EasyCap DC60
3. DFG/USB2-It

Development & testing environment

1. Windows 7 32bit & 64bit, Visual Studio 2008/1010,
2. MacBook Pro, Snow Leopard 64bit (Mac OSX 10.6), XCode 3.2
3. Ubuntu 10.04, GCC 4.3

Video source for testing

1. Desktop computer with a GeForce 7300 graphics card
2. Graphics card output include: S-Video, DVI, and VGA
3. Supports multiple TV-out standards, including NTSC & PAL

Software

Media Framework

Windows:

1. Direct Show
   • Direct Show for Video Capturing
   • DirectX SDK
   • Windows SDK

Mac OSX:

1. Sequence Grabber (old)
   • http://developer.apple.com/quicktime/qttutorial/seqgrab.html
2. QtKit Capture (New in Quick Time 7, backward compatible with Sequence Grabber)
   • Programming Guide: web and pdf

Linux:

1. V4L is the original video capture/overlay API of the linux kernel. It appeared late the 2.1.x development cycle in the linux kernel.
2. V4L2 is the second generation of the video4linux API which fixes a number of design bugs of the first version. It was integrated into the standard kernel in 2.5.x.

Related Implementations

Windows

1. videoInput Library (No License)

Linux

1. libfg (LGPL License)
2. gstreamer (GPL License)

Cross Platform

1. avcap, (GPL License)
   • "The avcap-library supports Video4Linux-Devices, Video4Linux2-Devices and AV/C-Devices (e.g. DV-Cams)"
   • "The Windows-version is basically a class wrapper for the DirectShow API and thus supports only devices with a WDM (Windows driver model) or an old VFW (Video for windows) compliant capture device driver. "
   • "The implementation for OS X uses the QuickTime SequenceGrabber-Component and has been tested with the built-in iSight, various USB-cams and DV-Cams. "
   • "qedit.h and dxtrans.h are missing both in the latest versions of Microsoft DirectX SDK (June 2010) and Microsoft SDK (v7.0). avcap uses qedit.h so here's what to do to make avcap work: Install the latest Microsoft SDK (v7.0) and define an environment variable WINSDK_DIR pointing to C:\Program Files\Microsoft SDKs\Windows\v7.0. Also install DirectX SDK (August 2007) and do the same with DXSDK_DIR C:\Program Files\Microsoft DirectX SDK (August 2007). In the avcap solution under ..\avcap-0.1.9\contrib, add in "Additional include dirs", $(DXSDK_DIR)\Include and modify the other entries to have $(WINSDK_DIR)\Samples\Multimedia\DirectShow\BaseClasses and $(WINSDK_DIR)\Include" Now try building avcap. You may still get a warning saying: "To compile qedit.h you must install the DirectX 9 SDK, to obtain the dxtrans.h header". For more information on the issue: [1]"
   • "strmbasd and strmbase are missing? Build BaseClasses.dsw located in C:\Program Files\Microsoft SDKs\Windows\v7.0\Samples\multimedia\directshow\baseclasses both in Debug and Release. Then you may place them in C:\Program Files\Microsoft SDKs\Windows\v7.0\Libs"
2. ARToolKit, (GPL License)
   • "ARToolKit\include\AR\video.h"
   • "Windows: with Microsoft DirectShow (VFW obsolete)."
   • "Linux: with Video4Linux library, GStreamer, IEEE1394 camera library and DV camera library."
   • "Macintosh: with QuickTime."
   • "SGI: with VL."
3. VideoGrabber in openFrameworks, (MIT License)
   • "ofx-dev\libs\openFrameworks\video\ofVideoGrabber.h"
   • "The ofVideoGrabber class wraps quicktime's sequence grabbing component to provide low level access to live cameras. On windows it now uses the directshow based videoInput library which offers mainy performance advantages over quicktime and does not require quicktime or WinVDig to be installed. A #define in ofConstants.h allows you to choose whether to use quicktime or directshow (default) for windows. In linux it uses by default unicap, although you can change to v4l through a #define in ofConstants in case some v4l device doesn't work properly with unicap."
4. phonon in Qt 4.4 and later , (LGPL License)
   • "Qt\src\plugins\phonon"
   • Using DirectShow 9, gstreamer, and QuickTime 7 in Qt 4.6
5. VLC Capture, (GPL License)
   • "vlc-1.1.0\modules\access"
   • VLC supported input media and format (http://www.videolan.org/vlc/features.html)
   • Supports DirectShow, V4L/V4L2, and iSight (internal only)
6. OpenCV, Camera Compatibility, (BSD License)
   • "opencv\src\highgui\cvcap_xxx"

Progress Report

1. A2D2_VideoGrabber_07_2010
2. A2D2_VideoGrabber_08_2010
3. A2D2_VideoGrabber_09_2010
4. A2D2_VideoGrabber_10_2010
5. A2D2_VideoGrabber_11_2010
6. A2D2_VideoGrabber_12_2010
7. A2D2_VideoGrabber_01_2011
8. A2D2_VideoGrabber_02_2011
9. A2D2_VideoGrabber_03_2011
10. A2D2_VideoGrabber_04_2011
11. A2D2_VideoGrabber_05_2011
12. A2D2_VideoGrabber_06_2011