ABSTRACT
Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address this problem, we have designed and implemented an open architecture and API for such cameras: the Frankencamera. It consists of a base hardware specification, a software stack based on Linux, and an API for C++. Our architecture permits control and synchronization of the sensor and image processing pipeline at the microsecond time scale, as well as the ability to incorporate and synchronize external hardware like lenses and flashes. This paper specifies our architecture and API, and it describes two reference implementations we have built. Using these implementations we demonstrate six computational photography applications: HDR viewfinding and capture, low-light viewfinding and capture, automated acquisition of extended dynamic range panoramas, foveal imaging, IMU-based hand shake detection, and rephotography. Our goal is to standardize the architecture and distribute Frankencameras to researchers and students, as a step towards creating a community of photographer-programmers who develop algorithms, applications, and hardware for computational cameras.
Supplemental Material
Available for Download
This archive contains supplemental materials for the paper as submitted. The html_api_docs directory contains the documentation for the FCam API described in the paper. See index.html for the start page. Since the API is under active development as of the time of this submission, please find the latest documentation if you wish to use the documentation for an actual programming reference. As of this writing, it is available at http://graphics.stanford.edu/papers/fcam.
- Adams, A., Gelfand, N., and Pulli, K. 2008. Viewfinder alignment. Computer Graphics Forum (Proc. Eurographics) 27, 2, 597--606.Google ScholarCross Ref
- Adobe, Inc., 2010. The Digital Negative Format. http://www.adobe.com/products/dng/.Google Scholar
- 2010. The Ångström Linux Distribution. http://www.angstrom-distribution.org/.Google Scholar
- Bae, S., Agarwala, A., and Durand, F. 2010. Computational re-photography. ACM Transactions on Graphics (To appear). Google ScholarDigital Library
- Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Cohen, M., Curless, B., and Kang, S. B. 2007. Using photographs to enhance videos of a static scene. In Proc. Eurographics Symposium on Rendering.Google Scholar
- Bramberger, M., Doblander, A., Maier, A., Rinner, B., and Schwabach, H. 2006. Distributed embedded smart cameras for surveillance applications. Computer 39, 2, 68--75. Google ScholarDigital Library
- 2010. The CHDK Project. http://chdk.wikia.com.Google Scholar
- Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics 26, 3 (July), 103:1--103:9. Google ScholarDigital Library
- Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of ACM SIGGRAPH 1997, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 369--378. Google ScholarDigital Library
- Durand, F., 2009. private communication.Google Scholar
- Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (Aug.), 673--678. Google ScholarDigital Library
- Filippov, A. 2003. Reconfigurable high resolution network camera. In Field-Programmable Custom Computing Machines, 2003. FCCM 2003. 11th Annual IEEE Symposium on, 276--277. Google ScholarDigital Library
- Greenberg, S., and Fitchett, C. 2001. Phidgets: easy development of physical interfaces through physical widgets. In UIST '01: Proc. of the 14th annual ACM symposium on user interface software and technology, ACM, New York, NY, USA, 209--218. Google ScholarDigital Library
- Hengstler, S., Prashanth, D., Fong, S., and Aghajan, H. 2007. Mesheye: a hybrid-resolution smart camera mote for applications in distributed intelligent surveillance. In IPSN '07: Proceedings of the 6th international conference on Information processing in sensor networks, 360--369. Google ScholarDigital Library
- Joshi, N., Kang, S. B., Zitnick, C. L., and Szeliski, R. 2010. Image deblurring using inertial measurement sensors. ACM Transactions on Graphics (Proc. SIGGRAPH) 29, 3 (Aug.). Google ScholarDigital Library
- Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High dynamic range video. In SIGGRAPH '03: ACM SIGGRAPH 2003 Papers, ACM, New York, NY, USA, 319--325. Google ScholarDigital Library
- Kleihorst, R., Schueler, B., Danilin, A., and Heijligers, M. 2006. Smart camera mote with high performance vision system. In ACM SenSys 2006 Workshop on Distributed Smart Cameras (DSC 2006).Google Scholar
- 2010. The Maemo Linux Distribution. http://maemo.org/.Google Scholar
- 2010. The Magic Lantern project. http://magiclantern.wikia.com/wiki/.Google Scholar
- Mann, S., and Picard, R. W. 1995. On being 'undigital' with digital cameras: Extending dynamic range by combining differently exposed pictures. In Proceedings of IS&T, 442--448.Google Scholar
- Mertens, T., Kautz, J., and Reeth, F. V. 2007. Exposure fusion. Proceedings of Pacific Graphics. Google ScholarDigital Library
- Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. In SIGGRAPH '04: ACM SIGGRAPH 2004 Papers, ACM, New York, NY, USA, 664--672. Google ScholarDigital Library
- Rahimi, M., Baer, R., Iroezi, O., Garcia, J. C., Warrior, J., Estrin, D., and Srivastava, M. 2005. Cyclops: in situ image sensing and interpretation in wireless sensor networks. In SenSys '05: Proceedings of the 3rd international conference on Embedded networked sensor systems, 192--204. Google ScholarDigital Library
- Raskar, R., and Tumblin, J. 2010. Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors. In Press. A K Peters, Natick, MA, USA. Google ScholarDigital Library
- Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: motion deblurring using fluttered shutter. In SIGGRAPH '06: ACM SIGGRAPH 2006 Papers, ACM, New York, NY, USA, 795--804. Google ScholarDigital Library
- Reinhard, E., Ward, G., Pattanaik, S., and Debevec, P. 2006. High Dynamic Range Imaging - Acquisition, Display and Image-based Lighting. Morgan Kaufman Publishers, 500 San-some Street, Suite 400, San Francisco, CA 94111. Google ScholarDigital Library
- Rowe, A., Goode, A., Goel, D., and Nourbakhsh, I. 2007. CMUcam3: An open programmable embedded vision sensor. Tech. Rep. RI-TR-07-13, Carnegie Mellon Robotics Institute, May.Google Scholar
- Tico, M., and Pulli, K. 2009. Image enhancement method via blur and noisy image fusion. In ICIP '09: IEEE International Conference on Image Processing, 1521--1525. Google ScholarDigital Library
- Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High performance imaging using large camera arrays. In SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, ACM, New York, NY, USA, 765--776. Google ScholarDigital Library
- Wolf, W., Ozer, B., and Lv, T. 2002. Smart cameras as embedded systems. Computer 35, 48--53. Google ScholarDigital Library
- Yodaiken, V. 1999. The RTLinux Manifesto. In Proceedings of The 5th Linux Expo.Google Scholar
Index Terms
The Frankencamera: an experimental platform for computational photography
Recommendations
The Frankencamera: an experimental platform for computational photography
Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address ...
The Frankencamera: an experimental platform for computational photography
Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address ...
Experimental Platforms for Computational Photography
Computational photography refers broadly to sensing strategies and algorithmic techniques that enhance or extend the capabilities of digital photography. Although interest in computational photography has steadily increased among graphics and vision ...
Comments