skip to main content
10.1145/1833349.1778766acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
research-article

The Frankencamera: an experimental platform for computational photography

Published:26 July 2010Publication History

ABSTRACT

Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address this problem, we have designed and implemented an open architecture and API for such cameras: the Frankencamera. It consists of a base hardware specification, a software stack based on Linux, and an API for C++. Our architecture permits control and synchronization of the sensor and image processing pipeline at the microsecond time scale, as well as the ability to incorporate and synchronize external hardware like lenses and flashes. This paper specifies our architecture and API, and it describes two reference implementations we have built. Using these implementations we demonstrate six computational photography applications: HDR viewfinding and capture, low-light viewfinding and capture, automated acquisition of extended dynamic range panoramas, foveal imaging, IMU-based hand shake detection, and rephotography. Our goal is to standardize the architecture and distribute Frankencameras to researchers and students, as a step towards creating a community of photographer-programmers who develop algorithms, applications, and hardware for computational cameras.

Skip Supplemental Material Section

Supplemental Material

References

  1. Adams, A., Gelfand, N., and Pulli, K. 2008. Viewfinder alignment. Computer Graphics Forum (Proc. Eurographics) 27, 2, 597--606.Google ScholarGoogle ScholarCross RefCross Ref
  2. Adobe, Inc., 2010. The Digital Negative Format. http://www.adobe.com/products/dng/.Google ScholarGoogle Scholar
  3. 2010. The Ångström Linux Distribution. http://www.angstrom-distribution.org/.Google ScholarGoogle Scholar
  4. Bae, S., Agarwala, A., and Durand, F. 2010. Computational re-photography. ACM Transactions on Graphics (To appear). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Cohen, M., Curless, B., and Kang, S. B. 2007. Using photographs to enhance videos of a static scene. In Proc. Eurographics Symposium on Rendering.Google ScholarGoogle Scholar
  6. Bramberger, M., Doblander, A., Maier, A., Rinner, B., and Schwabach, H. 2006. Distributed embedded smart cameras for surveillance applications. Computer 39, 2, 68--75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. 2010. The CHDK Project. http://chdk.wikia.com.Google ScholarGoogle Scholar
  8. Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics 26, 3 (July), 103:1--103:9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of ACM SIGGRAPH 1997, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 369--378. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Durand, F., 2009. private communication.Google ScholarGoogle Scholar
  11. Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (Aug.), 673--678. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Filippov, A. 2003. Reconfigurable high resolution network camera. In Field-Programmable Custom Computing Machines, 2003. FCCM 2003. 11th Annual IEEE Symposium on, 276--277. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Greenberg, S., and Fitchett, C. 2001. Phidgets: easy development of physical interfaces through physical widgets. In UIST '01: Proc. of the 14th annual ACM symposium on user interface software and technology, ACM, New York, NY, USA, 209--218. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hengstler, S., Prashanth, D., Fong, S., and Aghajan, H. 2007. Mesheye: a hybrid-resolution smart camera mote for applications in distributed intelligent surveillance. In IPSN '07: Proceedings of the 6th international conference on Information processing in sensor networks, 360--369. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Joshi, N., Kang, S. B., Zitnick, C. L., and Szeliski, R. 2010. Image deblurring using inertial measurement sensors. ACM Transactions on Graphics (Proc. SIGGRAPH) 29, 3 (Aug.). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High dynamic range video. In SIGGRAPH '03: ACM SIGGRAPH 2003 Papers, ACM, New York, NY, USA, 319--325. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Kleihorst, R., Schueler, B., Danilin, A., and Heijligers, M. 2006. Smart camera mote with high performance vision system. In ACM SenSys 2006 Workshop on Distributed Smart Cameras (DSC 2006).Google ScholarGoogle Scholar
  18. 2010. The Maemo Linux Distribution. http://maemo.org/.Google ScholarGoogle Scholar
  19. 2010. The Magic Lantern project. http://magiclantern.wikia.com/wiki/.Google ScholarGoogle Scholar
  20. Mann, S., and Picard, R. W. 1995. On being 'undigital' with digital cameras: Extending dynamic range by combining differently exposed pictures. In Proceedings of IS&T, 442--448.Google ScholarGoogle Scholar
  21. Mertens, T., Kautz, J., and Reeth, F. V. 2007. Exposure fusion. Proceedings of Pacific Graphics. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. In SIGGRAPH '04: ACM SIGGRAPH 2004 Papers, ACM, New York, NY, USA, 664--672. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Rahimi, M., Baer, R., Iroezi, O., Garcia, J. C., Warrior, J., Estrin, D., and Srivastava, M. 2005. Cyclops: in situ image sensing and interpretation in wireless sensor networks. In SenSys '05: Proceedings of the 3rd international conference on Embedded networked sensor systems, 192--204. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Raskar, R., and Tumblin, J. 2010. Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors. In Press. A K Peters, Natick, MA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: motion deblurring using fluttered shutter. In SIGGRAPH '06: ACM SIGGRAPH 2006 Papers, ACM, New York, NY, USA, 795--804. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Reinhard, E., Ward, G., Pattanaik, S., and Debevec, P. 2006. High Dynamic Range Imaging - Acquisition, Display and Image-based Lighting. Morgan Kaufman Publishers, 500 San-some Street, Suite 400, San Francisco, CA 94111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Rowe, A., Goode, A., Goel, D., and Nourbakhsh, I. 2007. CMUcam3: An open programmable embedded vision sensor. Tech. Rep. RI-TR-07-13, Carnegie Mellon Robotics Institute, May.Google ScholarGoogle Scholar
  28. Tico, M., and Pulli, K. 2009. Image enhancement method via blur and noisy image fusion. In ICIP '09: IEEE International Conference on Image Processing, 1521--1525. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High performance imaging using large camera arrays. In SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, ACM, New York, NY, USA, 765--776. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Wolf, W., Ozer, B., and Lv, T. 2002. Smart cameras as embedded systems. Computer 35, 48--53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Yodaiken, V. 1999. The RTLinux Manifesto. In Proceedings of The 5th Linux Expo.Google ScholarGoogle Scholar

Index Terms

  1. The Frankencamera: an experimental platform for computational photography

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      SIGGRAPH '10: ACM SIGGRAPH 2010 papers
      July 2010
      984 pages
      ISBN:9781450302104
      DOI:10.1145/1833349

      Copyright © 2010 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 26 July 2010

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      SIGGRAPH '10 Paper Acceptance Rate103of390submissions,26%Overall Acceptance Rate1,822of8,601submissions,21%

      Upcoming Conference

      SIGGRAPH '24

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader