Range sensing and environmental simulation for smart lighting applications

This thesis addresses "smart rooms", living and working spaces that intelligently react to the presence and behavior of their occupants. Smart rooms require sensors and algorithms that enable real-time occupancy detection, tracking, and pose analysis. We focus specifically on depth sensing using time-of-flight technology. We also investigate the photorealistic simulation of environments and lighting control systems, which are useful for prototyping smart rooms before they are built.;First, we propose a method for real-time person tracking and coarse pose recognition in a smart room using time-of-flight measurements. The time-of-flight images are severely downsampled to preserve the privacy of the occupants and simulate future applications that use single-pixel sensors in smart ceiling panels. The tracking algorithms use grayscale morphological image reconstruction to avoid false detections, and are designed not to mistakenly detect pieces of furniture as people. A maximum likelihood estimation method using a simple Markov model was implemented for robust pose classification. We show that the algorithms work effectively even when the sensors are spaced apart by 25cm, using both real-world experiments and environmental simulation.;Next, we extend the time-of-flight occupancy sensing framework to the new Smart Conference Room constructed on the Rensselaer Polytechnic Institute (RPI) campus, using a ceiling-mounted array of time-of-flight cameras. This large environment brings new challenges and requires new algorithm modifications. We discuss the challenges and propose corresponding solutions, including multi-sensor interference elimination, integration of sensors in the development environment of a smart lighting system, and calibration of height measurements for robust pose estimation. We also improve the tracking algorithm by using the location, height, area and shape information for blob matching. This up-scaled tracking system is validated with experiments on full-scale time-of-flight videos recorded in the conference room with multiple occupants walking around.;Finally, we propose a computer graphics simulation framework to pre-visualize and tune the parameters of an advanced lighting controller for a given illuminated environment. The goal is to show that the simulation framework makes it easy for a user to predict the controller's behavior and modify it with minimal effort. Our methodology involves off-line pre-computation of lightmaps created from photorealistic rendering of the scene in several basis lighting configurations, and the subsequent combination of these lightmaps in a video game engine. We demonstrate our framework in a series of experiments in a simulation of the Smart Conference Room, showing how the controller can be easily modified to explore different lighting behaviors and energy use tradeoffs. The result of each experiment is a computer-generated animation of the lighting in a room over time from a single viewpoint, accompanied by estimated measurements of source input, light sensor output, and energy usage. This research is aimed at both lighting designers seeking to quantitatively predict real-world controller behavior, and control algorithm researchers seeking to visualize results and explore design tradeoffs in realistic use cases. We also show how measurements of source and sensor specifications enable the output of the virtual sensors in the simulation to match the outputs of real sensors in the physical room when applying the same control law in both cases.