Embedded camera systems are ubiquitous, representing the most widely deployed example of a wireless embedded system. They capture a representation of the world - the surroundings illuminated by visible or infrared light. Despite their widespread usage, the architecture of embedded camera systems has remained unchanged, which leads to limitations. They visualize only a tiny portion of the world. Additionally, they are energy-intensive, leading to limited battery lifespan. We present PixelGen, which re-imagines embedded camera systems. Specifically, PixelGen combines sensors, transceivers, and low-resolution image and infrared vision sensors to capture a broader world representation. They are deliberately chosen for their simplicity, low bitrate, and power consumption, culminating in an energy-efficient platform. We show that despite the simplicity, the captured data can be processed using transformer-based image and language models to generate novel representations of the environment. For example, we demonstrate that it can allow the generation of high-definition images, while the camera utilises low-power, low-resolution monochrome cameras. Furthermore, the capabilities of PixelGen extend beyond traditional photography, enabling visualization of phenomena invisible to conventional cameras, such as sound waves. PixelGen can enable numerous novel applications, and we demonstrate that it enables unique visualization of the surroundings that are then projected on extended reality headsets. We believe, PixelGen goes beyond conventional cameras and opens new avenues for research and photography.

翻译：嵌入式相机系统无处不在，是无线嵌入式系统中部署最广泛的范例。它们捕捉世界的表征——由可见光或红外光照亮的周围环境。尽管应用广泛，嵌入式相机系统的架构却一成不变，这导致了诸多局限。它们仅能可视化世界中极小的一部分。此外，其能耗巨大，导致电池寿命受限。我们提出PixelGen，重新构想嵌入式相机系统。具体而言，PixelGen结合了传感器、收发器以及低分辨率图像和红外视觉传感器，以捕捉更广泛的世界表征。这些组件特意选用，因其结构简单、比特率低、功耗小，最终构建成一个节能的平台。我们证明，尽管数据来源简单，但通过基于Transformer的图像和语言模型处理所捕获的数据，可以生成环境的新颖表征。例如，我们展示该系统能够生成高清图像，而相机本身使用的是低功耗、低分辨率的单色相机。此外，PixelGen的能力超越了传统摄影，能够可视化传统相机无法捕捉的现象，例如声波。PixelGen能够实现众多新颖应用，我们演示了它如何实现对周围环境的独特可视化，并将其投射到扩展现实头显上。我们相信，PixelGen超越了传统相机的局限，为研究与摄影开辟了新途径。