Optica l computing via free-space-based structured optical materials allows to access optical information without the need for preprocessing or optoelectronic conversion. In this Perspective, the authors describe opportunities and challenges in their use for optical computing, information processing, computational imaging and sensing.