Traffic recognition is commonly done using deep packet inspection or packet‐based approaches. However, these methods require the listening device to be part of the network and raise privacy concerns. Traffic recognition models that operate directly at the spectrum level could, for instance, be used for smart spectrum management. To this extent, we present such an approach using deep learning methods. In particular, we present a convolutional neural network architecture that forms the basis for prediction models to recognize different transport protocols, burst traffic with different duty cycles, and different transmission rates. These models are trained with pure synthetic data to lighten the burden of data collection. As such, we validate recent successes in the area of robotics in the context of wireless networks. We compare the performance of two different datasets that contain spectrum images in either time or time‐frequency domain. Our evaluation shows that using time domain data results in an accuracy of at least 96% across all models. Time‐frequency information improves this accuracy even further. Furthermore, a validation with real‐life data shows that it is still possible to discriminate between different transmission rates with an accuracy of around 87%, while the detection of duty cycles and transport protocols takes place with accuracies of, respectively, around 73% and 78%. Finally, we also present a small‐scale real‐life prototype.