MSEmbGAN : Multi-Stitch Embroidery Synthesis via Region-Aware Texture Generation

Xinrong Hu, Chen Yang, Jin Huang, Lei Zhu, Ping Li, Bin Sheng, Tong-Yee Lee, Senior Member, IEEE

---

Abstract

Convolutional neural networks (CNNs) are widely used for embroidery feature synthesis from images, but they continue to fail in predicting diverse stitch types, which are commonly found in real embroidery images. Notably, CNN architectures cannot effectively extract stitch features, and there are too few labeled embroidery datasets. In this paper, we propose a multi-stitch embroidery generative adversarial network (MSEmbGAN) that uses a region-aware texture generation subnetwork to predict diverse embroidery features from images. To the best of our knowledge, our work is the first CNN-based generative adversarial network to succeed in this task. Our region-aware texture generation network detects multiple regions in the input image using a stitch classifier and generates a stitch texture for each based on its shape features. A structure generation network with a structure discriminator is also provided, which achieves full image structural consistency by forcing the shape and color features of the result to approximate the input image as much as possible. We also provide a new multi-stitch embroidery dataset labeled with three single-stitch and one multi-stitch types. To the best of our knowledge, our dataset is currently the largest embroidery dataset available, with more than 30K high-quality multi-stitch embroidery images, over 13K aligned content-embroidery images, and over 17K unaligned images. Quantitative and qualitative experimental results, including a qualitative user study, show that our MSEmbGAN outperforms current state-of-the-art embroidery synthesis and style-transfer methods in all areas.

---