A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 4 Issue 4
Oct.  2017

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 11.8, Top 4% (SCI Q1)
    CiteScore: 17.6, Top 3% (Q1)
    Google Scholar h5-index: 77, TOP 5
Turn off MathJax
Article Contents
Kunfeng Wang, Chao Gou, Yanjie Duan, Yilun Lin, Xinhu Zheng and Fei-Yue Wang, "Generative Adversarial Networks:Introduction and Outlook," IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 588-598, Oct. 2017. doi: 10.1109/JAS.2017.7510583
Citation: Kunfeng Wang, Chao Gou, Yanjie Duan, Yilun Lin, Xinhu Zheng and Fei-Yue Wang, "Generative Adversarial Networks:Introduction and Outlook," IEEE/CAA J. Autom. Sinica, vol. 4, no. 4, pp. 588-598, Oct. 2017. doi: 10.1109/JAS.2017.7510583

Generative Adversarial Networks:Introduction and Outlook

doi: 10.1109/JAS.2017.7510583
Funds:

the National Natural Science Foundation of China 61533019

the National Natural Science Foundation of China 71232006

the National Natural Science Foundation of China 91520301

More Information
  • Recently, generative adversarial networks (GANs) have become a research focus of artificial intelligence. Inspired by two-player zero-sum game, GANs comprise a generator and a discriminator, both trained under the adversarial learning idea. The goal of GANs is to estimate the potential distribution of real data samples and generate new samples from that distribution. Since their initiation, GANs have been widely studied due to their enormous prospect for applications, including image and vision computing, speech and language processing, etc. In this review paper, we summarize the state of the art of GANs and look into the future. Firstly, we survey GANs' proposal background, theoretic and implementation models, and application fields. Then, we discuss GANs' advantages and disadvantages, and their development trends. In particular, we investigate the relation between GANs and parallel intelligence, with the conclusion that GANs have a great potential in parallel systems research in terms of virtual-real interaction and integration. Clearly, GANs can provide substantial algorithmic support for parallel intelligence.

     

  • loading
  • Recommended by Associate Editor Derong Liu
  • [1]
    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets, " in Advances in Neural Information Processing Systems 27, Montreal, Quebec, Canada, 2014, pp. 2672-2680. http://dl.acm.org/citation.cfm?id=2969125
    [2]
    I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. New York, USA:MIT Press, 2016.
    [3]
    A. Radford, L. Metz, and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks, " arXiv: 1511. 06434, 2015. http://www.oalib.com/paper/4054347
    [4]
    L. J. Ratliff, S. A. Burden, and S. S. Sastry, "Characterization and computation of local Nash equilibria in continuous games, " in Proc. 51st Annu. Allerton Conf. Communication, Control, and Computing (Allerton), Monticello, IL, USA, 2013, pp. 917-924. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6736623
    [5]
    I. Goodfellow, "NIPS 2016 tutorial: generative adversarial networks, " arXiv: 1701. 00160, 2016. http://arxiv.org/abs/1701.00160
    [6]
    J. W. Li, W. Monroe, T. L. Shi, S. Jean, A. Ritter, and D. Jurafsky, "Adversarial learning for neural dialogue generation, " arXiv: 1701. 06547, 2017. http://arxiv.org/abs/1701.06547
    [7]
    L. T. Yu, W. N. Zhang, J. Wang, and Y. Yu, "SeqGAN: sequence generative adversarial nets with policy gradient, " arXiv: 1609. 05473, 2016. http://arxiv.org/abs/1609.05473
    [8]
    W. W. Hu and Y. Tan, "Generating adversarial malware examples for black-box attacks based on GAN, " arXiv: 1702. 05983, 2017. http://arxiv.org/abs/1702.05983
    [9]
    M. Chidambaram and Y. J. Qi, "Style transfer generative adversarial networks: Learning to play chess differently, " arXiv: 1702. 06762, 2017. http://arxiv.org/abs/1702.06762
    [10]
    Y. Bengio, "Learning deep architectures for AI, " Found. Trends Mach. Learn. , vol. 2, no. 1, pp. 1-127, Jan. 2009. http://dl.acm.org/citation.cfm?id=1658423.1658424&coll=DL&dl=GUIDE&CFID=358358199&CFTOKEN=27365575
    [11]
    D. P. Kingma and M. Welling, "Auto-encoding variational Bayes, " arXiv: 1312. 6114, 2013. http://www.oalib.com/paper/4042889
    [12]
    D. J. Rezende, S. Mohamed, and D. Wierstra, "Stochastic backpropagation and approximate inference in deep generative models, " arXiv: 1401. 4082, 2014. http://arxiv.org/abs/1401.4082
    [13]
    G. E. Hinton, T. J. Sejnowski, and D. H. Ackley, "Boltzmann machines: constraint satisfaction networks that learn, " Carnegiemellon Univ. , Pittsburgh, PA, USA, Tech. Rep. CMU-CS-84-119, 1984. https://www.researchgate.net/publication/239574281_Boltzmann_machines_Constraint_satisfaction_networks_that_learn
    [14]
    D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, "A learning algorithm for Boltzmann machines, " Cognit. Sci. , vol. 9, no. 1, pp. 147-169, Jan. 1985. http://www.sciencedirect.com/science/article/pii/S0364021385800124
    [15]
    G. E. Hinton, S. Osindero, and Y. W. Teh, "A fast learning algorithm for deep belief nets, " Neural Computat. , vol. 18, no. 7, pp. 1527-1554, May 2006.
    [16]
    Y. Bengio, É. Thibodeau-Laufer, G. Alain, and J. Yosinski, "Deep generative stochastic networks trainable by backprop, " arXiv: 1306. 1091, 2013. http://arxiv.org/abs/1306.1091
    [17]
    G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, Jul. 2006. http://www.ncbi.nlm.nih.gov/pubmed/16873662
    [18]
    Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning, " Nature, vol. 521, no. 7553, pp. 436-444, May 2015. http://www.nature.com/nature/journal/v521/n7553/abs/nature14539.html
    [19]
    A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks, " in Proc. 25th Int. Conf. Neural Information Processing Systems, Lake Tahoe, Nevada, USA, 2012, pp. 1097-1105. http://dl.acm.org/citation.cfm?id=2999257
    [20]
    K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun, "Deep residual learning for image recognition, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 770-778. https://www.computer.org/csdl/proceedings/cvpr/2016/8851/00/8851a770-abs.html
    [21]
    G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups, " IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82-97, Nov. 2012. http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=6296526
    [22]
    I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to sequence learning with neural networks, " in Advances in Neural Information Processing Systems 27: Annu. Conf. Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 2014, pp. 3104-3112. http://dl.acm.org/citation.cfm?id=2969033.2969173
    [23]
    D. He, W. Chen, L. W. Wang, and T. Y. Liu, "A game-theoretic machine learning approach for revenue maximization in sponsored search, " arXiv: 1406. 0728, 2014. http://dl.acm.org/citation.cfm?id=2540160
    [24]
    D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, "Mastering the game of Go with deep neural networks and tree search, " Nature, vol. 529, no. 7587, pp. 484-489, Jan. 2016. http://www.ncbi.nlm.nih.gov/pubmed/26819042
    [25]
    J. Schmidhuber, "Learning factorial codes by predictability minimization, " Neural Computat. , vol. 4, no. 6, pp. 863-879, Nov. 1992. http://dl.acm.org/citation.cfm?id=159784
    [26]
    Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, "Domain-adversarial training of neural networks, " J. Mach. Learn. Res. , vol. 17, no. 1, pp. 2096-2030, Jan. 2016. http://dl.acm.org/citation.cfm?id=2946704
    [27]
    W. Z. Chen, H. Wang, Y. Y. Li, H. Su, Z. H. Wang, C. H. Tu, D. Lischinski, D. Cohen-Or, and B. Q. Chen, "Synthesizing training images for boosting human 3D pose estimation, " in Proc. 4th Int. Conf. 3D Vision (3DV), Stanford, CA, USA, 2016, pp. 479-488. http://arxiv.org/abs/1604.02703
    [28]
    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, "Intriguing properties of neural networks, " arXiv: 1312. 6199, 2014. http://arxiv.org/abs/1312.6199
    [29]
    P. McDaniel, N. Papernot, and Z. B. Celik, "Machine learning in adversarial settings, " IEEE Secur. Priv. , vol. 14, no. 3, pp. 68-72, May-Jun. 2016. doi: 10.1109/MSP.2016.51
    [30]
    X. L. Wang, A. Shrivastava, and A. Gupta, "A-Fast-RCNN: hard positive generation via adversary for object detection, " arXiv: 1704. 03414, 2017. http://arxiv.org/abs/1704.03414
    [31]
    M. Arjovsky, S. Chintala, and L. Bottou, "Wasserstein GAN, " arXiv: 1701. 07875, 2017.
    [32]
    G. J. Qi, "Loss-sensitive generative adversarial networks on Lipschitz densities, " arXiv: 1701. 06264, 2017. http://arxiv.org/abs/1701.06264
    [33]
    A. Odena, "Semi-supervised learning with generative adversarial networks, " arXiv: 1606. 01583, 2016. http://arxiv.org/abs/1606.01583
    [34]
    M. Mirza and S. Osindero, "Conditional generative adversarial nets, " arXiv: 1411. 1784, 2014. http://www.oalib.com/paper/4066323
    [35]
    J. Donahue, P. Krähenbühl, and T. Darrell, "Adversarial feature learning, " arXiv: 1605. 09782, 2017. http://arxiv.org/abs/1605.09782
    [36]
    X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, "InfoGAN: interpretable representation learning by information maximizing generative adversarial nets, " in Proc. 30th Conf. Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 2172-2180. http://arxiv.org/abs/1606.03657
    [37]
    A. Odena, C. Olah, and J. Shlens, "Conditional image synthesis with auxiliary classifier GANs, " arXiv: 1610. 09585, 2017. http://arxiv.org/abs/1610.09585
    [38]
    D. Berthelot, T. Schumm, and L. Metz, "BEGAN: boundary equilibrium generative adversarial networks, " arXiv: 1703. 10717, 2017. http://arxiv.org/abs/1703.10717
    [39]
    I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, "Improved training of Wasserstein GANs, " arXiv: 1704. 00028, 2017. http://arxiv.org/abs/1704.00028
    [40]
    C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. H. Wang, and W. Z. Shi, "Photo-realistic single image super-resolution using a generative adversarial network, " arXiv: 1609. 04802, 2017. http://arxiv.org/abs/1609.04802
    [41]
    E. Santana and G. Hotz, "Learning a driving simulator, " arXiv: 1608. 01230, 2016. http://arxiv.org/abs/1608.01230
    [42]
    C. Gou, Y. Wu, K. Wang, F. -Y. Wang, and Q. Ji, "Learning-by-synthesis for accurate eye detection, " in Proc. 23rd Int. Conf. Pattern Recognition (ICPR), Cancun, Mexico, 2016, pp. 3362-3367. http://ieeexplore.ieee.org/document/7900153/
    [43]
    C. Gou, Y. Wu, K. Wang, K. F. Wang, F. -Y. Wang, and Q. Ji, "A joint cascaded framework for simultaneous eye detection and eye state estimation, " Pattern Recognit. , vol. 67, pp. 23-31, Jul. 2017. http://www.sciencedirect.com/science/article/pii/S0031320317300250
    [44]
    A. Shrivastava, T. Pfister, O. Tuzel, J. Susskind, W. D. Wang, and R. Webb, "Learning from simulated and unsupervised images through adversarial training, " arXiv: 1612. 07828, 2016. http://arxiv.org/abs/1612.07828
    [45]
    R. Huang, S. Zhang, T. Y. Li, and R. He, "Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis, " arXiv: 1704. 04086, 2017. http://arxiv.org/abs/1704.04086
    [46]
    J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks, " arXiv: 1703. 10593, 2017. http://arxiv.org/abs/1703.10593
    [47]
    Y. Z. Zhang, Z. Gan, and L. Carin, "Generating text via adversarial training, " Proc. Workshop on Adversarial Training, Barcelona, Spain, 2016.
    [48]
    S. Pascual, A. Bonafonte, and J. Serrá, "SEGAN: speech enhancement generative adversarial network, " arXiv: 1703. 09452, 2017. http://arxiv.org/abs/1703.09452
    [49]
    S. Reed, Z. Akata, X. C. Yan, L. Logeswaran, B. Schiele, and H. Lee, "Generative adversarial text to image synthesis, " in Proc. 33rd Int. Conf. Machine Learning, New York, NY, USA, vol. 48, pp. 1060-1069, 2016. http://dl.acm.org/citation.cfm?id=3045503
    [50]
    H. Zhang, T. Xu, H. S. Li, S. T. Zhang, X. L. Huang, X. G. Wang, and D. Metaxas, "StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks, " arXiv: 1612. 03242, 2016. http://arxiv.org/abs/1612.03242
    [51]
    J. Ho and S. Ermon, "Generative adversarial imitation learning, " in Proc. 30th Conf. Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 4565-4573. http://arxiv.org/abs/1606.03476
    [52]
    C. Finn, P. Christiano, P. Abbeel, and S. Levine, "A connection between generative adversarial networks, inverse reinforcement learning, and energy-based models, " arXiv: 1611. 03852, 2016. http://arxiv.org/abs/1611.03852
    [53]
    D. Pfau and O. Vinyals, "Connecting generative adversarial networks and actor-critic methods, " arXiv: 1610. 01945, 2017. http://arxiv.org/abs/1610.01945
    [54]
    E. Choi, S. Biswal, B. Malin, J. Duke, W. F. Stewart, and J. M. Sun, "Generating multi-label discrete electronic health records using generative adversarial networks, " arXiv: 1703. 06490, 2017. http://arxiv.org/abs/1703.06490v1
    [55]
    F. -Y. Wang, "Parallel system methods for management and control of complex systems, " Control Dec. , vol. 19, no. 5, pp. 485-489, 514, May 2004. http://en.cnki.com.cn/Article_en/CJFDTotal-KZYC200405001.htm
    [56]
    F. -Y. Wang, "Computational experiments for behavior analysis and decision evaluation of complex systems, " J. Syst. Simulat. vol. 16, no. 5, pp. 893-897, May 2004. http://en.cnki.com.cn/Article_en/CJFDTotal-XTFZ200405008.htm
    [57]
    F. -Y. Wang, J. Zhang, Q. L. Wei, X. H. Zheng, and L. Li, "PDP: parallel dynamic programming, " IEEE/CAA J. Automat. Sinica, vol. 4, no. 1, pp. 1-5, Jan. 2017. http://ieeexplore.ieee.org/document/7815546/
    [58]
    T. X. Bai, S. Wang, Z. Shen, D. P. Cao, N. N. Zheng, and F. -Y. Wang, "Parallel robotics and parallel unmanned systems: framework, structure, process, platform and applications, " Acta Automat. Sinica, vol. 43, no. 2, pp. 161-175, Feb. 2017. http://www.en.cnki.com.cn/Article_en/CJFDTOTAL-MOTO201702001.htm
    [59]
    F. -Y. Wang, X. Wang, L. X. Li, and L. Li, "Steps toward parallel intelligence, " IEEE/CAA J. Automat. Sinica, vol. 3, no. 4, pp. 345-348, Oct. 2016. http://ieeexplore.ieee.org/document/7589480/
    [60]
    K. F. Wang, C. Gou, and F. -Y. Wang, "Parallel vision: an ACP-based approach to intelligent vision computing, " Acta Automat. Sinica, vol. 42, no. 10, pp. 1490-1500, Oct. 2016. http://www.aas.net.cn/EN/Y2016/V42/I10/1490
    [61]
    F. -Y. Wang, "On the modeling, analysis, control and management of complex systems, " Complex Syst. Complex. Sci. , vol. 3, no. 2, pp. 26-34, Jun. 2006. http://en.cnki.com.cn/Article_en/CJFDTotal-FZXT200602003.htm
    [62]
    F. -Y. Wang, D. R. Liu, G. Xiong, C. J. Cheng, and D. B. Zhao, "Parallel control theory of complex systems and applications, " Complex Syst. Complex. Sci. , vol. 9, no. 3, pp. 1-12, Sep. 2012. http://en.cnki.com.cn/Article_en/CJFDTOTAL-FZXT201203002.htm
    [63]
    F. -Y. Wang, "Parallel control: a method for data-driven and computational control, " Acta Automat. Sinica, vol. 39, no. 4, pp. 293-302, Apr. 2013. http://en.cnki.com.cn/Article_en/CJFDTOTAL-MOTO201304002.htm
    [64]
    L. Li, Y. L. Lin, D. P. Cao, N. N. Zheng, and F. -Y. Wang, "Parallel learning—a new framework for machine learning, " Acta Automat. Sinica, vol. 43, no. 1, pp. 1-8, Jan. 2017. http://www.en.cnki.com.cn/Article_en/CJFDTOTAL-MOTO201701002.htm

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(10)

    Article Metrics

    Article views (2505) PDF downloads(365) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return