DeepSheet

A sheet music generator based on deep learning

Yu Lun Hsu, Chi Po Lin, Bo Chen Lin, Hsu-Chan Kuo, Wen Huang Cheng, Min-Chun Hu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Sheet music has long been regarded as one of the most effective medias for musicians, music players, and amateurs to communicate with each other. It is also an intuitive way for non-professionals to learn how to play a musical instrument or sing a song. However, not all composers have willingness to share their own sheet music, especially those protected by strict copyright regulations. For amateurs or novice musicians without the capability to distinguish the chords by their own ears, it would be difficult for them to enjoy the happiness of playing other's music, or singing songs. In order to provide the beginners with an effective way to play music or sing songs without a given sheet music, we developed a sheet music generator-'DeepSheet', which was engineered based on deep learning techniques. Basically, the DeepSheet system is mainly composed of three distinctive components as follows: 1. voice separation of the audio song, 2. chord estimation of the background music, and 3. alignment of lyrics and music beats. The experimental results drawn from more than 150 songs of the Beatles and Queen indicated that the newly developed DeepSheet system has the capability and sensibility to generate sheet music with accuracy of approximately 76%.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages285-290
Number of pages6
ISBN (Electronic)9781538605608
DOIs
Publication statusPublished - 2017 Sep 5
Event2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017 - Hong Kong, Hong Kong
Duration: 2017 Jul 102017 Jul 14

Publication series

Name2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017

Other

Other2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017
CountryHong Kong
CityHong Kong
Period17-07-1017-07-14

Fingerprint

Musical instruments
Deep learning

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Media Technology

Cite this

Hsu, Y. L., Lin, C. P., Lin, B. C., Kuo, H-C., Cheng, W. H., & Hu, M-C. (2017). DeepSheet: A sheet music generator based on deep learning. In 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017 (pp. 285-290). [8026272] (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMEW.2017.8026272
Hsu, Yu Lun ; Lin, Chi Po ; Lin, Bo Chen ; Kuo, Hsu-Chan ; Cheng, Wen Huang ; Hu, Min-Chun. / DeepSheet : A sheet music generator based on deep learning. 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 285-290 (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017).
@inproceedings{23bbcc64ee8c4b20a8bd32ca90e9a5fc,
title = "DeepSheet: A sheet music generator based on deep learning",
abstract = "Sheet music has long been regarded as one of the most effective medias for musicians, music players, and amateurs to communicate with each other. It is also an intuitive way for non-professionals to learn how to play a musical instrument or sing a song. However, not all composers have willingness to share their own sheet music, especially those protected by strict copyright regulations. For amateurs or novice musicians without the capability to distinguish the chords by their own ears, it would be difficult for them to enjoy the happiness of playing other's music, or singing songs. In order to provide the beginners with an effective way to play music or sing songs without a given sheet music, we developed a sheet music generator-'DeepSheet', which was engineered based on deep learning techniques. Basically, the DeepSheet system is mainly composed of three distinctive components as follows: 1. voice separation of the audio song, 2. chord estimation of the background music, and 3. alignment of lyrics and music beats. The experimental results drawn from more than 150 songs of the Beatles and Queen indicated that the newly developed DeepSheet system has the capability and sensibility to generate sheet music with accuracy of approximately 76{\%}.",
author = "Hsu, {Yu Lun} and Lin, {Chi Po} and Lin, {Bo Chen} and Hsu-Chan Kuo and Cheng, {Wen Huang} and Min-Chun Hu",
year = "2017",
month = "9",
day = "5",
doi = "10.1109/ICMEW.2017.8026272",
language = "English",
series = "2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "285--290",
booktitle = "2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017",
address = "United States",

}

Hsu, YL, Lin, CP, Lin, BC, Kuo, H-C, Cheng, WH & Hu, M-C 2017, DeepSheet: A sheet music generator based on deep learning. in 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017., 8026272, 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017, Institute of Electrical and Electronics Engineers Inc., pp. 285-290, 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017, Hong Kong, Hong Kong, 17-07-10. https://doi.org/10.1109/ICMEW.2017.8026272

DeepSheet : A sheet music generator based on deep learning. / Hsu, Yu Lun; Lin, Chi Po; Lin, Bo Chen; Kuo, Hsu-Chan; Cheng, Wen Huang; Hu, Min-Chun.

2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 285-290 8026272 (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - DeepSheet

T2 - A sheet music generator based on deep learning

AU - Hsu, Yu Lun

AU - Lin, Chi Po

AU - Lin, Bo Chen

AU - Kuo, Hsu-Chan

AU - Cheng, Wen Huang

AU - Hu, Min-Chun

PY - 2017/9/5

Y1 - 2017/9/5

N2 - Sheet music has long been regarded as one of the most effective medias for musicians, music players, and amateurs to communicate with each other. It is also an intuitive way for non-professionals to learn how to play a musical instrument or sing a song. However, not all composers have willingness to share their own sheet music, especially those protected by strict copyright regulations. For amateurs or novice musicians without the capability to distinguish the chords by their own ears, it would be difficult for them to enjoy the happiness of playing other's music, or singing songs. In order to provide the beginners with an effective way to play music or sing songs without a given sheet music, we developed a sheet music generator-'DeepSheet', which was engineered based on deep learning techniques. Basically, the DeepSheet system is mainly composed of three distinctive components as follows: 1. voice separation of the audio song, 2. chord estimation of the background music, and 3. alignment of lyrics and music beats. The experimental results drawn from more than 150 songs of the Beatles and Queen indicated that the newly developed DeepSheet system has the capability and sensibility to generate sheet music with accuracy of approximately 76%.

AB - Sheet music has long been regarded as one of the most effective medias for musicians, music players, and amateurs to communicate with each other. It is also an intuitive way for non-professionals to learn how to play a musical instrument or sing a song. However, not all composers have willingness to share their own sheet music, especially those protected by strict copyright regulations. For amateurs or novice musicians without the capability to distinguish the chords by their own ears, it would be difficult for them to enjoy the happiness of playing other's music, or singing songs. In order to provide the beginners with an effective way to play music or sing songs without a given sheet music, we developed a sheet music generator-'DeepSheet', which was engineered based on deep learning techniques. Basically, the DeepSheet system is mainly composed of three distinctive components as follows: 1. voice separation of the audio song, 2. chord estimation of the background music, and 3. alignment of lyrics and music beats. The experimental results drawn from more than 150 songs of the Beatles and Queen indicated that the newly developed DeepSheet system has the capability and sensibility to generate sheet music with accuracy of approximately 76%.

UR - http://www.scopus.com/inward/record.url?scp=85031665400&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031665400&partnerID=8YFLogxK

U2 - 10.1109/ICMEW.2017.8026272

DO - 10.1109/ICMEW.2017.8026272

M3 - Conference contribution

T3 - 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017

SP - 285

EP - 290

BT - 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Hsu YL, Lin CP, Lin BC, Kuo H-C, Cheng WH, Hu M-C. DeepSheet: A sheet music generator based on deep learning. In 2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 285-290. 8026272. (2017 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2017). https://doi.org/10.1109/ICMEW.2017.8026272