Project Proposal

The goal, planned features & our approach

The Team

Faculty Sponsor

Client

Meetings with the Client

Goal and Motivation

The goal is to help students learn sheet music and improve their singing ability. This will be accomplished by developing an application to provide choir students with real-time, interactive feedback as they practice by analyzing pitch, duration, dynamic, tempo, rhythm, and phrasing compared to the sheet music provided by a choir instructor. The app will provide static and dynamic exercises developed based on the sheet music. The app will also provide an integrated communication tool between student peers and between students and teachers to facilitate a conductive learning environment. We focus first on choir students. Choir members are often expected to learn and practice on their own, but many members either don’t practice, don’t know how to practice, or don’t practice correctly. The impact of these circumstances is felt when the choir practices; time is wasted reviewing and teaching students their parts instead of working on integrating the choir together and focusing on the minutia of the piece. Students tend to struggle when practicing independently for reasons such as lack of training, discipline or self-confidence. Current applications exist which can analyze music and provide feedback for students by showing the discrepancy between what the student is performing and the performance that was expected in regards to pitch and duration. The applications that provide this feedback rely on a set series of exercises and abstract away the actual sheet music and multi-part harmony. They also do not provide interactive guidance to students who are struggling.

Approach

Core Features for Singers

1. A web and mobile application which provides playback features and a method to record the student’s performance to provide real-time, dynamic feedback.
2. Statistical analysis
1. Statistics would be presented in a variety of visual and textual formats and will be integrated into the sheet music directly.
2. While the student is singing their part, the app will perform calculations on a variety of musical areas including highlighting discrepancies in perceived and expected pitch, duration, dynamic, tempo, rhythm, and phrasing.
3. Exercises
1. Based on user feedback, exercises will be provided to help students in the general areas of pitch, duration, dynamic, tempo, rhythm, and phrasing.
2. After the app has analyzed a student’s performance, exercises will be developed dynamically to target areas of difficulty.
4. An integrated communication tool will allow students to query their peers and teacher about their current progress, to get suggestions for help, and to provide peer support to fellow students.

Core Features for Teachers

1. Statistics about individual student performance
1. The teacher will be able to see areas where the most students are struggling and/or where particular part(s) of the choir (e.g. Sopranos) are struggling.
2. The integrated communication tool will provide a connection to remote or struggling students. Through this connection, teachers can provide guidance, reminders, and additional feedback.

Additional Features for Singers

1. Sheet music image parser
2. Support other instruments as input
3. Teacher-curated exercises

Additional Features for Teachers

1. The teacher will be able to upload their own copies of their sheet music per copyright laws into the app which can then be used for rehearsal.
2. Digital sheet music editing
3. Integration with other sheet music databases while adhering to copyright laws.

Novel Features and Functionalities

The main novel feature of this approach lies in the real-time, interactive feedback that assists students as they practice. Current systems provide feedback such as showing you the visual discrepancy between the pitch and duration that you are singing versus what is expected and expect you to fix your singing yourself. Our approach will gather statistics across multiple musical areas including pitch, duration, dynamic, tempo, rhythm, and phrasing. Using these statistics, the app will formulate exercises to guide the student to perfect the areas in the score where they are struggling. For example, if the student is struggling with a certain few bars in the music then the app will formulate an exercise with just the rhythm and will get the student to perfect that rhythm before then focusing on the pitch and then putting them together. These interactive sessions will help students who lack training to improve.

The other novel feature will be the communication tools between the student and the teacher. Our app will provide the teacher with overall statistics over all the practice sessions and will help the teacher to guide group practice by highlighting areas where the group is struggling in the piece. Some choirs have a choir director who is trained to guide practice and already do this in practice. This feature would help choirs whose choir director is inexperienced or in choirs like our acapella group on campus which is student led and lacks a dedicated choir director.

Technical Challenges

• We plan to use JavaScript to represent sheet music and playback, but we don’t know much about JavaScript
• We plan to develop a mobile app for this project, but we don’t currently have much experience in mobile app development
• We plan to develop a mobile app for this project, but we don’t currently have much experience in mobile app development

Milestone 1 (September 30, 2019)

• Compare technical tools for vocal input, digital sheet music representation, and playback of digital sheet music.
• Provide small (“hello world”) demo(s) to evaluate the tools for vocal input, digital sheet music representation, playback of digital sheet music.
• Resolve Technical challenges in learning JavaScript, getting experience with mobile app development, and gaining understanding and experience with machine learning.
• Compare and select collaboration tools for software development, documents/presentations, communication, and task calendar.
• Create Requirement Document
• Create Design Document
• Create Test Plan

Milestone 2 (October 28, 2019)

• Implement, test, and demo sheet music importation, digital representation, and playback.
• Implement, test, and demo vocal recording, digital representation, and integration with music playback.

Milestone 3 (November 25, 2019)

• Implement, test, and demo basic statistic gathering in areas of pitch and duration.
• Implement, test, and demo development of interactive exercises based on statistics gathered concerning pitch and duration.

Task Matrix