• Evaluation and Assessment:
    Self-Directed Assessment

    by WolfBrown Associates


    The Weill Music Institute strives for excellence in all its programs, and each year many of its initiatives are examined using an assessment inquiry question. Successful assessments inform the planning of future seasons in ways that move a program forward and increase its ability to meet the needs of those served. It is particularly vital to assess a program in its pilot year, since tremendous growth often occurs. Additionally, when patterns of documentation are established early in the development of a program, this documentation is more likely to contribute to the growth of the program, as well as to its ability to be shared.

  • Internal Processes Leading to Successful Events

    In 2009–2010, we asked ourselves a question: What are the actions staff members have taken to create successful events in Musical Connections settings? We had two primary goals in shaping that question:

    • • To contribute to the external evaluation being completed by WolfBrown Associates, who evaluated the impact of the Musical Connections program through observations, interviews, and research of participants
    • • To understand the strengths and weaknesses of the planning in 2009–2010 in order to improve those practices for the next year

    To assess the program, our initial task was to document the daily work of Musical Connections to develop and share a deeper understanding of how the program plays out on the ground and what makes in more and less successful.

    1. Operational Evaluation

    The devil is in the details. It does matter if there is a good microphone. It does make a big difference if all the musicians know the time for the rehearsal or concert. It is important to know the rules for interactions in a juvenile justice facility. Without that kind of precision, the excellence of Musical Connections would never make it through the first set. As a result, the Carnegie Hall staff have designed a thorough tracking process designed to help staff document what it takes to implement the program well and to develop more thorough and efficient ways of making a very complex program run smoothly.


    Examples of these tools:

    Download Spreadsheet:
     Staff Assessment Tool Crossroads 
    Download Spreadsheet: Staff Assessment Tool Crossroads: Manuel
    Download Spreadsheet: Staff Assessment Tool Crossroads: All  

    2. Documenting Performances to Discover What Works

    As program evaluators, our initial task was to document the daily work of Musical Connections to develop and share a deeper understanding of how the program plays out on the ground and what makes it more and less successful. To do this, we

    • • selected a subset of performers and partners that represented a cross-section of musical genres, artists, and community partners for intensive study
    • • observed these artists and their work closely, listening in on planning sessions, attending set-up and performances, taking part in post-performance conversations, and so on
    • • interviewed artists, partners, and participants to get their perspectives on the experience
    • • attended professional development sessions to understand the emerging issues, challenges, and rewards of the program

     

    From our experiences, we generated a set of field notes for each event, coded for items to think about and act on. We shared these notes with program staff and artists, inviting feedback and further thinking.

    3. Results

    A major result was a summary, “What Works in Musical Connections,” that we shared with participating artists, asking them for reactions, extensions, and examples. Based on our “What Works” discussions, it was clear that quality in Musical Connections includes much more than technical musical ability. Across sites as varied as hospital corridors and auditoriums in maximum security prisons, successful Musical Connections artists

    • • interact from the moment they cross the threshold
    • • invite others into a meaningful musical and human experience
    • • have a deep respect and curiosity for the music that non-professionals create
    • • are willing to use their musical skills to develop others’ musical lives


    Download PDF: What Works in Musical Connections  

    4. Musical Evaluation

    Even with an emerging consensus on what works in Musical Connections, no one wants a one-size-fits-all approach to excellence. Quality looks, sounds, and feels different when a string quartet plays for a family in a critical care ward and when a world musician and her band play songs about love and freedom in a women’s prison. To spark and sustain these discussions, staff of Musical Connections are developing a set of exchanges between musicians and among staff and musicians to keep the discussion alive.

    •  Peer-to-Peer Observation: Ensembles regularly go out after events to chew over the highs, lows, and surprises of the night’s performance. Taking a page from those sessions, Musical Connections staff members have designed a process for peer-to-peer observation. Beginning in Year 2, the entire roster of musicians will be traveling to one another’s performances to listen, participate, and think about what’s good or effective work. As the program develops a national network of sites, the hope is for a vigorous exchange about how to conduct this kind of peer-to-peer learning.
    • Download PDF: Artist Observation Form
    •  Mid-Year Conversations with Artists: Musicians who step up to do this new kind of work need and want a clear sense of what counts as an effective performance and an equally clear sense of whether they are hitting their marks. Musical Connections senior staff hold a mid-year feedback session with each ensemble to discuss their work so far and to field questions and comments from the artists.
    • Like the peer-to-peer exchange, this is a new and evolving tool designed to make everyone—not just musicians—understand what defines high quality performance in Musical Connections. Like the peer observations, the guidelines for these feedback conversations will evolve with time and experience. Again, as a national network develops, the input of national partners will help to build a more robust approach.