Ngaa-bi-nya and data collection

Following is a detailed description of how to apply the Ngaa-bi-nya framework to data collection. The prompts can be used to both guide the questions and interrogate the answer.

Respecting the landscape

The landscape domain of Ngaa-bi-nya represents the broadest context a program is located in and influenced by. It is akin to a ‘system’ level and the external influences identified in an ecological model of health^[1]Bronfenbrenner, U. (1979). The ecology of human development. Cambridge, MA: Harvard University Press, and signifies the whole-of-life perspective of the Aboriginal definition of health. Landscape factors influence the foundations and capacity of programs to deliver care, and therefore underpin other domains of Ngaa-bi-nya. Prompts to understand the landscape are outlined below; to understand landscape factors, data should be gathered and discussed about as many of the following items as possible.

History

Consider …

• the history of the local area, and experiences of the traditional owners
• the history of program establishment
• the extent to which the program acknowledges the act and nature of colonisation, dispossession and disempowerment experienced by many Aboriginal and Torres Strait Islander people, such as child removal, racism, exclusion, poverty, trauma and poor health occurring across generations.

Environment

Consider …

• local Aboriginal and Torres Strait Islander population characteristics, including proportion, recent changes and comparisons to local, state and national proportions
• the location’s socioeconomic position, housing affordability, education and employment rates
• differences and similarities between the lives of Aboriginal and Torres Strait Islander peoples compared to others in the local area
• proximity to and accessibility of major health and social support services, and barriers and enablers of this accessibility.

Programs and services

Consider …

• other programs that influence the program, its resources, services and accessibility
• service-level collaborations and shared-care arrangements with other programs and services
• the extent to which roles, responsibilities and expectations across related organisations and sectors are articulated and realised
• the extent to which coordinated mechanisms for sharing information, resources and responsibilities occur with other related programs and services
• investment in relevant infrastructure to meet need and serve future generations, such as building and using new technologies, service delivery guidelines, and workforce development and supervision strategies.

Self-determination

Consider …

• the extent to which local Aboriginal and Torres Strait Islander people have been engaged in identifying needs and setting priorities
• the role of Aboriginal and Torres Strait Islander people in program design, delivery and governance
• the role and influence of Aboriginal and Torres Strait Islander people in research, monitoring and evaluation related to the program
• the extent to which Aboriginal and Torres Strait Islander people’s ways of relating, caregiving and doing business are embedded throughout.

Policy

Consider …

• the alignment between legislation and policies in relation to the issues addressed by the program, and any shifts in that alignment
• the role Aboriginal and Torres Strait Islander people have in policy development and reviews that impact on the program
• the extent of intergovernmental and intersectoral collaboration to support the program
• the impact of quality assurance and accreditation processes on the program
• possible sources of data about landscape factors include population- and service-level data, books and media, research publications, service delivery reports, program websites, and interviews with program participants, staff, informal supports, key stakeholders and community members.

Data sources

Possible sources of data about landscape factors include population- and service-level data, books and media, research publications, service delivery reports, program websites, and interviews with program participants, staff, informal supports, key stakeholders, and community members.

Resources

Usually, physical and financial resources only are counted in program evaluations, which mean a wide range of other resources are often overlooked. The resources domain of the Ngaa-bi-nya framework seeks to identify the human, material, non-material and in-kind resources, and informal economies and relationships that often support Aboriginal and Torres Strait Islander programs.

Financial resources

Consider the adequacy of financial resources to …

• meet the demand for services and support, and the needs of the local Aboriginal and Torres Strait Islander people
• support Aboriginal and Torres Strait Islander workforce development
• support program monitoring and evaluation and participation in research
• support transfer of knowledge and policy advocacy.

Human resources

Consider …

• how program development processes draw on local Aboriginal and Torres Strait Islander people’s knowledges and resources
• the mix of skills, experience and roles among program staff, support staff and volunteers
• networks that support the program, and how they are used
• the in-kind contributions, volunteer community participation and informal supports drawn into the program, and their roles and outputs
• the types of culturally-relevant training and supervision that program staff, board members and other significant support people undertake, and provide to others.

Material resources

Consider …

• the use of data and other evidence to inform program design and development
• the extent to which equipment and information technology are sufficient to meet program needs, with growth needs identified
• the extent to which the physical location is suitable to meet program needs and future growth
• plans, theories, stakeholders, mapping, investments and relevant community and political contributions that have been made for the program and its sustainability.

Possible sources

Possible sources of data about resources include service agreements, strategic plans, service delivery record-keeping, budgets, income and expenditure statements, service and staff reports, relationship mapping, media, and interviews with program participants, staff, informal supports, key stakeholders and community members.

Ways of working

In this domain the focus is on the delivery of programs and understanding the types of activities, relationships, frameworks, principles and accountability mechanisms that support program delivery. Prompts reflect culturally-safe, holistic processes through which Aboriginal and Torres Strait Islander people work to support others. Particularly drawn on here are work of Bulman and Hayes^[2]Bulman, J., & Hayes, R. (2011). Mibbinbah and spirit healing: Fostering safe, friendly spaces for Indigenous males in Australia. International Journal of Men’s Health, 10(1), 6–25, ^[3]Haswell, M., Blignault, I., Fitzpatrick, S., & Jackson Pulver, L. (2013). The social and emotional wellbeing of Indigenous youth: Reviewing and extending the evidence and examining its … Continue reading and Williams^[4]Williams, M. (2015). Connective services: Post-prison release support in an urban Aboriginal population (Doctoral dissertation). Sydney: UNSW Sydney, Australia.

Holistic caregiving principles

Consider …

• caregiving is holistic, addressing multiple social, emotional, mental, environmental, spiritual and physical elements of wellbeing
• caregiving is provided beyond the individual person to families and across generations
• the program is sensitive to the impacts of trauma, developing opportunities for individual and collective healing
• the program addresses social determinants of health and wellbeing
• intergovernmental and interdisciplinary partnerships support the program
• actions are taken at workforce, community, services and systems levels.

Quality caregiving in practice

Consider how program staff and other care providers …

• negotiate individual and group safety, shared agreements and boundaries with program participants, including reflecting on and enhancing these
• are positive role models making progress in their own healthcare, relationships and wellbeing
• develop trusting relationships with program participants through which to influence change
• are non-judgmental, ethical, responsive and solution-focussed
• promote and achieve participant engagement in programs
• celebrate achievements, remembering days of significance, anniversaries and turning points
• facilitate connection to Aboriginal and Torres Strait Islander culture and identity, through strengthening individual, group and community connections
• provide coordinated case management and wrap-around care, supported by partnerships, shared-care arrangements and referral pathways with other services and supports
• build capacity among participants through leadership, role-modelling, mentoring and planning, including for knowledge and skills development, autonomy and empowerment
• create opportunities for program participants’ family and loved ones to feel comfortable, access support, and strengthen their capacity to provide support
• provide timely continuity of care and follow-up.

Staff support and development

Consider …

• mechanisms for recognising and celebrating achievements by staff, volunteers and governance committees
• the extent to which staff influence the development of the program, aligned with their experience and issues they witness in the community
• whether the aim of the program is meaningful to and motivates staff, including through periods of change and difficulty
• the extent to which staff feel safe and supported to discuss challenges and make improvements
• peer support opportunities among staff, to share strategies and solutions, address challenges and grow together
• the access that program staff, volunteers and other support people have to education, training and skills development, and career pathways.

Sustainability

Consider …

• the extent to which Aboriginal and Torres Strait Islander people experienced in relevant program delivery are involved
• the extent to which the program is self-determined by the Aboriginal and Torres Strait Islander community, and community controlled
• how timeframes and expectations of the program align
• the extent to which local community needs, resources available, actions taken and broader policies align
• how change is managed at program and workforce levels, especially so that they are respectful of cultural protocols, history and inputs over time, community needs and likely characteristics of future generations
• the extent to which collaboration with other services and supports occurs, with agreements and reviews to support these
• the engagement of program staff, volunteers and other supports in translating learnings about program successes to other contexts, including through presentations, networking, advisory roles, policy submissions and media.

Evaluation

Consider …

• the extent to which accountability, monitoring and evaluation processes are embedded in the program, and resourced
• the extent to which culturally-relevant data collection tools are used, and the involvement of Aboriginal and Torres Strait Islander people in leading evaluation design, data collection, analysis and reporting, and other knowledge exchange actions
• the impact of conducting evaluation and other quality assurance processes on the program

Data sources

Sources of data to best understand ways of working include service agreements, strategic plans, service delivery and staff reports, media, program webpages, presentations, networking, and individual and group interviews with program participants, staff, informal supports, key stakeholders and community members.

Learnings

The learnings domain of Ngaa-bi-nya prompts users to reflect on insights gained and what the range of stakeholders and participants have learned, in addition to assessing the extent to which program objectives were met. This domain is to understand progress made, including in empowerment, attitude shifts, relationship strengthening and self-determination of Aboriginal and Torres Strait Islander peoples. Learnings relate to movement – of ideas, of actions, of purpose, of ways of being, of ways of relating. These are important Wiradjuri values, and are important to other Aboriginal peoples^[5]Sheehan, N. (2004). Indigenous knowledge and higher education: Instigating relational education in a neo-colonial context (PhD thesis). University of Queensland, Brisbane, Australia. Given few tools are available to assess effectiveness of programs generally, of holistic care or of making progress, particularly from Aboriginal and Torres Strait Islander people’s perspectives, a dedicated process of critical reflection in dialogue with the Evaluation Reference Group is recommended.

Self-determination

Consider …

• when, how and what aspects of self-determination and rights were or were not experienced, including informed leadership of local community members in priority-setting and planning, and community-control in service delivery
• the commitments made to making progress as well as to outcomes, with progress expressed also through perseverance, despite challenges, constraints and set-backs
• whether non-Indigenous people undertake critical reflection on their standpoints, engage in cultural awareness and anti-racism training or demonstrate commitment to strengthening personal and professional relationships with Aboriginal and Torres Strait Islander people.

Aboriginal and Torres Strait Islander cultural care

Consider …

• the extent to which appropriate sharing of knowledges and processes strengthen Aboriginal and Torres Strait Islander people’s cultural identity and personal security
• the extent to which the program preserves Aboriginal and Torres Strait Islander cultural heritage
• the extent to which the program strengthens connection to country, waterways, knowledges, material items and people
• the ways that taking responsibility for future generations are expressed by Aboriginal and Torres Strait Islander Elders, leaders and others involved in program delivery, including through role-modelling, mentoring and knowledge exchange
• the opportunities for relationships to begin, or to strengthen, alter, cease or otherwise transform
• the strength of ties, and how relationships are experienced such as through trust, integrity, equality, reciprocity, flexibility and sharing
• the use of strengths-based program delivery and connecting people to supports they prioritise, rather than assessing for and identifying deficits or making comparisons to non-Indigenous peoples.

Healing

Consider …

• the extent to which there are safe processes for reflection, access to therapeutic care, identifying inner strengths and addressing trauma, suffering, loss and grieving in individual as well as group contexts
• the extent to which the program addresses experiences of racism
• the opportunities for greater understanding and empathy for others and strengthening relationships, including with self, older and younger generations, and people of other cultures
• the opportunities to enhance re/connections between Aboriginal and Torres Strait Islander peoples and cultural identity, knowledges and practices
• opportunities to support others as part of one’s own healing process.

Developing the evidence base

Consider …

• how the program contributes to an evidence base by developing culturally relevant tools, methods, measures, indicators and benchmarking, as well as record-keeping and monitoring
• the opportunities for building the capacity of Aboriginal and Torres Strait Islander staff and community members in research, monitoring, and evaluation
• strategies and processes for decolonise research, evaluation, and program delivery, including raising awareness among enabling systems, and mainstream researchers, partners, and stakeholders.

Data sources

Possible sources of data for learning include service records and reports, routinely collected administrative and linked data, pre- and post-data collection, cost–benefit analyses, observations, field notes and case studies, and relationship and resource mapping. Critical reflection is recommended such as on adaptations of programs implemented elsewhere and change management, with feedback from the Evaluation Reference Group and other stakeholders, and individual and group interviews with program participants, staff, informal supports, key stakeholders, and community members.

Note: This text was first published at Evaluation Journal of Australasia 2018, Vol. 18(1) 6–20.