Notes on: Sablan, J (2019) can You Really Measure That? Combining Critical Race Theory and Quantitative Methods. American Educational Research Journal. 56 (1): 178 – 203  DOI: 10.3102/0002831218798325 Dave Harris [Very informative. I could only follow the basics of the statistical modelling. I learned about non-dominant cultural capital and welcomed the stuff on how people of colour resist racism with culturl capital of their own. I am sure working class people do -- I have suggested one bit of this is know-how on 'poaching'] The argument is sustained by using a survey of undergraduates to quantify their community cultural wealth (CCW) using a CRT framework. We can use quantitative data to look at public consumption of educational policy research, including problems of access and equity to students of colour. However CRT training usually thinks of stats and quantitative reasoning as 'related to white supremacist origins', so that strong statistical analysis is usually seen as 'entirely antithetical' (178). Nevertheless there have been a range of quantitative studies to examine racial disparity, ranging from 'teacher-student racial mismatch to student discipline disparities and college affordability'. [and lots of soft quantification]. CRT students might doubt whether statistical and causal inferences can possibly get at the complexities of structural racism, however. Nevertheless CRT can be an appropriate framework for quantitative studies, and quants have been an important part of critical legal studies in the past. It needs to reach its full potential. We might demonstrate this by looking at 'community cultural wealth'  (CCW) a concept often cited in CRT that emphasises students' assets. Reviewing CRT first, it's clear that the roots lie in critical legal studies, noticing that attention to race was missing, and legal scholarship, including its intersection with gender discrimination law. White supremacy and structural racism was ignored in legal process, and CRT used legal analysis to show how the implementation of the law subordinated racial groups, say in various legal cases where apparent wins for desegregation actually revealed an alignment between white interests and black goals all along [presumably some sort of reformist result that maintained white supremacy -- see  examples here]. Generally, 'legal and civil rights progress is intimately tied to the establishment of white supremacy' (179). The same sort of argument can be extended to studies in education [reference to Ladson-Billings and Tate 1995). In education, CRT was adopted to emphasise the centrality of race, racism and white supremacy in educational structures and social practice. Previous theories of gender and class were insufficient. There should be a theoretically developed focus, for example to explain why students of colour were seen as deficient in their achievements. Instead, the structures in education and the 'assets communities of colour bring to school even when faced with oppression' should be recognised. Basic tenets should be adhered to: race, racism and intersections are endemic; dominant frameworks and ideologies that are white supremacist should be challenged; scholarship works toward social Justice and empowerment and 'elimination of racism and poverty' (180); experiential knowledge of people of colour is a legitimate way of understanding, such as through storytelling; CRT is inter- or transdisciplinary. We therefore get both the theory and methodology We can define theory as 'representation of knowledge based on the systematised framework of concepts' so CRT implies that the tenets above are principles to guide enquiry. But there is also methodology which centres the needs and experiences of people of colour as a tool to collect and analyse data. This methodology is theoretically grounded. It challenges traditional research paradigms and offers liberatory frameworks. It focuses on the experiences of students of colour. It takes interdisciplinary perspectives. However, it is 'not explicitly qualitative' although there is a preponderance of qualitative research, clearly rooted in the role of stories and counter stories, 'parables, chronicles, poetry, fiction and revisionist histories' [legacy of romanticiesed indigenous stuff?] which may also help shift away from positivist emphases. White centred or majoritarian research diminishes the voices and stories of people of colour so a counterstory deliberately challenges these master narratives. Social science research on the whole is not seen as fully equipped to reflect oppressed communities, including indigenous and colonised populations. Hence the perceived incompatibility between CRT and quants, and the tendency to favour qualitative methods. One dubious assumption about quants is that it is bias free, for example which seems to contradict the need to take a definitive stance. Quants tend to use positivistic paradigms, and qualitative methods might be better suited for critical paradigms and alternative epistemologies — but this is 'simplistic and limiting because it equates methods with paradigms which can and should be distinct… Critiquing the limits of post-positivism does not negate the potential use of QMs' (181). Counter stories and quantitative methods tend to focus on individuals, while quants emphasise groups and summary statistics in an attempt to test models and hypotheses and this might oversimplify the relations between variables, and underestimate the complexity of race relations. But equating counterstory and qualitative enquiry is also too limited — the response to majoritarian narratives is 'an equally important function of the counterstory' and response can be 'accomplished through quantitative studies' (182) A critical intent might be crucial in the choice of methods. Quants and statistics have an unfortunate historical origin in the methods used to justify the biological inferiority of black people and in eugenics, but 'this history does not beget contemporary methods' and 'qualitative methods are not immune to critique grounded in perspectives of racism and colonialism such as the use of ethnography in stereotyping indigenous and native communities' (182). Causal inference might be a problem, in attributing causes to socially constructed notions like race, and the process of isolating variables apparently free from selection bias. For CRT, race is clearly a nonrandom variable and racism is endemic. However 'there may be some value to studies that unearth the context of how policies and practices differentially affect groups'. Strong teaching and graduate school preparation should include high quality use of quantitative methods across educational fields including in CRT research, for example in graduate school mentorship. They might inform greater theorising on race, when encountering explanations other than race or racism for differences in educational attainment and experiences, for example. Improving understanding of both quants and CRT would be an appropriate response rather than dismissing one approach. Issues regarding objectivity, historical origins and statistical interpretations do limit the use of quants in critical race research, but many researchers are also calling for expanded mixed and quantitative methodologies in CRT, and concern regarding the lack of attention to race and racism in education policy literature. Empirical work is an important driver of CRT development and so 'ensuring the full range of methodological options is key' (183).. Some researchers have used quants and this use should be improved [Several useful-looking references are given for researchers who have used quants, including the very appealing  Milkman, K. L., Akinola, M., & Chugh, D. (2015). What happens before: A field experiment exploring how pay and representation differentially shape bias on the pathway into organizations. Journal of Applied Psychology, 100, 1678–1712] A 'quantitative criticalist' uses quants to reveal inequities, especially systemic ones, and to question models and analytic practices, a line that 'originated in the German Frankfurt school' (183). CRT draws more from civil rights movement and makes more explicit connections to race and its intersections, although it also features realism as a central part and sees social science research as the foundation for making arguments about race-based discrimination. It has used quants to complement qualitative counter storytelling [simple counts mostly — soft quantification?] Combining the two, we can identify a critical approach to quants is one that emphasises the assets of students of colour and looks at the overarching structure of racism and racial inequity. The specific method has been self identified as;'QuantCrit'. It has explored micro-aggressions, racial battle fatigue, and has analysed achievement and demographic data, for example by disaggregating Asian and native Hawaiian Pacific island populations by ethnicity, social class and immigration status. There has been argument for '"quantitative intersectionality"' for example looking at how Chicana/o students progress through the educational pipeline and disaggregating the data by gender, class and citizenship status. [Sounds a bit like Sewell -- another political impulse for UK CRT enthusiasts to reject quants of course]. There is a full acceptance that numbers do not speak for themselves, that quantitative analysis is grounded in experiential knowledge and standpoints, and that research is designed to advance social justice, and that transdisciplinary approaches are necessary. The latter points to parallel studies such as stratification economics, and social psychological concepts of racial bias. Stratification economics broadens human capital deficiencies by looking at differences in structural resources; psychological literature has also influenced work on microaggression and on racial battle fatigue, structural equation modelling helps researchers understand how exposure to racial microaggression  contributes to stress in students of colour. Quants in CRT studies in education still leaves much to be desired. There is no definite incompatibility, but there is still a need for development, and more methodological guidance to move into actual research practice. For example 'not all areas of quantitative methodology are fully used in CRT scholarship' (185). We have lots of descriptive and demographic statistics, but we should adopt more culturally responsive methods. Descriptive statistics need to push on to establish underlying causes or motivations. This should produce more experimental designs, like those that have documented racial discrimination in professors' perceptions of prospective graduate students. There is concern about causal modelling, but we might still proceed with 'predictive, experimental/quasiexperimental and evaluative modelling'. Generally we might move beyond descriptive statistics. Measurement theory might be explored more and connected more with CRT tenets and theories, pursuing perhaps the work on psychological models of racism and stress. So predictive regression models, causal inference via quasiexperimental studies and 'exploratory and confirmatory theory building (e.g. exploratory factor analysis… Structural equation modelling)' might be fully developed. There may be doubts about the measurability of some of these complex subjects, and psychometrics again has dubious origins, but 'when taken with an appropriate lens' measurement theory, even survey methodology can adequately contribute. For example 'counterstories can be incorporated into scale development and validation techniques can refine asset based theories' (186) we might use self-report surveys [and develop the idea of community cultural wealth — CCW] CCW has become a specific approach within CRT, and we might well use quants and measurements here. It is well grounded in CRT frameworks, and these are well cited 'yet [offer?] empirically underdeveloped form of quantitative methods' (187). It refers to the assets students of colour bring to schooling: 'aspirational capital: the ability to maintain hopes and dreams for the future; familial capital: connections to and knowledge of family and kinship networks; navigational capital: the ability to navigate through schooling institutions that were not designed with communities of colour in mind; resistant capital: the knowledge of and motivation to transform oppressive structures' [Thank God someone has picked up on these capacities to resist!] This counters 'dominant notions of cultural capital as in Bourdieu!. That tends to be deficit minded! There is a concept of 'non-dominant cultural capital, the assets that students bring from their home communities that can guide academic achievement. The issue also points to broader issues 'such as racial capitalism', where schools actually are able to engage students CCW, even exploit it. [Again another reference looks interesting  Carter, P. L. (2005). Keepin’ it real: School success beyond Black and White. New York, NY: Oxford University Press] We have to be careful when quantifying cultural capital and we can develop this when quantifying CCW to avoid reductionism. The intention is to focus on aspirational, familial, navigational and resistant capital. These forms have an exchange value but how schools value them and how students use them is 'up for future empirical study' (188) They can be operationalised in four separate scales, following  Yosso, T. J. (2005). Whose culture has capital? A critical race theory discussion of community cultural wealth. Race Ethnicity and Education, 8, 69–91. doi:10.1080/1361332052000341006. Sablan did an online survey of undergrads into open access institutions in the US Pacific recruiting 'Asian American Native American Pacific Islanders' -- AANAPISIs. The majority were Pacific Island or Asian-American, followed by Filipino, Chamorro, Micronesian and then other Asian or Pacific islander. 1.4% were white and 19% multi-ethnic. The majority (75%) were female. Over 60% of students were first entrance to college, the majority were from low family incomes, apparently typical of most students of this kind, typically neglected in educational research. A series of scales of non-dominant cultural capital were developed and used in research. The scales were developed on aspirational capital, familial, navigational and resistant. Each scale had 7 to 8 items which asked students to agree on a six-point scale with how a statement described them. There was content validity testing, expert review, pilot testing and cognitive interviewing. A review of the literature was pursued to address content validity. Expert reviewers also reviewed the scales and then cultural community leaders. A pilot survey was completed by a small group of similar students at a different institution, and the results were used to test the dependability of the design as well as student fatigue — all agreed that the survey questions were understandable and of an appropriate length, and other feedback was incorporated. Cognitive interviews involve responders from a similar population being asked questions related to their interpretation of the survey questions, a kind of '"think aloud"' (190) where respondents take the survey and then respond to questions about the questions, explaining what they think it means and how they got to the answer, ways in which the questions might be improved, interpretations of questions that might contradict the content intended by the researcher, to see how the items reflected the theoretical components that were supposed to be operationalised. Unclear questions were reworded or deleted. {Examples pp 189] Classical test theory assumes that the observed score is composed of the true score plus error, latent construct refers to traits that are by their nature unobservable and cannot be measured. We can use these to see how responses to a set of items generates a score that approximates to what is being sought — non-dominant cultural capital. We can also use factor validity, using factor analysis, to see how underlying factors predict the variance in the scale items [actual equation used on (191)  -- the basis of it seems to be isolating the common factors that underlie the item, then the factors unique to each item, then the loading of each item on the factor and the random measurement error of each item — we can then examine factor loadings as a measure of how related the items are to each other, and pick out those above a particular value. This business of rejecting items that did not reach an acceptable value applied to all the tests discussed here]. There are still limitations — she/they only validated the scale with one study and it would have been better to try on several, especially diverse populations, nevertheless, she/they is satisfied that the sampling procedure is effective enough, even though more work is needed. Operationalising CCW in a quantitative approach is definitely 'a worthy endeavour' overall and it can substantially improve on previous studies. The scale showed 'preliminary reliability and validity evidence' [coefficients are shown lying between zero and one and all of them scored above 0.7 which is the normal threshold for reliability measures]. Further statistical analysis was used to extract factors that best represented the variance of the item and to examine the potentially correlated nature of the multiple factors. Again everything fell within recommended thresholds, meaning that the process 'could result in internally consistent scales' although 'further validation work' would be ideal for future studies. Navigational capital and familial capital seem to be satisfactory items in measuring underlying factors. Statistically they both seemed to load onto one common factor which explained around 50% of the variance with acceptable structure coefficients. [Each separate scale seemed to be pretty high-scoring]. Aspirational capital and resistant capital might need more modification to 'best capture a measurement of these [underlying] factors' (194), and some items did not sufficiently load onto a factor, and were deleted from the scale — aspirations from or related to the family, the aspiration to surpass parents' educational and occupational success. The main factors for resistant capital seem to be identification of oppression in society and motivation to transform oppressive structures, which happens to be consistent with how it is described in the CRT literature. Overall, measurement theory can help analyse the CRT discussion of CCW. It is possible to score satisfactory factors and then use them in multivariate models. This shows that an empirical model of CRT and quants can be useful, and additional multivariate analyses similarly might be conducted in terms of college access and CCW. It certainly seems that aspirational capital, familial capital, navigational capital and resistant capital 'have preliminary reliability and validity evidence', but there may be need to improve the empirical fit especially with resistant capital and its alignment with CRT, but at least we now have guidelines about how to design and implement projects. So qualitative methods are not the only way to present counter narratives. When it comes to asset based critical race theories, they do seem to be combinable with quants especially measurement theory. There are implications for policy and practice, and there are implications also for the advancement of social justice. Conventional quantitative studies might be less useful if cultural assets are not operationalised. Institutionalised racism is often absent from theoretical frameworks, but critical quants should go on to engage the full potential instead of just relying on trying to establish non-neutrality. There are possible ways for quants and CRT to be congruent and compatible. There is much potential. We can move beyond a lot of CRT quantitative studies that are descriptive in nature. We should be able to develop reliability and validity techniques of measurement theory to produce more 'theoretically and methodologically driven studies' (197) we can then go on to discuss more robust quantitative techniques. The case study here turning on CCW illustrates the tensions of this dialogue between quants and CRT. The work is theoretically robust and experientially rich, but there are still tensions. Do the quantitative analyses oversimplify the complexity by making it a matter of statistical variables? We still need to discuss how to measure and analyse CCW, and how it might be an appropriate framework to understand college access and readiness. We might need to think about further measures of it. This should include quantitative analysis and different scales in different populations and intersection identities. What is it that makes quants critical, and how do we put quantitative analysis into critical practice. Actually running a regression model or whatever looks similar whether critical or non-critical studies are being pursued, and usually conventional practices 'have long been interpreted in ways that run counter to CRT tenets' (198). Perhaps the intent matters more than the methods?, CRT notes that cultural assets are not valued by the dominant school culture rather than just describing the picture [or using the findings to support a deficit theory?]. We need more methodological guidelines, which implies conventional teaching, although there may be methodological practices beyond what is conventionally taught that will be useful to issues of race and racism. Unfortunately, limits have been introduced by the effort to criticise 'the nonneutrality or even racialised dangers of numbers'. Instead, we need much more 'methodological doing and teaching while also highlighting the important practice implications' especially from 'studies that are critically informed and intended'. [same for reserch methods generally I reckon] There are implications for institutions designed with students' cultures in mind. There is a criticism of deficit interpretations which often inform policy, and caution about existing psychometric tests. Well-designed quantitative work might challenge these policies. We 'should move beyond simple descriptive statistics of racial difference to inferential measurement and theoretical modelling'(198), if only to provide the context for counter stories. We will be able to test theories and assumptions for example those that emphasise students cultural assets.