Tuesday, November 30, 2010

Testimony From Benetech’s Daniel Guzmán Helps Establish Legal Precedent and Convictions for Forced Disappearance in Guatemala

I announced in this blog last month that judges in Guatemala had found two former police officers guilty in the 1984 forced disappearance of Guatemalan student and union leader Edgar Fernando García. Expert testimony by Benetech statistical consultant Daniel Guzmán provided critical evidence in the conviction of the former Guatemalan National Police officers Abraham Lancerio Gómez and Héctor Roderico Ramírez.

Gómez and Ramírez were each sentenced to the maximum term of 40 years in prison for their role in García’s disappearance. This historical ruling has established forced disappearance as a crime in Guatemala and provided government prosecutors with a key legal precedent needed to investigate higher ranking officers for their possible role in the case. You can read more about the verdict here.

The entire staff here at Benetech is extremely proud of Daniel Guzmán and his colleagues at the Benetech Human Rights Program who have spent four years analyzing random samples of the estimated 31.7 million documents in the Guatemalan National Police Archive. Guzmán’s testimony in the García case was based on his analysis of this archive which was discovered in a military munitions storehouse near Guatemala City in 2005. The archive includes documents generated during Guatemala’s internal armed conflict which took place from 1960 to 1996. An estimated 40,000 Guatemalans disappeared during this period of violence. While the Guatemalan National Police were disbanded after the country's 1996 Peace Accords, very few people have ever been held accountable for crimes that took place during the 36 years of violence.

Guzmán and the other statisticians, programmers, demographers and data analysts of Benetech’s Human Rights Data Analysis Group (HRDAG) believe that scientific arguments can help clarify the past and end impunity. But in order to prosecute perpetrators of human rights violations, the courts rightly demand evidence. HRDAG analyzes the patterns and magnitude of human rights violations to determine how many of the killed and disappeared have never been accounted for - and who is most responsible.

Guzmán conducted a comparative analysis between 667 documents pertaining to García that were found in the Archive and estimates from the entire Archive. This analysis, which was submitted as evidence in the trial, showed that units responsible for the direction and coordination of the National Police were acquainted with more than 73% of the documents related to the García case as opposed to 30% of all documents in the entire Archive. Guzmán’s findings helped support arguments by prosecutors that high-level National Police officers were aware of orders given for the planning and design of operations like the one that resulted in García’s disappearance. Guzmán’s testimony also helped confirm the credibility of our Guatemalan partners who preserve and examine the documents in the archive, a group known as the National Police Archives in Guatemala (Archivo Histórico de la Policia Nacional) or AHPN.

The García case is the first in Guatemala based primarily on archive documents and paves the way for judges to trust these records - and statistical findings - as evidence in future trials. Prosecutors announced that Benetech’s analysis of archive contents will be a key part of future investigations. The scientific methods used by HRDAG to analyze the archive has set standards of scientific rigor that helps overcome political arguments about these records. HRDAG’s analysis establishes scientific paradigms for examining large collections of human rights data in other parts of the world.

Calculating scientifically sound statistics and quantitative findings to support human rights claims offers a powerful mechanism to help halt the cycle of violence and create lasting social change. Unless the human rights community is ready with unimpeachable information about past abuses, it cannot make the most of opportunities for official acknowledgment, accountability or reform. Twenty-six years after his disappearance, the family of Edgar Fernando García knows that this opportunity for justice has not been lost.

Tuesday, November 09, 2010

Signals - Stoplights for student success

At the STS meeting in Kyoto, I had the pleasure of meeting France Córdova, the president of Purdue University. I took my first full pattern recognition course at Purdue long ago and far away (my brother Bill is a Boilermaker/Purdue alum, too).

France mentioned some cool education technology that had been developed at Purdue, called Signals - Stoplights for student success. Signals blends two key ideas:
  1. The patterns of student failure can be spotted early: much earlier than existing systems relying on failing midterms! Purdue can spot patterns that indicate a much higher chance of failure, and intervene early.
  2. Simple communications design that everybody gets: green light, yellow light, red light. And, get these signals to both students and faculty.
I was able to talk to one of the senior team at Purdue about this at the recent launch of Purdue's Silicon Valley presence. He was quite realistic about the limitations of the system. For example, a really bright student who can pull everything together just before the exams might present with a red light a third of the way into the quarter, but still be ok. But, the weight of evidence is that they can associate learning management system data with an increased likelihood of failure, and then intervene.

This really made me think about students with disabilities. Disabled Student Services offices can't supervise the students they serve: there's not enough resources for that, and college students don't need babysitting. But, a system that cost-effectively identifies any students at risk of failure is a great tool for all students, and might also serve as a flag for some of the disability services interventions that might be needed. For example, an accessible version of a key book or item of course content.

Purdue has made a deal with Sungard to support Signals more widely at other universities: I hope this kind of tool makes a much wider impact!

Sunday, November 07, 2010

The Tech Awards - Technology Benefiting Humanity

A Guest Beneblog by Teresa Throckmorton, Benetech's CFO

Walking into the Santa Clara Convention Center last night looked very much like so many other black tie events – but that’s where the similarity ended.

I just attended the 2010 Tech Awards Gala Event. Awards were presented in five areas: Environment, Economic Development, Education, Equality and Health. Inside there were 20 stations set up with this year’s laureates. Talking with each of these passionate and amazing individuals was fascinating. It was hard to stop talking to one – so I could move on to the next. The impacts these people and their organizations make are being felt world-wide. Using technology as a base for knowing change is possible, change is happening.

What did I learn last night? I learned that in some parts of India everyone has a cell phone but few have indoor plumbing or access to clean water – change is happening. I learned it’s now possible to give immunizations and antibiotics with a needle-free injection system; to capture a blood smear on a small device, called a CellScope, send the image over a cell phone and get a diagnosis; to generate clean electricity for lights and gas for cooking from rice husks and that it is possible to change the world.

I entered thinking Benetech was a lone voice for social change using technology. I left proud to be a part of this growing family of voices dedicated to making change.

Thursday, November 04, 2010

Work on What Matters — Social Edge

I'm having a conversation over at Social Edge on What Matters. Feel free to join in the conversation and share your views on the topic. Here's my kick-off post for the thread:

I was impressed when I heard Tim O'Reilly, one of the main thought leaders in information technology, recommending to all tech folks last year that they Work on Stuff that Matters. Tim's point wasn't that all tech developers should go to work for nonprofits, it was that people should step back and think about what matters to them. Life is too short to throw your professional life away on stuff you don't care about.

Like many techies, I came to work on technology because I loved doing it. We get a charge out of figuring things out, and understanding how the world works in a deep ways. Almost all the geeks I know want to do something important, something meaningful, whether exploring something new in cosmology, designing a building that could better resist an earthquake, cure a disease or design a new and faster chip.

I see this hunger for meaning in most of the people I'm privileged to meet: from the college freshman to the fresh grad to the mid-career professional and the senior executive. We all want to work on Stuff That Matters. And, the opportunities to do so have never been better.

Business as usual, done without regard to the big picture for society, hasn't worked out so well. There is a new wave of leaders who want business to both make money and do right by society and the planet. And, the same spirit of innovation for a purpose is affecting nonprofits and government as well.

If you're looking for a new job, look for a company or organization that offers the chance to work on Stuff That Matters. We need you to get engaged now, in business, government and the social sector, to build the better world we all need, and that our children and grandchildren will need.
  • So, how did you get to work on Stuff That Matters?
  • How do you make your team feel that their work Matters?
  • How are we going to shift expectations so that more and more organizations see their purpose as meaning through supporting larger society, not just their shareholders or their narrow product or projects?

Join Jim Fruchterman, CEO of Benetech, in a conversation about how we can make working on Stuff That Matters the rule instead of the exception!

Monday, November 01, 2010

Wikileaks War Data Reveal Underreporting of Iraqi Civilian Casualties

A Guest Beneblog by Anita Gohdes, Jeff Klingner, Megan Price and Patrick Ball

The recent release of almost 400,000 secret US military files on the war in Iraq through Wikileaks has attracted wide media coverage.

These documents, officially known as the significant acts database (SIGACTS), add new insights to the ongoing debate on how many casualties have occurred in Iraq since the beginning of the war. The unofficial Iraq Body Count (IBC), which tracks civilian casualties in Iraq based on press reports and administrative records, has initiated a comparison of their own data to the deaths documented in the SIGACTS data. In a commendable effort, they are recoding the SIGACTS data to correct coding errors and in order to match it with their own database. They have estimated that the SIGACTS describes 15,000 civilian deaths previously undocumented by IBC (BBC’s report is here). Most of these previously-unknown deaths occurred in small incidents, in which 1-3 people were killed. In academic articles, on blogs and at conferences about quantifying war casualties, there has been debate about whether deaths that occur in small incidents tend to be systematically underreported in IBC’s press sources. This new evidence is consistent with the hypothesis that IBC’s press data underreport small events in Iraq.

In reaction to the release of SIGACTS data, Les Roberts of Columbia University wrote that both IBC and SIGACTS are ‘systematically prone to under-report deaths’ and that his previously published estimations of total death tolls are likely to be well below the actual numbers he and his colleagues previously estimated. Roberts has cautioned the use of press sources for the purpose of counting casualties, as they fail to report a significant proportion of violent events. Jacob Shapiro of Princeton University, presents a parallel argument. He reminds readers that SIGACTS includes every death that was recorded by the Multi-National forces in Iraq, not every death that occurred during the ongoing war. SIGACTS’s reporting standards changed over time and the reporting procedure varied across units. In particular, IBC has noted that the SIGACTS data do not include any civilian casualties from 2004 operations in Fallujah.

This underreporting is not surprising. Both SIGACTS and the press and other data published by the IBC are convenience samples, i.e. they are not generated with the help of a random selection process. Both SIGACTS and IBC are well-run, careful projects, but even very good direct observations that collect information on violence systematically will tend to accumulate data that is unrepresentative of the actual conflict patterns they are attempting to uncover. Lists of data that fall into this category are thus unsuitable for drawing inferences on any population apart from the list itself. Simple lists of deaths are inadequate to characterize an entire country in the midst of a highly politicized war.

The hypothesized reasons for under-reporting in Iraq mentioned by Roberts and Shapiro are not unique to Iraq or the IBC and SIGACTS databases: changing reporting patterns across time and space due to organizational changes (as was the case with SIGACTS), better coverage of events in urban areas (as was the case for both IBC and SIGACTS in Baghdad), and varying levels of victim visibility, depending on victim and perpetrator characteristics, are factors that influence almost every database that collects information on violent events. In our experience, reporting and recording bias varies dramatically and can rarely be distinguished from the actual patterns of violence. In the case of Iraq it is therefore important to keep in mind that a) no list is (or will be) complete and b) new, independent sources of data are needed to understand the reporting biases of any single source.

A possible solution to overcome the bias of single lists is to use a statistical method known as multiple systems estimation (MSE), which can provide estimates for those cases that weren’t recorded in any list. MSE, also known as the capture-tag-recapture method, corresponds to the idea that each death has the possibility of being recorded by one, two, or more data sources. Depending on the degree of overlap of cases between the sources, the number of deaths that were not reported to any source will differ. This method has been used for estimating large-scale killings in Guatemala, Kosovo, Perú, Srebrenica, East Timor, and Colombia, among others.

For the two lists available for Iraq (SIGACTS and IBC) a simple two-system MSE model (the Lincoln-Peterson estimator) can be applied. For the period covered by SIGACTS (2004-2009), IBC reports 15,000 civilian deaths found only in the SIGACTS data, 27,000 found only recorded by IBC’s press sources, and 64, 000 recorded in both, for a total of 106,000. The two-system estimation gives us a slightly higher number of: (15k+64k)*(27k+64k)/(64k) = 112,000 civilian casualties, including an estimated 6,000 deaths recorded by neither source. This estimate assumes that the collecting patterns of IBC and SIGACTS were independent of each other, which is unlikely.

Roberts suspects that both sources cover similar cases, mostly coming from the Iraqi Government, focused on Baghdad and that both under-report small events and incidences of single killings. In statistical terms, the two sources are likely to be positively correlated, which biases the estimate downward. The true number of deaths is thus likely to be larger than 112,000 cases. In order to obtain more accurate estimates of the number of deaths, a third source would be required. Three or more sources would allow us to account for similar (or dissimilar) reporting patterns of the different sources.

The revelation of the Wikileaks war log data has altered our understanding of civilian casualties in Iraq: We now know that there were more small-event casualties than previously thought and that such casualties are underreported. Further information about the ‘new deaths’ revealed in the war logs could improve our understanding with regard to victim characteristics and patterns of violence across time and space. We welcome the important work IBC is doing to correct errors in the SIGACTS coding and to match their database with the SIGACTS data. Perhaps a third source will emerge, and from the three datasets, estimates could be made which would correct for systematic underreporting across types of events, regions, religious sect or period. Hopefully, a better understanding of how violence is reported in Iraq will help us to better correct for reporting bias in casualty figures in other conflict situations.