punkish: Measuring Information Accessibility

name: intro
class: center, middle

# Measuring Information Accessibility
## A proposal

•

Puneet Kishor (Plazi)

Released under a [CC0 Public Domain Dedication](https://creativecommons.org/publicdomain/zero/1.0/).

<div class="slidefooter">
    <a href="/Measuring-Information-Accessibility"><img src="/_lib/img/PunkishEidesisOrg-small.gif"></a> 
    <a rel="license" href="https://creativecommons.org/publicdomain/zero/1.0/"><img src="/_lib/img/zero.png" alt="CC0" width="30" height="30"></a>
    <div class="navlinks">
        <a href="javascript:slideshow.gotoPreviousSlide()">prev</a> 
        <a href="javascript:slideshow.gotoNextSlide()">next</a>
    </div>
</div>
---

layout: true

## Help

* Notes are hidden, but may be seen by pressing **P** on your keyboard.
* Press **C** to clone a show.
* Press **H** for other keyboard shortcuts.

???

notes here

---

## Why

We all talk about accessibility and freedom, but what does it actually mean?

???

Can we recognize free when we see it? Can we measure accessibility? Can we rate one source of information as being more or less accessible than another?

There is no consensus on information, let alone on information accessibility, yet we talk about it, profess and promote it, lament its absence and fear its demise, and base policies on it. This is a proposal to define and measure information accessibility.

---

## What we know

*   Information production, sharing and use are increasing very rapidly \[1]
*   Modes of information production, sharing and use are changing \[2]
*   Technological, legal and other mechanisms to restrict information sharing are also increasing \[3]
*   Restrictions are exacerbated when information crosses national and jurisdictional boundaries \[4]

<hr>

*   \[1] Gantz 2008; Lyman and Varian 2003
*   \[2] Benkler 2007; Weber 2005
*   \[3] Mccurry et al. 2006; NRC 2003
*   \[4] Special 301 Report  2007

???

Technology is making both creation, sharing and use as well restriction possible. Technology is also changing the cultural expectations of what is ok and what is not, and changing the economics of the entire information chain.

---

## What we believe

*   Restriction to information can stifle both of the above \[1]
*   Access to information is necessary for both the scientific process as well as for entrepreneurship

<hr>

*   \[1] Lessig 2001

---

## A reasonable assumption

Information !will be produced and shared! with increasing ease, while access to and use of information !can and will be restricted! with increased vigilance.

In this scenario, being able to quantify or otherwise indicate the extent of accessibility of information would be a reasonably useful capability whether one is advocating freedom or restriction of information.

---

## What I advocate

*   Increase information accessibility
*   Specify different ways in which accessibility can be increased

<hr>

*   The above is a reasonable expectation, and has policy implications.

---

## What I don't know

*   How can I scientifically measure information accessibility?
*   How do I determine if the attempts to increase information accessibiility are successful?

<hr>

*   The above is the motivation for this proposal.

---

## Not just my problem

*   Measuring and specifying levels of information accessibility seems to be very difficult \[1]
*   Not only do we not have an established method to measure information accessibility, we don't even have an established definition of information accessibility \[1]

<hr>

*   \[1] Peterson Bishop 1998

---

## Potential benefits of being able to measure accessibility

*   Evaluate compliance with open access regulations
*   Enable flow to the public of benefits from information created with public funds
*   Invigorate entrepreneurship
*   Make the scientific process more efficient via reduced duplication, fewer resources spent on collecting information, etc.
*   Identify conditions that make for more accessible information

<hr>

*   The above are assertions, and have policy implications.

---

## The Bigger Picture

*   Information fuels science, science fuels innovation, innovation is the life-blood of entrepreneurism
*   Open access to information can lead to innovative results \[1-3]
    *   "Publicly available data was significantly ... associated with a 69% increase in citations, independently of journal impact factor, date of publication, and author country of origin." \[4]

<hr>

*   \[1] Lakhani, *et. al.* 2007
*   \[2] "Density Hales-Jewett" Theorem described in Rehmeyer 2009
*   \[3] a seven years old initiative on sharing of data to find the biological markers of Alzheimer's disease described in Kolata 2010
*   \[4] Piwowar 2010

---

## The Bigger Picture... is cloudy

*   But open access to data is threatened \[1]

<hr>

*   \[1] Reichman and Uhlir 2003

---

## Things are not what they seem

"Mars Inc. Cacao Genome Database claims Open Access, public domain: falls short" \[1]

*   Claims to be public domain
*   Funded by USDA, and collaborators from IBM, Clemson University; University of California-Davis; National Center for Genome Resources; Indiana University; HudsonAlpha Institute for Biotechnology; and Washington State University
*   Yet, has license restrictions of all kinds
    *   no mention of commercial use
    *   "anytime the User consults the data base through the cocoa genome database web site, he/she shall be bound to the same obligations under this IAA. Should the User store the information and data for future use he/she shall be bound to the same obligations under this IAA"
    *   "The User shall not transfer the information referred to in this agreement, or any copy of them, to a third party without obtaining written authorization from the Providers which will only be provided subject to the third party user entering into this same IAA"
    *   "The User shall not claim legal ownership over the information and data found in the data base nor seek intellectual property protection under any form over these information, data and data base. For clarity, the user agrees not to claim any of the sequences disclosed in these databases in any patent application"

<hr>

*   \[1] http://zzzoot.blogspot.com/2010/09/mars-inc-cacao-genome-database-claims.html

---

## Steps

*   Define information in general
*   Define information in science
*   Define information accessibility
*   Define indicators of it
*   Identify a problem domain and a study set
*   Measure the !intensity! of indicators for the study set
*   Rank elements of study set on the information accessibility scale
*   Validate results

---

## Define information

*   I will define !datum! as a descriptor of a tangible thing or an intangible idea
*   When that datum is used to assert a state about what it describes, that is !information!
*   In keeping with DoD's assertion, meaning derived from data is information, but that information can also be a part of data at another level \[1]
*   This is an important characteristic of information: information is the raw material for more information.

<hr>

\[1] ATIS Telecom Glossary 2007

---

## Information in science

*   Data
    *   Raw data
    *   Interpreted data
*   Software
    *   Source code
    *   Binary
*   Literature
    *   Grey literature
        *   slideshows, videos, audio files, photographs, working papers
    *   Formal literature
        *   reviewed papers, book chapters, monographs, conference proceedings

---

## Information restriction in science

*   Different reasons for restricting \[1]
*   Scientists motivated by different reasons in different fields and at different stages in their career \[2]

<hr>

*   \[1] 1999 meeting on !Secrecy in Science! organized by the AAAS Committee on Scientific Freedom and Responsibility
*   \[2] AAAS 2007

---

## Information accessibility

Information is the very basis of science, it is its raw material, and its final product. Science informs us, it affects our existing assertions and enables us to make new assertions.

It stands to reason that the more easily we can get to more information, the more likely we will be able to expeditiously proceed in our scientific quest.

---

## But, what do we mean by information accessibility

*   Is it not available at all? 
*   Is it difficult to understand? 
*   Is it not documented? 
*   Does it have to be paid for, and is its price too high? 
*   Is it encrypted? 
*   Is its reuse constrained by onerous legal clauses and expectations of reciprocal obligations?

---

## Five factors affecting accessibility

*   Culture
    *   is it acceptable?
*   Law
    *   is it legal?
*   Technology
    *   is it possible?
*   Semantics
    *   is it understandable?
*   Economics
    *   is it affordable?

<hr>

*   The above are loosely based on the Creative Commons licenses that are offered in three formats: human readable, legal text, and machine parsable. The addition of culture and economics rounds out the indicators from a social perspective.

---

## Indicators

![indicators of accessibility](indicators w=683 h=233)

---

## The case for culture

*   Culture makes it !acceptable!
    *   academics cite each others' works without being required legally \[1]
    *   kids download music and videos, genuinely surprised that it is possibly illegal to do so
    *   Programmers create and share programs based on internal motivations such as fom  altruism, community identification, ego-gratification and learning

<hr>

*   \[1] Science Commons, a project of Creative Commons, the non-profit creator of the immensely popular Creative Commons licenses, based its recently released CC0 protocol for scientific data on the "use of scientific norms to express the wishes of the data provider"
*   \[2] Hars and Ou 2001; Lerner and Tirole 2002; Hertel, Niedner, and Herrmann 2003

---

## The case for law

*   Law makes it !legal! via instruments such as
    *   Patents protect a specific expression of an idea for a device or a technique
    *   Trade secrets protect proprietary and strategic information
    *   Trademarks and service marks display proprietorship in the mark and its associated use
    *   Copyright licenses are unilateral restrictions on the use of the content
    *   Contracts are agreements between two parties stating the obligations of both

---

## The case for technology

*   Technology makes it !possible!
    *   Implement implement distribution or protection mechanisms \[1]
    *   Technology also makes barriers possible
        *   information format not amenable to digital dissemination
        *   not having it on a publicly reachable server
        *   not available at convenient times
        *   requiring a convoluted and onerous authentication procedure as a prerequisite to access
        *   opaque data formats
        *   lack of documentation or software tools to unpack and digest information
        *   an onerous, non-standard process for data extraction

<hr>

*   \[1] Awre and Jacobs 2006

---

## The case for semantics

*   Semantics makes it !understandable!
    *   not just !"poe-tah-toe" vs. "poe-tay-toe"!
    *   making information amenable to manipulation with the help of software programs, a vision of the semantic web \[1]
    *   adding useful metadata to it
    *   for eg., Wikipedia vs. DBPedia \[2]

<hr>

*   \[1] Halb, Raimond, and Hausenblas 2008
*   \[2] Auer et al. 2007

---

## The case for economics

*   Economics makes it !affordable!
    *   monetary restriction imposed on information by simply charging for it
    *   while recouping the cost of collecting, creating, managing and making information available and even profiting from doing so is understandable, doing so for publicly funded information is double-dipping
    *   the average price of scientific journals in social and biomedical sciences increased by 39% between 2000 and 2006, more than twice as much as the increase in the retail price index in that period \[1]
    *   coupled with budget cuts at libraries, a grim outlook was predicted for the ability of libraries to provide timely information to its users \[2]

<hr>

*   \[1] White and Creaser 2007
*   \[2] Van Orsdel and Born 2009

---

## Technology

<table class='stripe'>
<tr><th></th><th colspan='2'>data</th><th colspan='2'>software</th><th colspan='2'>literature</th></tr>
<tr><td></td><td>raw</td><td>interpreted</td><td>binary</td><td>source code</td><td>informal</td><td>formal</td></tr>
<tr><td>Is it digital?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Is it discoverable?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Is it in plain text format?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>If it is not in plain text format, is the software required to read it free and open source?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Is it in a format that is free and open source?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Is the information format published?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Can it be accessed without logging into an account first?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>If logging into an account is required, can an account be created without first requiring approval?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
</table>

---

## Law

---

## Culture

---

## Economics

---

## Semantics

<table class='stripe'>
<tr><th></th><th colspan='2'>data</th><th colspan='2'>software</th><th colspan='2'>literature</th></tr>
<tr><td></td><td>raw</td><td>interpreted</td><td>binary</td><td>source code</td><td>informal</td><td>formal</td></tr>
<tr><td>Semantics</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Is the information well-structured allowing query via computer programs?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Is the structure of the information published?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Are their published APIs that facilitate information query?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td>Is there a publicly available endpoint for querying the information?</td><td></td><td></td><td></td><td></td><td></td><td></td></tr>
</table>

---

## Forest models as a focus group

*   A well-defined field and group of practioners
*   History, literature, culture
*   As science-y as science can be
*   Data, lots of data, much of it by necessity from outside sources
*   Small dataset (between 50 and 100)

---

## Studying Forest Models

*   Very highly parameterized
*   Requiring many data values that may be difficult to obtain
*   Many of the underlying assumptions in most of the models are the same, yet researchers create new models, replicating much of the work done by others

> "scientists (should) share code for different forest models to avoid duplication, so their effort can focus on the parts of the model that really vary" \[1]

<hr>

*   \[1] Landsberg and Gower 1997

---

## Topical

Because of the increased interest in carbon calculations driven by the concerns over global climate change, ecosystem process models are also crossing over into the realm of entrepreneurism, as modelers are taking steps toward commercializing the processes or the service delivery of the processes.

---

## Forest Models

<table class="stripe">
<thead>
<tr><th>name</th><th>organization</th><th>author</th><th>country</th><th>year</th></tr>
</thead>
<tfoot>
<tr><th>name</th><th>organization</th><th>author</th><th>country</th><th>year</th></tr>
</tfoot>
<tbody>
<tr><td>PP1</td><td>INRA</td><td>Lemoine B.</td><td>France</td><td>1991</td></tr>
<tr><td>Laricio</td><td>CEMAGREF; IDF; INRA</td><td>Meredieu C.</td><td>France</td><td>1998</td></tr>
<tr><td>Fagacées-Hêtre</td><td>ENGREF; INRA</td><td>Jean-François Dhôte</td><td>France</td><td>1996</td></tr>
<tr><td>Fagacées-Chêne</td><td>ENGREF; INRA</td><td>Jean-François Dhôte</td><td>France</td><td>1998</td></tr>
<tr><td>ASPECTS</td><td>Université de Liège</td><td>Daniel Rasse & Louis...</td><td>Belgium</td><td>2001</td></tr>
<tr><td>Pukkala in CAPSIS</td><td>CEMAGREF</td><td>Goreaud d´apres Pukk...</td><td>France</td><td>2001</td></tr>
<tr><td>SIMOPCOSOR</td><td>CEMAGREF; ENGREF</td><td>Goreaud</td><td>France</td><td>2000</td></tr>
<tr><td>Eucalypt</td><td>CIRAD; ECOsa</td><td>L. Saint-André</td><td>France</td><td>2002</td></tr>
<tr><td>Mountain</td><td>CEMAGREF</td><td>B. Courbaud</td><td>France</td><td>2002</td></tr>
<tr><td>PNN</td><td>INRA</td><td>Ph. Dreyfus</td><td>France</td><td>1993</td></tr>
<tr><td>Ventoux</td><td>INRA</td><td>Ph. Dreyfus</td><td>France</td><td>2002</td></tr>
<tr><td>Selva</td><td>CIRAD</td><td>S. Gourlet-Fleury</td><td>France</td><td>1997</td></tr>
<tr><td>CA1</td><td>INRA</td><td>François Courbet</td><td>France</td><td>2003</td></tr>
<tr><td>PP3</td><td>INRA</td><td>C. Meredieu, Ph. Dre...</td><td>France</td><td>2002</td></tr>
<tr><td>Biomass PPM gal</td><td>Universidad de Santi...</td><td>Balboa et al.</td><td>Spain</td><td>2005</td></tr>
<tr><td>Biomass Rad Gal</td><td>Universidad de Santi...</td><td>Balboa et al.</td><td>Spain</td><td>2005</td></tr>
<tr><td>Eucalypt gal</td><td>Universidad de Santi...</td><td>Balboa et al.</td><td>Spain</td><td>2005</td></tr>
<tr><td>Allometric Qcus Gal</td><td>Universidad de Santi...</td><td>Balboa et al.</td><td>Spain</td><td>2005</td></tr>
<tr><td>Allometric Bet GAl</td><td>Universidad de Santi...</td><td>Balboa et al.</td><td>Spain</td><td>2005</td></tr>
<tr><td>med_shrubs</td><td>INRA</td><td>D. Armand, M. Etienn...</td><td>France</td><td>1993</td></tr>
<tr><td>RODAL</td><td>FORECOTECH</td><td>PALAHI; PUKKALA; TRS...</td><td>Spain</td><td>2005</td></tr>
<tr><td>PiroPinus</td><td>UTAD</td><td>Fernandes</td><td>Portugal</td><td>2001</td></tr>
<tr><td>Unevenaged stands</td><td>AFOCEL</td><td>AFOCEL</td><td>France</td><td>2012</td></tr>
<tr><td>MOSES</td><td>BOKU</td><td>Hubert Hasenauer</td><td>Austria</td><td>1998</td></tr>
<tr><td>BioS-BCG</td><td>BOKU</td><td>Hubert Hasenauer</td><td>Austria</td><td>1998</td></tr>
<tr><td>PrognAus</td><td>BOKU</td><td>Hubert Hasenauer</td><td>Austria</td><td>2012</td></tr>
<tr><td>PP3</td><td>INRA</td><td>INRA</td><td>France</td><td>2002</td></tr>
<tr><td>AFODOUG</td><td>AFOCEL</td><td>AFOCEL</td><td>France</td><td>1999</td></tr>
<tr><td>AFOEPI</td><td>AFOCEL</td><td>AFOCEL</td><td>France</td><td>1999</td></tr>
<tr><td>AFOPIN</td><td>AFOCEL</td><td>AFOCEL</td><td>France</td><td>1999</td></tr>
<tr><td>Evenaged stands</td><td>AFOCEL</td><td>AFOCEL</td><td>France</td><td>2012</td></tr>
<tr><td>3PG-EG-PT</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>2012</td></tr>
<tr><td>GLOBULUS 2.1</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>2001</td></tr>
<tr><td>GLOBTREE</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>2003</td></tr>
<tr><td>PBLEIRIA e PBRAVO</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>1987</td></tr>
<tr><td>PBIRROL</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>2003</td></tr>
<tr><td>MODISPINASTER</td><td>UTAD</td><td>Teresa Fonseca</td><td>Portugal</td><td>2004</td></tr>
<tr><td>PINASTER-tree</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>2002</td></tr>
<tr><td>Dryads</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>2002</td></tr>
<tr><td>CASTANEA</td><td>ISA</td><td>ISA</td><td>Portugal</td><td>2004</td></tr>
<tr><td>RODAL</td><td>FORECOTECH</td><td>Pukkala et al.</td><td>Spain</td><td>2003</td></tr>
<tr><td>SILVES</td><td>INIA</td><td>Río Gaztelurrutia, M...</td><td>Spain</td><td>1998</td></tr>
<tr><td>ERVITI</td><td>University Politechn...</td><td>Erviti, J. J.</td><td>Spain</td><td>1991</td></tr>
<tr><td>GESMO</td><td>Universidad de Santi...</td><td>Castedo, F. & Diégue...</td><td>Spain</td><td>2004</td></tr>
<tr><td>GOTILWA+</td><td>CREAF</td><td>Gracia, C. et al.</td><td>Spain</td><td>2002</td></tr>
<tr><td>MARIOLA</td><td>INIA</td><td>Sanchez-Gonzáles, M....</td><td>Spain</td><td>2005</td></tr>
<tr><td>CAÑADAS</td><td>INIA</td><td>Cañadas, N.</td><td>Spain</td><td>2000</td></tr>
<tr><td>ESPINEL</td><td>IKT</td><td>Espinel, S. et al.</td><td>Spain</td><td>1997</td></tr>
<tr><td>ALVAREZ</td><td>Universidad de Santi...</td><td>Alvarez Gonzalez, J....</td><td>Spain</td><td>1999</td></tr>
<tr><td>PALAHÍ</td><td>FORECOTECH</td><td>Palahí, M. et al.</td><td>Spain</td><td>2002</td></tr>
<tr><td>SILVA</td><td>Technische Universit...</td><td>Pretzsch, H. et al.</td><td>Germany</td><td>1992</td></tr>
<tr><td>BALANCE</td><td>Technische Universit...</td><td>Grote, R. et al.</td><td>Germany</td><td>1999</td></tr>
<tr><td>WALDPLANER</td><td>Forest Research Stat...</td><td>Nagel, J.</td><td>Germany</td><td>2006</td></tr>
<tr><td>BWIN</td><td>Forest Research Stat...</td><td>Nagel, J.</td><td>Germany</td><td>2005</td></tr>
<tr><td>PEP 2.0</td><td>Forstliche Versuchs-...</td><td>Yue, C. et al.</td><td>Germany</td><td>2006</td></tr>
<tr><td>W+</td><td>Forstliche Versuchs-...</td><td>Weise, U. et al.</td><td>Germany</td><td>2000</td></tr>
<tr><td>SIZE CLASS MODEL</td><td>Forstliche Versuchs-...</td><td>Kandler, G.</td><td>Germany</td><td>2012</td></tr>
<tr><td>WEHAM</td><td>Forstliche Versuchs-...</td><td>Bosch, B.</td><td>Germany</td><td>2002</td></tr>
<tr><td>Soderberg</td><td>SLU, Department of F...</td><td>Soderberg, U.</td><td>Sweden</td><td>1986</td></tr>
<tr><td>EKO</td><td>SLU, Southern Swedis...</td><td>Eko, P-M.</td><td>Sweden</td><td>1985</td></tr>
<tr><td>AGESTAM</td><td>SLU, Southern Swedis...</td><td>Agestam, E.</td><td>Sweden</td><td>2012</td></tr>
<tr><td>ProdMod</td><td>SLU, Southern Swedis...</td><td>Eko, P-M.</td><td>Sweden</td><td>1995</td></tr>
<tr><td>Hugin</td><td>SLU, Department of F...</td><td>Lundstrom, A. et al.</td><td>Sweden</td><td>2012</td></tr>
<tr><td>Forest management pl...</td><td>SLU, Department of F...</td><td>Jonsson, B. et al.</td><td>Sweden</td><td>1993</td></tr>
<tr><td>EFISCEN</td><td>Alterra & EFI</td><td>Nabuurs</td><td>Europe</td><td>2010</td></tr>
<tr><td>SIBYLA</td><td>Technical Univerzity...</td><td>Marek Fabrika et. al</td><td>Slovakia</td><td>2005</td></tr>
<tr><td>PinasterSiteIndex</td><td>University of Northe...</td><td>Oscar Garcia</td><td>Canada</td><td>2005</td></tr>
</tbody>
</table>

---

## Steps

*   Identify the study set
*   Create a questionnaire-based survey that measures the models per the factors
*   Administer the questionnaire
*   Conduct open-ended survey of the impression of openness of !other! models
*   Analyze results

---

## Analysis

*   Since I have already decided on the indicators of accessibility, the first problem is simple — measure them
*   However, the dependent variable (free or restricted) depends on the viewpoint of the viewer

---

## Strategy

*   Use Charles Ragin's Quantitative Comparative Analysis (QCA) \[1]
*   Suitable for small sets and fuzzy sets (fsQCA)
*   Assumes that not any one but many different conditions occurring concurrently affect the outcome — !causal conjunctures! in Ragin's words
*   Favorably reviewed for its suitability for studying small sample sets,  especially when the responses are not many and also not very varied \[2]
*   Precedent for using it for measuring improvisation, closely tied to innovation \[3]

<hr>

*   \[1] Ragin 1989; Ragin 2008
*   \[2] Achen 2005
*   \[3] Miner, Bassoff, and Moorman 2001

---

## Quantitative Comparative Analysis

*   Ask questions that have binary (yes|no) answers
*   If a question is not binary, reformulate it so it does become a binary question
*   Construct a truth-table
*   Discard causal conditions that contradict the outcome variable
*   Narrow down the contradictions to build a combinatorial model with a minimum of causal combinations with contradictory outcomes.

---

## Analyze Responses

<table class="stripe">
<tr><th colspan="3">Independent Variables</th></tr>
<tr><th colspan="2">Number of independent vars</th><td>6</td></tr>
<tr><th>number of false independent vars</th><th>possible combinations</th><th>dependent var</th></tr>
<tr><td>0</td><td>1</td><td>free</td></tr>
<tr><td>1</td><td>6</td><td>free</td></tr>
<tr><td>2</td><td>15</td><td>free</td></tr>
<tr><td>3</td><td>20</td><td>indeterminate</td></tr>
<tr><td>4</td><td>15</td><td>closed</td></tr>
<tr><td>5</td><td>6</td><td>closed</td></tr>
<tr><td>6</td><td>1</td><td>closed</td></tr>
<tr><th>Total</th><td>64</td><td></td></tr>
</table>

---

## Expected End Result

*   A set of conditions that can provide a reliable measure of the openness of infromation of a particular information source
*   A methodology to measure information accessibility of information sources allowing comparison between them

---

## Expected Difficulties

*   Nearly impossible to create a control group to study conditions of both restrictive as well as easily accessible information
*   All indicators are ordinal, which is not terribly useful
*   The indicators are not uniform in their measure of openness
*   There is no reliable and accurate ground-truthing method other than the opinions of those involved with the models
*   Scientists are likely to respond differently when talking about their own data, that is, as data publishers, than when talking about data from others, that is, as data consumers
*  The independent variables are measurable, but the dependent variables are, for the most part, opinion
*   Freeness can't be measured unless defined, and the way it is defined decides how and what can be measured

---

## References

*   AAAS. 2007. International Intellectual Property Experiences : A Report of Four Countries. Washington DC: Project on Science and Intellectual Property in the Public Interest.
*   Achen, Christopher H. 2005. Two cheers for Charles Ragin. Studies in Comparative International Development 40:27-32.
*   ATIS Telecom Glossary. 2007. The Alliance For Telecommunication Industry Solutions.
*   Auer, Sören, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. The Semantic Web. Edited by K. Aberer, K.-S. Choi, N. Noy, D. Allemang, K.-I. Lee, L. Nixon, J. Golbeck, P. Mika, D. Maynard, R. Mizoguchi, G. Schreiber and P. Cudré-Mauroux. Vol. 4825, Science. Berlin, Heidelberg: Springer Berlin Heidelberg.
*   Awre, Chris, and Neil Jacobs. 2006. The technology of open access: Chandos Publishing.
*   Battaglia, M. 1998. Process-based forest productivity models and their application in forest management. Forest Ecology and Management 102:13-32.
*   Benkler, Yochai. 2002. Coase's Penguin, or, Linux and The Nature of the Firm. The Yale Law Journal 112:369-446.
*   ———. 2007. Wealth of Networks: Yale University Press.
*   Commons, Science. 2009. Protocol for Implementing Open Access Data  2009.
*   Tosmana - Tool for Small-N Analysis 1.3.1, Trier.
*   Drucker, Peter F. 1993. Innovation and Entrepreneurship. 1st editio ed: Collins.
*   QCA(GUI) Package In R 0.6. University of Bucharest, Bucharest.
*   Fox, Christopher John. 1983. Information and misinformation : an investigation of the notions of information, misinformation, informing, and misinforming. Westport, Conn.: Greenwood Press.
*   Gantz, John F. 2008. The Diverse and Exploding Digital Universe.
*   Gantz, John F, John Mcarthur, and Stephen Minton. 2007. The Expanding Digital Universe. EMC Corporation.
*   Halb, Wolfgang, Yves Raimond, and Michael Hausenblas. 2008. Building Linked Data For Both Humans and Machines. Paper read at Workshop on Linked Data on the Web.
*   Hars, A., and S. Ou. 2001. Working for Free? Motivations of Participating in Open Source Projects. In 34th Hawaii International Conference on Systems Sciences.
*   Hertel, G., S. Niedner, and S. Herrmann. 2003. Motivation of Software Developers in Open Source Projects: an Internet-based Survey of Contributors to the Linux Kernel. Research Policy 32:1159-1177.
*   Hitchcock, Steve, Tim Brody, Christopher Gutteridge, Les Carr, and Stevan Harnad. 2003. The Impact of OAI-based Search on Access to Research Journal Papers. Serials 16.
*   Innovation Measurement: Tracking the State of Innovation in the American Economy. 2008. Advisory Committee On Measuring Innovation In The 21st Century Economy.
*   Johnsen, Kurt, Lisa Samuelson, Robert Teskey, Steve Mcnulty, and Tom Fox. 2001. Process Models as Tools in Forestry Research and Management. Forest Science 47.
*   Lakhani, Karim R., Lars Bo Jeppesen, Peter A Lohse, and Jill A Panetta. 2007. The Value of Openness in Scientific Problem Solving. In Working papers. Cambridge, MA: Harvard Business School.
*   Landsberg, Joseph J., and Stith T. Gower. 1997. Applications of Physiological Ecology to Forest Management: Elsevier.
*   Lerner, Josh, and Jean Tirole. 2002. The Simple Economics of Open Source. Journal of Industrial Economics 52:197-234.
*   Lessig, Lawrence. 2001. The Future of Ideas: the fate of the commons in a connected world: Random House.
*   Losee, Robert M. 1997. A Discipline Independent Definition of Information. Journal of the American Society for Information Science 48:254-269.
*   Lyman, Peter, and Hal R. Varian. 2003. How Much Information? Berkeley, CA.
*   Mccurry, D, James Campbell, M Lutz, Harlan Onsrud, and K Williams. 2006. Managing intellectual property issues in a commons of geographic data. Digital Libraries Proceeding:347.
*   NRC. 2003. The Role Of Scientific And Technical Data And Information In The Public Domain: The National Academies Press.
*   Peterson Bishop, Ann. 1998. Logins and Bailouts: Measuring Access, Use, and Success in Digital Libraries. Journal of Electronic Publishing 4:27-38.
*   Piowar, Heather. "Foundational studies for measuring the impact, prevalence, and patterns of publicly sharing biomedical research data." Dissertation, University of Pittsburgh, 2010.
*   Porté, A, and H.H. Bartelink. 2002. Modelling mixed forest growth: a review of models for forest management. Ecological Modelling 150:141-188.
*   Ragin, C. C., and S. I. Strand. 2008. Using Qualitative Comparative Analysis to Study Causal Order: Comment on Caren and Panofsky (2005). Sociological Methods & Research 36:431-441.
*   Ragin, Charles. 1989. Comparative method : moving beyond qualitative and quantitative strategies. 1st ed. Berkeley, CA: University of California Press.
*   ———. 1989. The Logic of the Comparative Method and The Algebra. Journal of Quantitative Anthropology 1:373-398.
*   ———. 1989. The Logic of the Comparative Method and The Algebra. Journal of Quantitative Anthropology 1 (4):373-398.
*   ———. 2008. Redesigning Social Inquiry. Chicago: University of Chicago Press.
*   User's Guide to Fuzzy-Set/Qualitative Comparative Analysis. Department of Sociology, University of Arizona, Tucson, 85271.
*   Raymond, Eric Steven. 2000. The Cathedral and the Bazaar.
*   Rennolls, K. 2003. Forest Modelling: Conserving the past and building the future. In Modelling Forest Systems: CABI.
*   Special 301 Report. 2007. Office of the United States Trade Representative.
*   Sweezy, Paul M. 1943. Professor Schumpeter's Theory of Innovation. The Review of Economics and Statistics 25:93.
*   Thornton, Patricia H. 1999. The Sociology of Entrepreneurship. Annual Review of Sociology 25:19-46.
*   Van Orsdel, Lee C., and Kathleen Born. 2009. Reality Bites : Periodicals Price Survey. In Library Journal.
*   Weber, Steven. 2005. The Success of Open Source: Harvard University Press.
*   White, Sonya, and Claire Creaser. 2007. Trends in Scholarly Journal Prices 2000-2006. In LISU Occasional Paper: Loughborough University.