Open access law journals must provide information required in new DOAJ application form by year’s end

Orange open access logo in shape of an open padlockThe Directory of Open Access Journals (DOAJ) released its DOAJ Journal Application Form and specified a deadline for currently indexed journals in a March 19, 2014 press release.  A draft application form had been released in September 2013 and was the subject of an October 2013 post on this blog.  Recommendations for compliance are discussed in that post.  Here I highlight changes to the application and Seal of Approval criteria and the deadline now set for currently indexed journals to comply.

Changes in the application form and Seal of Approval criteria

Changes in the new form include a requirement that journals publish at least five articles per year to be indexed in DOAJ.  A few key changes were also made to the Seal of Approval criteria.

The number of criteria required to be awarded the DOAJ Seal of Approval was reduced from seven to six.  The DOAJ omitted the Seal requirement that the journal allow authors “to retain the copyright of the work without restriction,” although this continues to be part of the inclusion criteria (see #50-52 in the new application form).

Also, journals will not be required to “use[] DOIs as permanent article identifiers” to receive the Seal, as stated in an earlier version of the draft form now replaced by a newer version.  The Seal of Approval criteria now only state that journals must “provide permanent identifiers in the papers published,” including DOIs (digital object identifiers), Handles, ARK, EzID, or some other type of article identifier.  This is good news for the majority of open access law journals published in the United States that do not use DOIs.  Use of article-level identifiers of some kind continues to be part of the inclusion criteria (see #26 in the new application form).

There are a few other tweaks to the Seal of Approval criteria, including the addition of the CC-BY-NC license as a permissible license.

Deadline for currently indexed journals

There are currently 33 U.S. law journals indexed in the DOAJ.  To continue this listing, by the end of 2014 journals need to provide the information requested in the new application, which contains as many as 52 fields.  Some of the fields request a URL pointing to the journal’s policies and procedures, substantiating that the criteria for inclusion have been met.  Law librarians can support open access journals published by their law schools by alerting them to the new information required and helping them develop and/or implement the required policies, procedures, and practices.

New search interface

The DOAJ also has a new search interface.  For a list of open access law journals currently indexed in the DOAJ, search by subject “law” and then drill down to “journal” in “Journals vs. Articles,” and then to Subject “Law,” and then select the desired Journal Country.


Infographic: Do U.S. law journals need reform?

A recent study regarding whether U.S. law journals need reform was conducted by Richard A. Wise, Lucy S. McGough, James W. Bowers, Douglas P. Peters, Joseph C. Miller, Heather K. Terrell, Brett Holfeld, and Joe H. Neal.  The study consisted of a survey distributed to student editors, law professors, judges, and lawyers using a convenience sampling method.  A convenience sample is not necessarily representative of the groups from which the respondents were selected, so to the extent representativeness is in question (respondents are described in the article and some limitations are identified by the authors) the study’s findings cannot be generalized to the larger population.  Still, the study paints a portrait of those surveyed that is worth considering.  The infographic below depicts some of the study’s results, published last year, along with other statistics.

Law Journal Infographic.001


Davies, Ross E. “The Increasingly Lengthy Long Run of the Law Reviews: Law Review Business 2012—Circulation and Production.” Journal of Law 3 (2013): 345-72.

Washington & Lee University. Law Journals: Submissions & Ranking [Web site] (2013).

Wise, Richard. A. et al. “Do Law Reviews Need Reform? A Survey of Law Professors, Student Editors, Attorneys, and Judges.” Loyola Law Review 59 (2013): 1-75.

Wolotira, Alena. “From a Trickle to a Flood: A Case Study of the Current Index to Legal Periodicals to Examine the Swell of American Law Journals Published in the Last Fifty Years.” Legal Reference Services Quarterly 31 (2012): 150-83.

Create Your Own

Infographics are a great way to promote your journal or to visualize data or information presented in an article.  There are free web-based templates that can get you started, including Piktochart,, and  I used KeyNote, Apple’s presentation application, and infographic templates and elements from Jumsoft.

Student-edited law journals: Let’s keep a good thing going

A recent essay by Adam Liptak in the New York Times criticizes law journals on several fronts, including the fact that they are largely student-run. “These student editors are mostly bright and work hard, but they are young, part-time amateurs who know little about the law or about editing prose.”[1]  Liptak questions why students are responsible for defining the body of legal scholarship.  The essay goes on to criticize law journals as under-cited, irrelevant, obscure, and plagued by bias in the selection process.

“In-house” student editorial work

The fact that students edit and select a large proportion of the articles written by law faculty and other experts is indeed something that is not done in other fields, making it an easy target for criticism.  Law journal publishing is also unique in that operations have mostly been retained in-house, compared to other disciplines where journal publishing is concentrated in a few large commercial publishers.[2]  By contrast, law journal publishing is a non-commercial enterprise that is supported in part by extracurricular student editorial work.  Many students find the experience educational, rewarding, and practical, with the added benefit of improved employment prospects.

Journal service benefits students

WPA Poster with a group of young people holding sign reading "Be Kind to Books Club, Are you a member?" Arlington Gregg, artist (Chicago: WPA Illinois Art Project, 1940). Library of Congress Work Projects Administration Poster Collection. No known restrictions on publication.

Arlington Gregg, artist (1940)
WPA Illinois Art Project
Library of Congress
WPA Poster Collection
No known publication restrictions

Students have been editing law journals since the late 1800s.  There are probably many reasons this became the dominant model, but in the process students have become important beneficiaries.  Some would now argue that “the law review’s primary purpose is educational”—the experience not only “teach[es] students to write, edit, and think critically” but faculty-author interaction with students throughout the editorial process is in turn an “additional teaching responsibilit[y].”[3]  The skills learned by journal editors are also quite practical and are particularly useful in appellate or federal trial court practice, judicial clerkships, specialized practice areas, or any work involving extensive legal analysis and writing, including academic professions.

Then-editor-in-chief of the Cardozo Journal of Conflict Resolution remarked last year in another cycle of law journal criticism that not only do students “grow[] tremendously” by editing the work of faculty and other authors, they benefit from the experiences of both producing their own scholarly works and “of managing a large team, controlling a significant budget, and interacting with leading scholars from around the world.”[4]

More recently, the senior notes editor of Michigan State Law Review responded to Liptak by highlighting the educational quality of journal participation and the resulting improved employment prospects.  A student editor’s work is

a significant learning experience. Members of the journal spend two years of their law school careers working tirelessly to produce well-written and well-researched articles.  Legal recruiters even seek out law review editors, in part because of the amount of time dedicated to the journal.  A New York City attorney was quoted in U.S. News saying that large law firms are looking for the “smart workhorses,” student editors who are used to devoting 20 hours per week (or more) to their law review tasks on top of the demanding work of a law student.[5]

Also in response to Liptak, editor-in-chief of the NYU Law Moot Court Board journal focused on networking and the practical experience gained in serving on a journal:

We network with alumnae and peers, judges and practitioners. We produce a final product. We foster mentorships with advisors and professors. We work on tight, often high-pressure deadlines. We explore fascinating issues of law. We vastly improve our research, writing and editing skills. In short, it prepares us to be lawyers.[6]

Editors-in-chief of the N.Y.U. Review of Law & Social Change countered some of Liptak’s charges with a description of the journal’s practitioner-oriented focus and highlighted student opportunities for collaborative article writing with practitioners.[7]

Undergraduate research journal motif

Universities around the world are increasingly publishing undergraduate journals that provide students with opportunities to conduct research and write within their disciplines, collaborate with faculty on research and scholarship, and edit a scholarly journal.[8]  The growth of this phenomenon is some indication that the experience is significant in education.

The best time for law students to learn to write and publish within their discipline is in law school.  Some colleges may offer pre-law programs along with opportunities to serve on undergraduate student-edited law journals.  There is, however, no set pre-law path, and students come to law school with majors from a variety of fields.[9]  A juris doctor is also a terminal degree in the sense that it is generally the degree pursued to practice law in the United States.[10]  That makes law school the appropriate place for students to experience journal participation before going into practice.

The problem, as perceived by Liptak and many others, is that law students are not only selecting and editing the work of their peers but also that of faculty, practitioners, judges, and other authors.

Student-edited journals and peer review

Of the 999 law journals published in the United States 66% are student-edited with no formal peer review, according to data from Washington & Lee University School of Law’s Law Journals: Submissions and Rankings for 2012.[11]  While peer review was a feature of some early law journals,[12] it currently only characterizes about 16% of U.S. law journals, some of which may also be student-edited or peer-edited.  The remaining 18% are peer-edited by legal professionals (but not peer-reviewed).[13]

These statistics notwithstanding, many of the 600+ student-edited journals do involve experts and faculty in the article selection process.  Not reflected in the Washington & Lee listing are the 19 student-edited journals that utilize the peer-review services of PRSM (the Peer Reviewed Scholarship Marketplace)[14] and other student-edited journals that expressly use some form of peer review[15] or that more informally receive faculty input on selections.  So peer review is being used to some extent, it is just not yet well-documented.

In the absence of peer review, student editors have been defended as being quite capable of selecting logically sound articles based on their substantive interests without having subject-matter expertise,[16] and there are some faculty, student editors, judges, and attorneys who maintain that peer review is not necessary.[17]

Evaluating the need for editorial change

Criticizing student-edited law journals seems like an age-old tradition, but many journals have been responding to the critiques with new policies and procedures.  As calls for reform persist[18] evaluating the need for change to established editorial processes might be something for some journals to consider. If systemic reforms are proposed, I would hope that they keep in view the student-editor perspective on law journal publishing.

[1] Adam Liptak, “The Lackluster Reviews That Lawyers Love to Hate,” Sidebar, New York Times, October 21, 2013,

[2] Mark Ware and Michael Mabe. The STM Report: An Overview of Scientific and Scholarly Journal Publishing, 3rd ed. (The Hague: International Association of Scientific, Technical and Medical Publishers, 2012): 33,

[3] Cameron Stracher, “Reading, Writing, and Citing: In Praise of Law Reviews,” New York Law School Law Review 52, no. 3 (2007/08): 360.

[4] Brian Farkas, “Fixing Law Review Critics,” Inside Higher Ed, November 30, 2012,

[5] Katie Wendt, “Shaping Legal Scholarship: A Student Perspective,” Michigan State Law Review Blog, October 27, 2013,

[6] Amanda Levendowski, “Why Law Journals Matter,” Oh Snapski (blog), October 21, 2013,

[7] Nicholas Melvoin and Andrew Neidhardt, “How Law Reviews Can Get It Right,” RLSC Blog, October 23, 2013,

[8] The Council on Undergraduate Research (CUR) lists 119 undergraduate journals internationally.  The journals listed by CUR not only publish undergraduate student research but some may also publish graduate student and faculty work.  The journals may be exclusively edited and reviewed by students or have faculty involvement in the editorial process, including some with peer review.

[9] Pre-Law Committee, ABA Section of Legal Education and Admissions to the Bar, “Pre-Law: Preparing for Law School,” accessed November 2, 2013,

[10] The educational requirements for sitting for a state bar may also include foreign law degrees and other forms of education or prior bar admissions.  National Conference of Bar Examiners and American Bar Association Section of Legal Education and Admissions to the Bar, Comprehensive Guide to Bar Admission Requirements 2013: 8–16.

[11] Washington & Lee School of Law, Law Library, “Law Journals: Submissions and Ranking, 2005–2012,” accessed November 3, 2013,  Search U.S. and Student-edited.  Note that online supplements are included in this list.

[12] Michael I. Swygert and Jon W. Bruce, “The Historical Origins, Founding, and Early Development of Student-edited Law Reviews,” Hastings Law Journal 36 (1984–1985): 783–786.

[13] In Washington & Lee’s law journal listing, search U.S. and Refereed and then U.S. and Peer-edited to obtain these statistics.  Unfortunately, the listing does not identify which Refereed journals are also student-edited.

[14] Peer Reviewed Scholarship Marketplace, “Current Members,” accessed November 5, 2013,

[15] Stanford Law Review (peer review amenable to shortened selection time frames), Harvard Law Review (faculty peer review), University of Chicago Law Review (occasional peer review), and New York University Review of Law & Social Change (expert practitioner review), among likely others.

[16] Stracher, “Reading, Writing, and Citing,” 355 and note 45 (for the proposition that students are capable of “identifying articles that make valid and logical points,” citing Natalie C. Cotton, “The Competence of Students as Editors of Law Reviews: A Response to Judge Posner,” University of Pennsylvania Law Review 154 (2006): 958).

[17] Richard A. Wise, Lucy S. McGough, James W. Bowers, Douglas P. Peters, Joseph C. Miller, Heather K. Terrell, Brett Holfeld, and Joe H. Neal, “Do Law Reviews Need Reform? A Survey of Law Professors, Student Editors, Attorneys, and Judges,” Loyola Law Review 59 (Spring 2013): 68.

[18] Wise et al., “Do Law Reviews Need Reform?”

Save Page Now and get a permalink from the Internet Archive

The Internet Archive (IA) announced some changes to the WaybackMachine a few days ago, including a tool that allows users to submit a link to the IA for archiving and then receive a permalink if the submit was successful.  The Save Page Now tool is available on the WaybackMachine home page and is fully described by blogger Karen Coyle.

Previously, there was not a reliable way for users to request that links be archived in the IA.[1]  Captured pages also would not be visible for months.  New captures are now visible in roughly an hour, and users instantly receive a permalink for an archived page.

The IA is also undertaking other related initiatives, including fixing broken links in WordPress and Wikipedia and possibly archiving all Wikipedia links.  IA’s Alexis Rossi presented these new changes at a recent Internet Archive celebration.  A video recording is available on the IA website.

The tool described in my post on linkrot does more than archive webpages.  It captures ephemeral content like pop-ups and search results pages that are not archived or displayed by the IA. also allows users to see the links they have created from their account dashboard.  Still, Save Page Now is a great tool for students, academics, and bloggers who want to push cited content into the archive and permanently link to an archived page.

HT @LibraryJournal and @mfgaede

[1] Jason Hennessey and Steven X. Ge, “A Cross Disciplinary Study of Link Decay and the Effectiveness of Mitigation Techniques,” in “Proceedings of the Tenth Annual MCBIOS Conference,” supplement, BMC Bioinformatics 14, no. S5 (2013): Discussion, doi:10.1186/1471-2105-14-S14-S5,

Cover art remix of Bound By Law comic by Keith Aoki, James, Boyle, Jennifer Jenkins (Duke CSPD)
Remix by @ninarose 15 CC BY-NC-SA 2.5

This week’s feature is a shout out for my YouTube playlist, Journal Views. I’ve been collecting videos on topics like law publishing history, journal publishing on tablets, critiques of law journal publishing, social media usage by academic journals, university press book publishing for the wider academic publishing context, open access law journal publishing, and open access generally (what it is and what kind of impact it can have). Take a peek and let me know what you think. Bonus points to readers who suggest YouTube videos to add to my collection.

DOAJ raises inclusion bar and adopts a Seal of Approval difficult for many open access law journals to attain

Open Access Week is a good time to raise awareness of recent policy changes made by the Directory of Open Access Journals (DOAJ) that affect all listed journals, including 36 law journals published in the United States.  Importantly, these changes could cause some journals to lose their DOAJ listing and others to be disqualified from receiving the new DOAJ Seal of Approval for Open Access Journals.

What DOAJ is and what it is seeking to accomplish

DOAJ is a directory and article search engine.  It lists open access journals that contain scientific and scholarly content developed through quality control processes (i.e., via an editorial board or peer review system) and that meet specified coverage and access criteria.[1]  DOAJ has defined open access journals as those (1) that do not charge their users or institutions for access and (2) that allow users to “ ‘read, download, copy, distribute, print, search, or link to the full texts of these articles.’ ”[2]

In an effort to establish itself as “a trusted and reliable information resource that identifies good quality open access journals and filters out disreputable publishers,” DOAJ has raised the bar on journal inclusion.  Following a one-month public comment period that began in June, last month DOAJ finalized changes to its inclusion criteria and put finishing touches on the DOAJ Seal.[3]  These new criteria are currently contained in the draft application, released by DOAJ in September 2013 along with a summary of pubic comments

Criteria for inclusion in DOAJOrange open access logo in shape of an open padlock

To be listed in DOAJ, journals will have to complete a form that currently numbers 33 fields.  Some of the fields pertain to basic journal information, but the bulk of the application sets forth the requirements related to the journal’s funding model, use of permanent article-level identifiers, publication format, available metadata, editorial processes, and levels of openness.[4]  The specific requirements are too numerous to cover here but should be carefully reviewed by journal editors, since currently listed journals will be re-evaluated for compliance by DOAJ within the next year.[5]  The draft application does state that failure to meet all criteria will not automatically disqualify the journal.  Details about how long currently listed journals have to comply will be forthcoming.[6]  Journals can subscribe to DOAJ’s RSS feed or follow @DOAJplus on Twitter for updates.

Law librarian support of law journal publishing

Law librarians at U.S. institutions that publish DOAJ-listed journals could support them during the transition.  Editors could be alerted to the changes and provided with a policy inventory checklist covering the new inclusion criteria.  Librarians might also help journals develop policies and mechanisms for detecting plagiarism and for assigning permanent article-level identifiers.  Law librarians Benjamin Keele and Michelle Pearse provide excellent guidance on these issues in their article, “How Librarians Can Help Improve Law Journal Publishing.”[7]

Qualifying for the new DOAJ Seal of Approval

Close-up photo of a content seal basking in the sun

Seal of approval, by Kyknoord
Flickr CCBY 2.0

Entirely new to DOAJ is the development of a Seal of Approval for Open Access Journals.  DOAJ has had a SPARC Europe Seal since 2008 for publishers that provide journal content under the generous Creative Commons CC-BY license and article-level metadata to DOAJ.  The new seal adds five other requirements.  To obtain a Seal of Approval, a journal must

  • Use DOIs (digital object identifiers) as permanent article-level identifiers
  • Provide article-level metadata to DOAJ
  • Have a long-term digital preservation arrangement in place with an external party[8]
  • Embed machine-readable CC copyright information in its articles’ metadata
  • Allow generous reuse and remixing of its content, in accordance with a CC-BY license
  • Allow the author(s) to retain the copyright of the work without restriction
  • Have a deposit policy registered in a deposit policy directory[9]

While some of these criteria may present a hurdle to listed law journals, this post focuses on the DOI requirement, as these identifiers have not been widely used in legal scholarship and impose an additional cost on providing free content.

“What’s a DOI?”

Most law journals don’t use DOIs, so you may be wondering what they are.  A DOI is an alphanumeric string of characters assigned to a digital object to persistently identify the object and point to its location on the Internet via an associated uniform resource locator (URL).[10]  To assign DOIs to articles, a journal must be a fee-paying member of CrossRef, a DOI link registration agency for scholarly publications.  The per-article deposit fee is $1.00.  Journals would have to provide and maintain the accuracy of article-level metadata and submit the citations of each article.[11]  There are other persistent identifier systems out there, but use of the DOI system is exclusively required to obtain the DOAJ Seal.

Few law reviews assign article-level DOIs,[12] and open access law journals are no different.  There are currently 196 open access law journals indexed in DOAJ, including 36 published in the United States.[13]  Of the U.S. journals, 72%, or 26 journals, do not use DOIs.[14]  This criterion alone disqualifies most listed law journals from receiving the DOAJ Seal.  Not having the seal could carry the taint of being a less credible journal even though the seal requirements are technical.

Consortia, library support for journals that adopt DOI

While DOAJ acknowledged in its summary of public comments that there are other permanent identifier systems and that some publisher groups don’t use DOIs, it retained the DOI criterion as a “best practice in academic publishing.”[15]  DOAJ will explore small-publisher solutions with CrossRef.[16]

Open access law journals with limited resources could form a journal consortium to share CrossRef costs.[17]  Maintaining accurate metadata is, however, particularly difficult for student-edited journals with editorial boards that change from year to year.[18]  Contracting with a vendor to provide DOI services may be an option.[19]  Alternatively, librarians or library consortia could support journals by providing the expertise and continuity needed to manage DOIs.

DOAJ listing is critical, but what about the DOAJ Seal?

Retaining a journal’s DOAJ listing is critical as both a discovery and a credentialing aid.  But how important to open access law journals is the DOAJ Seal?  Can law journals feasibly meet other Seal criteria, like the required CC-BY license?  I’d love to hear from law journals, librarians, or other readers below or on the L.J. eds. Facebook Page.

[2] Ibid. (quoting the Budapest Open Access Initiative).

[3] DOAJ, “DOAJ Publishes Response to Public Feedback on Revised Selection Criteria and a Roadmap,” news release, September 16, 2013,

[4] DOAJ, “Draft Application for Inclusion in DOAJ,” September 2013,

[5] DOAJ, “DOAJ Announces New Selection Criteria,” news release, June 12, 2013,

[6] DOAJ, “Summary of All Feedback Received During the Open Period to Draft 1 of the Revised DOAJ Selection Criteria,” item #10, news release, September 2013 (v.2),

[7] Benjamin J. Keele and Michelle Pearse, “How Librarians Can Help Improve Law Journal Publishing,” Law Library Journal 104, no. 3 (2012): 391–95,

[8] Examples are LOCKSS, CLOCKSS, and Portico.

[9] DOAJ, “Draft Application.”  An example of a deposit policy directory is SHERPA/RoMEO.

[10] “FastFacts,”, July 25, 2013,

[11] Ibid.

[12] Benjamin J. Keele, “A Primer on Digital Object Identifiers for Law Librarians,” Trends in Law Library Management and Technology 20 (2010): 38,

[13] Based on a manual count.  “Browse by Subject>Law and Political Science >Law,” DOAJ, October 23, 2013,

[14] CrossRef, “Title List,” October 21, 2013,  A list of U.S. law journals currently listed in DOAJ, along with a notation for those not using DOIs, is included in my Notes app on the L.J. eds. Facebook Page.

[15] DOAJ, “Summary of All Feedback,” item #8.

[16] Ibid.

[17] Keele discussed this as a possible option for law journals in general.  Keele, “Primer,” 39.

[18] Ibid.

[19] Ibid.

Update on 2014 DOAJ application

Bepress recently published a blog post in DC Telegraph with updated information about the new Directory of Open Access Journals (DOAJ) application. Orange open access logo in shape of an open padlockWhile one of my previous posts indicated that journals would be required to complete the application by the end of 2014, bepress reports that the DOAJ will be “reassessing all its existing stable of journals in 2015. When DOAJ is ready, one of its staff members will be contacting journals’ editors/publishers directly to invite the journal to reapply to the DOAJ database.”

Although numerous problems persist with student-reviewed law journals, these problems do not seem significant enough to alter the status quo significantly or generate competition from peer-reviewed journals. For better or worse, the student-reviewed law model is here to stay.

—Tyler Olkowski, Crimson Staff Writer

From “Despite Alternatives, Student-Run Law Reviews Here to Stay,” The Harvard Crimson (March 13, 2014)

Attorney blogger tips: Law journals on Twitter and engaging the bench and bar

Attorney Steve Klepper offers some great insights on what having an online presence can do for law journals in his blog post, “How Twitter Can Save Law Reviews.”  In view of the Liptak critique, he suggests that journals engage the bench and bar in online conversations and encourage submissions from these groups.  Klepper also points out that effectively using social media to promote articles might make a journal a more attractive venue for publication with authors.  He also offers some useful suggestions on how to use (and not use) Twitter and describes why online companions are a practitioner-friendly venue for publication.  It’s worth a read.

A linkrot remedy for legal citations: links and a preserve for cited sources

Linkrot made headlines last month as a malady infecting half of the high Court’s citations to uniform resource locators (URLs)—“In Supreme Court Opinions, Web Links to Nowhere” reads the headline in the New York Times Sidebar.[1]

Photo of a broken bridge by FarTripper. Flickr, CC BY-NC-ND 2.0.

The broken bridge by FarTripper
Flickr, CC BY-NC-ND 2.0

And in another, Wall Street Journal blogger Jacob Gershman reports, “Supreme Court Says It’s Resigned to ‘Linkrot.’”[2]  While a Supreme Court spokeswoman quoted in the blog post confirms that the Court archives its cited web sources in the case files, lack of access to a disappeared Internet source relied upon in an opinion is problematic. There is always the Internet Archive’s Wayback Machine, but its intermittent captures do not include every page on the Internet[3] and cannot reliably retrieve the cited source.

According to a recent study, some law journal citations are just as blighted by rot. This post looks at the study and describes a just-launched service to remedy rotten links in law journal citations.  But first, let’s get our rot lingo right.

Linkrot and reference rot defined

Photo showing banana hands with crown rot

Crown rot of banana hands
Scot Nelson. Flickr, CC BY-NC-SA 2.0

The term linkrot personally conjures up images of stem rot or fruit drop induced by plant pathogens, and for good reason.  Linkrot is when the URL that is the link between the citation and the cited source no longer works.[4]  It is not to be confused with the closely related reference rot, where the cited link works but the content on the page has changed.[5]  In either case, the desired fruit can no longer be found at the end of the vine.

The rot problem

Over the years there have been a few studies on the extent of linkrot in Supreme Court decisions, law journals, and legal and policy materials,[6] as well as in medical, computer science, and library science scholarship.[7]  The study that triggered the headlines described above was reported in a working paper posted to the Social Science Research Network by Harvard law and computer science professor Jonathan Zittrain and law student Kendra Albert.  The authors distinguish their study from the others as one that deploys a methodology designed to ferret out reference rot in working links.[8]

Zittrain and Albert investigated linkrot and reference rot in three Harvard law journals[9] and in all U.S. Supreme Court decisions (a total of 555 URLs since 1996, when the first hyperlink was cited).[10]  In “spot-checking” those URLS that linked to a working page, the authors found that over 70% of the URLs in the law journal citations and 50% of the URLs cited in U.S. Supreme Court decisions “do not link to the originally cited information.”[11]

The study also critiques efforts and suggestions aimed at solving the problem, including WebCite, a permanent linking and archiving service that is presently threatening to close unless donations are forthcoming; the digital object identifier system (DOI), which would require participation by the content creators; and the Internet Archive, where coverage is hit or miss and where storage and ownership are centered in a single organization.[12]  The authors then discuss a new permanent linking and archiving service that operates across a distributed network of libraries—[13]

The solution is a webpage archiving system developed by the Harvard Law School Library and the Harvard Innovation Lab.[14]  Founding members of the project include CloudFlare, Inc., the Digital Public Library of America, the Internet Archive, and 26 academic law libraries in the United States and England.[15]  The service allows authors or editors to create a permanent, unalterable cache of Internet content being cited in a legal or scholarly work[16] that is then retrievable by a link assigned to the cache.[17]

According to the Terms of Service, the user must have the rights needed to both direct that the content be archived and to grant Perma whatever rights it needs to provide the service in relation to the content.[18]  The user also warrants that archiving the content does not violate the copyrights or rights of others.[19]  The content cached must be freely available on the Internet (cannot be paid-only or login-only content) and must be “cited in a legal or scholarly work.”[20]  Content is submitted at the user’s risk.[21]

The guide Permanent Links for Law Review Citations: Perma, DOI, and Handle, prepared by the Boston College Law Library, describes the target links for archiving as ephemeral websites like blogs and working papers and other content that does not already have a stable URL.[22]  Screens shots of content viewed online, like pop-ups or ads that are temporally experienced, are other possible candidates for archiving when cited in a scholarly work.[23]  Content that may be difficult to archive using is discussed in the guide.[24]

The service is still being beta tested and, from what I can tell, implemented at some law schools.  Although it is not yet open to the public, Perma is considering requests for beta access on the home page.  When the service is up and running, it will be free.[25]

How works

The way this will work is that an author writing a scholarly article pastes the URL of the cited Internet source into the box on the Perma home page.   Perma downloads the content at the URL and creates a new link.[26]  This potentially creates an immediate record of the cited content at the moment the author types the citation into a work, preventing reference rot from the get go.  For the Perma links and caches to be permanent, however, they need to be vested.[27]  That’s where you, the law journals, come in.

Journals can open a vesting account by asking a participating library to sponsor it.   The library can sponsor a journal if it is “a scholarly journal of the sort that a library would subscribe to and archive.”[28]  The role of the vesting members (or journal staff) is to verify that the link in fact points to the cited material and that it is being cited in a scholarly work, warranting its being vested.[29]  Journals with vesting accounts may also create links for citations to URLs when an author has not provided them.[30]

Photo of Two Women Standing in a Kitchen Pantry. Pantry Contains Preserved Fruits and Vegetables, 1946

Mrs. Fidel Romero Proudly Exhibits Her Canned Food
New Mexico, 1946; Local Identifier: NWDNS-33-S-12785
Still Picture Records Section (College Park, MD)
National Archives and Records Administration
No known restrictions on use

Vested links and content are forever in and will not be deleted or taken down unless legally ordered.[31]  Unvested links that an author creates are preserved for two years with the option to renew.[32]  Once a link is vested, what makes the cache permanent is a distributed network of libraries around the country that stores and preserves copies of the data in the event that it is compromised at one location.[33]

What an archived page looks like

This is a link [] to Zittrain’s blog post on The Future of the Internet and How to Stop It.  The metadata on the page indicates that the cache was created on October 2, 2013, at 1:40 p.m.  From here, you have several viewing options.  You can view the cache (1) as a live web site as it presently exists (the default view), (2) as an archived site (no new content since the link was created can be viewed but links from the page cached are active), (3) as a screen capture of the page (no clickable links), or (4) as extracted text containing only the main body of the archived page and none of the page’s sidebars, footers, and other features.  

Access to analytics

Perma tracks the number of views of a cache and displays the statistic at the top of the page.  By creating a new link to that now-archived content, the question arises whether the web analytics will be available to the creator of the archived page.[34]  On Zittrain’s page cited above, the default view is to the live site.  In fiddling with a link to my own blog, accessing the blog from the link appeared to register views in’s statistics.  Also, Perma recommends citing the source URL with the link following in brackets,[35] which could still generate clicks to the original site.

Other possible uses

Perma is focusing the service on citations in legal scholarship for the time being, but as Zittrain told the New York Times, “there is no reason . . . why it could not also work for the Supreme Court.”[36]  Another possible use of links is in the open repository of U.S. case law being developed by the Free Law Project at the UC Berkeley School of Information.[37]

Although vesting accounts are intended for “scholarly journals and similar entities,”[38] professor blogs may be likely candidates for vesting accounts as well, to the extent that they cite to ephemeral sources.   It would also be a great tool for noncommercial scholarly journal publishers in other fields, were the capacity there.

Kudos to the creators and partners

Fictional catalog card created for with the annotations "BUILT & RUN BY LIBRARIES," "No more 404-Not Found," "Linkrot Remedy," "Preserve Your Cited Sources", “Built & Run by Libraries”
Catalog card created by ninarose15
Online card generator by jblyberg,

This is a much-needed, cheap (free!), and easy-to-use tool for authors and law journals to capture, preserve, and permanently link to Internet sources cited in footnotes, before they disappear.  Kudos to the law libraries, legal experts, researchers, and innovators involved!

[1] Adam Liptak, “In Supreme Court Opinions, Web Links to Nowhere,” Sidebar, New York Times, September 23, 2103,

[2] Jacob Gershman, “Supreme Court Says It’s Resigned to ‘Linkrot,’ ” Law Blog, Wall Street Journal, September 26, 2013,

[3] Nicholas Tomaiuolo, “DOIs, URLs, LoCKSS, and Missing Links,” Searcher (July/August 2006): 22.

[4] Jonathan Zittrain and Kendra Albert, “Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations” (Social Science Research Network, working paper series, September 21, 2013): 1,

[5] Ibid.

[6] Ibid., 2–4.

[7] Tomaiuolo, “Missing Links,” 19–20.

[8] Zittrain and Albert, “Perma,” 4.

[9] The journals that were the subject of the study are the Harvard Law Review, Harvard Law School Human Rights Journal, and the Harvard Journal of Law and Technology, beginning with volumes in 1999, 1997, and 1996, respectively (ibid., 1–2, 5).

[10] Ibid., 7–8.

[11] Ibid., 2, 6, 8.

[12] Ibid., 9–10.

[13] Ibid., 10–11.

[14] Ian Chant, “ Aims to Bring Staying Power to Online Legal Citations,” Library Journal October 2, 2013,

[15] “About Perma,”, accessed October 12, 2013,

[16] “Perma Terms of Service,” ¶ 5, effective September 23, 2013, accessed October 12, 2013,

[17] “About Perma,”

[18] Ibid., ¶ 5(a)(iii) & (c).

[19] Ibid., ¶ 5(a)(iv).

[20] Ibid., ¶ 5(a)(i).

[21] Ibid., ¶¶ 5(b), 8, 10(c).

[22] Boston College Law Library, Permanent Links for Law Review Citations: Perma, DOI, and Handle (October 1, 2013): 1,

[23] Zittrain and Albert, “Perma,” 11.

[24] Boston College Law Library, Permanent Links, 2–3.

[25] “Perma Terms,” ¶ 10(c).

[26] “About Perma,”

[27] “General/User FAQs,”, accessed October 12, 2013,

[28] “Librarian FAQs,”, accessed October 12, 2013,

[29] “About Perma,”; “General/User FAQs,”

[30] “Vesting Member FAQs,”, accessed October 12, 2013,

[31] “General/User FAQs,”

[32] “About Perma,”

[33] Chant, “Staying Power.”

[34] Ed Summers, “On,” Inkdroid (blog), September 26, 2013,

[35] “Vesting Member FAQs,”

[36] Liptak, “Web Links to Nowhere.”

[37] Chant, “Staying Power.”

[38] “General/User FAQs,”

Create a free website or blog at

%d bloggers like this: