Journal for Dissemination

Announcements

ErdemirA 16:19, 27 July 2010 (EDT) I've received an e-mail from http://www.webmedcentral.com. It appears that they are trying a new model: free and open publishing immediately followed by author solicited open review process (see http://www.webmedcentral.com/about_us/Our_Peer_Review_Policy). Unfortunately, most reviewed articles are simply samples; it will be interesting to see how that experiment goes. It seems that their business model will likely rely on future advertising.

ErdemirA 09:24, 7 June 2010 (EDT) Zohara Cohen of NIH/NIBIB sent me an interesting link to a survey related to open access publishing, see http://www.surveymonkey.com/s/soap_survey_a. This survey is conducted by the Study of Open Access Publishing (SOAPs), an EU funded project. I believe the results of this study will be of interest to many of you.

ErdemirA 19:30, 18 March 2010 (EDT) PLoS One has some mechanisms for publication and dissemination of software, see the heading "Reporting Software Articles" in PLoS One Guidelines for Authors. I wonder similar mechanisms can be established for models and data used for simulation-based medicine. It seems like there is a gap between dissemination of models and data for simulation when compared to dissemination of software used for model development and simulation. Another note is related to open review, which is not necessarily implemented in PLoS journals but available in a few open access journals. Any comments?

ErdemirA 12:58, 10 February 2010 (EST) I have been discussing the social aspects of science with my colleague Ton van den Bogert and he pinpointed many interesting issues. I am including him to our discussions. He can provide insight not only from a scientists point of view but also from the perspective of professional societies. He is affiliated with International Society of Biomechanics and has been largely involved with Biomch-L (http://www.biomch-l.org/), a prominent information exchange platform for those working on biomechanics.

Ton pointed out a few interesting information on open access publishing:

I took the liberty to quote some of our e-mail communications, maybe all can provide some feedback.

About infrastructure and rating system (Luis & Joy, any comments?):

"...Are there online publishers that already have infrastructure for posting reviews, public comments, ratings, comments on the comments and reviews,and rating of the comments and reviews.
It would be a huge undertaking to develop such infrastructure, or maybe not, with some funding and an experienced web programmer.
Some elements of this exist on amazon.com and cnet.com and it's easy to imagine how this could help people navigate the huge universe of scientific publications and find out what is good and useful..."
"...Another interesting model is Netflix. When you give a rating, it uses that information to suggest other things you might like, because it finds people with similar tastes. It this is an effective way to navigate the universe of science (or movies), this is an incentive to give ratings..."

About public participation (Phil, any comments?):

"...I looked at PLOS Computational Biology. Randomly picked an article from 2008 that had more than 5000 views, only two ratings (1-5 stars), and only one comment. The comment was from the authors themselves, correcting an error in an equation. So the public participation is rather disappointing...
Also I did not see reviewer comments posted, do you know of a journal that does that?..."

Also discussed issues are funding and business model, basically who will pick up the tab, a collaborative effort between societies, funding agencies, industry, and universities and/or research institutions?

ErdemirA 12:37, 10 February 2010 (EST) The guest editorial on dissemination was published in Biomedical Computation Review (see http://biomedicalcomputationreview.org/6/1/index.html). I got one feedback from Sargis Dallakyan, which was quoted in here (also check his blog at http://mostlyrx.blogspot.com)

"... One of the differences between scientists and open source developers, that I can see, is that open source development is done by programmers who stay longer at their position. In contrast, postdocs, who write large number of publications, are not interested in investing more time to make their results more accessible to public.
That being said, there are also many open source project that are no longer being developed. What I think is missing in modern scientific publication are some of the Web 2.0 features that make sites like Wikipedia a success. One interesting development in this respect is PLoS Currents: Influenza ..."

ErdemirA 10:38, 25 November 2009 (EST) A guest editorial was submitted to Biomedical Computation Review related to this topic. Hopefully, this will spark some discussions within the larger community of physics based modeling and simulation groups. For comments, please send me an e-mail, erdemira@ccf.org .

ErdemirA 15:06, 3 November 2009 (EST) First conference call was held on November 3, 2009 at 1:00 PM EST. Attendees were present from the Working Group 6, Simbios, and Kitware. Brief meeting notes are in following:

  • Kitware group provided a brief overview of the infrastructure they have been using for Insight Journal. The slides will be provided by Kitware.
  • Ahmet Erdemir summarized the intentions and the new model for Journal for Dissemination in Simulation-Based Medicine, referring to this wiki page.
  • Overlaps and differences of the proposed journal and traditional publishing, insight journal and SimTk architecture were discussed (and probably need to be incorporated in the issues section below).
    • Is it appropriate to call this effort a journal? A repository instead?
    • How is this different than publishing in a traditional journal and providing data in SimTk? Another mechanism to give credit to investigators who share information? Faster? A new rating system?
    • Is this a possible means for investigators to share information when traditional publishing of data, models, and software is not possible? Should the proposed mechanism compete or complement traditional publishing?
    • Is the rating system allows appreciation of investigators similar to traditional publishing?
  • Can traditional publishers require and accommodate scientific dissemination?
  • Initial funding necessary to launch such a journal was briefly addressed by Kitware group (~$40k) and future sustainability was mentioned by Ahmet Erdemir, based on the funding section below.

ErdemirA 13:11, 31 October 2009 (EDT) Joy invited me to write a guest editorial at Biomedical Computation Review about this topic. It will be a great opportunity to spread the idea to a large number of investigators, who are likely to support dissemination. For comments, please send me an e-mail, erdemira@ccf.org .

ErdemirA 12:48, 31 October 2009 (EDT) For all those who are interested in this topic and want to contribute to this page; if you do not have access to edit this page, please contact Ahmet Erdemir, erdemira@ccf.org . It is anticipated that MSM consortium will be interested in this discussion (hence the page is here) but for convenience we may setup a project site at SimTk.

History

Dissemination of data, models, and software driving scientific investigations requires considerable amount of resources and time, in order to bring the information into a usable form. The infrastructure necessary to provide access to the information, promote its use, and establish confidence by its users further complicates the problem. As the information and tools become complex and include multiple disciplines, physics and scales, a structured mechanism to access the disseminated information and to judge its quality become a significant need. Currently, dissemination of data, models, and software is conducted mainly through two mechanisms: traditional publishing, and online repositories created by the investigators.

Dissemination using traditional journal publishing is an attractive option for investigators, as the mechanism facilitates recognition of their work. Publication by an investigator has a merit for advancement of academic career, commonly dictated by the institutions they are associated with. On the other hand, publishing in a traditional journal commonly requires answering a specific scientific question, where data, models, and software are used and hopefully disseminated as a by-product. Traditional journals usually lack public access and suffer from slow processing of submissions. In addition, there is no opportunity for the information to be available to others, if the work cannot successfully go through the sometimes inadequate review process. Although some journals allow dissemination of additional information related to the article, their infrastructure is not necessarily suitable for detailed provision of data, models, and software, and their descriptions.

Online repositories, either through individual researcher websites or in a more organized manner such as SimTk, is an alternative for dissemination of data, models, and software. The mechanism can certainly support traditional publishing, with the investigators providing links to their sites in their publications. The advantage of online repositories is that dissemination is immediate, it does not go through a slow review process. Of course, the investigator may choose traditional publishing before they disseminate their data but the mechanism provides the means for timely distribution of information. There usually is no rating mechanism (exceptions are noted), that can increase the credibility of the disseminated information. The end-user is responsible to judge if the information is right or wrong. He/she can rely on the credibility of the investigator based on traditional publication record. Usually, investigators dedicated to public dissemination of their research, and/or those enforced by the funding agencies to maintain such a practice select this pathway. It seems that lack of a reward mechanism, appreciated by the scientific community and their institutes (citation for example), hinders common use of this pathway.

The discussion on a journal to meet advantages of both these pathways, to encourage data availability for everyone, has started between Ahmet Erdemir and Trent Guess, in an IMAG Tissue Mechanics Working Group meeting at the Annual Meeting of the American Society of Biomechanics in 2009. The idea evolved through e-mail communications with interested parties listed below. This page intends to summarize those discussions and also provide the specifics of such a journal and requirements to launch one.

Goals

  • Provide a free and open access dissemination platform for data, models, and software in simulation-based medicine
  • Provide a platform for expert and public rating, therefore certification, of data, models, and software in simulation-based medicine
  • Expedite and encourage dissemination of data, models, and software in simulation-based medicine
  • Recognize investigators who disseminate data, models, and software in simulation-based medicine

The journal should be aimed to provide a service to the community and should be designed for readers and authors. While recognition of scientists is important to encourage dissemination, the journal should not be solely a platform for building reputation of researchers or for measuring their performance.

Interested Party

Working Group 6

  • Ahmet Erdemir
  • Trent Guess

PLoS Computational Biology

  • Philip Bourne

Insight Journal

  • Stephen Aylward
  • Luis Ibanez
  • Julien Jomier
  • Will Schroeder

International Society of Biomechanics

  • Ton van den Bogert

NIBIB/NIH

  • Grace Peng

Simbios/SimTk

  • Joy Ku

Infrastructure

An online infrastructure should be established, requiring minimal effort from authors, raters, editors. The platform should allow dynamic evolution of the article, its rating, and its recognition. Available journal management systems, e.g. Topaz, can provide the answer.

Submission Process

Submissions will be online. Asking the authors to comply a standard submission document outline and data, model, and software upload process will facilitate automated generation of the article. Immediate availability of the article at the journal site will also be possible (encouraged action). During submission, the author may have the decision to put the article immediately on public display or not. If the authors want to rely on a more traditional review system and wait for acceptance for dissemination, it is possible. If the article is rejected, the authors can still provide it at the journal site for dissemination and rating by the publication (see below). Otherwise, they can withdraw the article to send the work to more traditional journals.

Rating of Articles

Given minimum standards for dissemination, every data, model, and/or software is worth sharing and publishing. Current review mechanisms in traditional journals may not necessarily match this philosophy. In addition, the traditional review system seems to slow down the process, delaying sharing of the information for months. On the other end of the spectrum, the article with the data, model, and/or software, can be provided immediately, with the rating of the article, its criticism, and its reproducibility relies on online ratings by prospective users and the public. While many examples of the latter mechanism illustrated its utility and accuracy, it may not be suitable for scientific publishing, when the peer review process is seen as a measure of quality. For the journal for dissemination, a new proposed rating system can benefit from the advantages of both mechanisms:

Minimum Required for Dissemination

To ensure immediate availability of the submission to the community, the journal can assess:

  • the submission's suitability to the goals of the journal,
  • the article's inclusion of the motivation and procedures to collect data and/or develop model and software,
  • accompaniment with relevant data, model, and/or software, and
  • accompaniment with relevant documentation on how to use the data, model, and/or software.

Ideally, all submissions incorporating the above should be shared and go through the rating process regardless the extent of the disseminated information.

Rating System

  • The article and associated data, model, and software is published and open to public rating immediately.
  • The journal also assigns raters (more or less in the same research area) to make sure that each article gets at least a handful of ratings.
  • The journal assigned raters conclude their work. Also informed by available public ratings at that time, the journal assigns a status to the article based on a minimum rating score, i.e. a featured article status, which may be desirable by traditional investigators. The article will be published regardless the rating.
  • Negative ratings by the journal assigned reviewers can be challenged by the authors, by submitting a revision and request for re-evaluation.
  • Negative ratings by the journal assigned reviewers can be challenged by public raters in future. Positive rating by the public may likely bring the article to a higher status, e.g. a featured article.
  • All submissions appear in databases, including their ratings

Rating Categories

Rating categories may include many aspects important for the user community as well as for academic appreciation. While none of these should prevent publication (therefore dissemination), the rating categories should be able to inform the reader and the potential user of the information about the stature of the article. The rating system and the categories are not necessarily designed to prevent sharing but more designed to increase confidence and credibility.

  • Potential of the reproducibility of information (validation and verification)
  • Novelty of information or its nature confirming reproducibility of previous investigations
  • Completeness of information, i.e. are the descriptions adequate enough to use and/or reproduce the information?
  • Potential utility of information, i.e. do the authors provide examples on future uses of the information?
  • Open access, e.g. does the disseminated data and model require special software which is not free or openly avaliable?
  • Usability
  • Compliance to standards

Recognition of Authors

Providing a Digital Object Identifier (DOI) to all submissions will allow recognition of the authors by their peers, by the institutes they are associated with, and also by the funding agencies. This approach is simply aimed at rewarding each individual for sharing their data, models, and/or software, regardless the perceived quality of their work. More information on DOI can be found in doi info & guidelines. In addition, the rating process will also provide a measure of appreciation that can inform the credibility of the author's work (similar to number of citations and/or h-index). The ultimate goal is to provide this rating in bibliographic databases.

Recognition of Raters

If the rater selects, the rating process can be open, providing information on the rater. This mechanism allows giving credit to the raters for their efforts in dissemination. If the infrastructure also allows rating of the raters, regardless the rater is anonymous or uses a pseudonym or real name, the credibility and confidence to the rating system may increase.

Access

It makes sense that a journal dedicated for dissemination should be accessible to everyone freely and openly. The interested parties agree that the open access philosophy followed by PLoS, BioMed Central, and Insight Journal should be utilized for the proposed journal, too.

Promotion of Journal

The success of the journal will likely rely on its credibility. A strong editorial board will facilitate initial submissions to until the journal's acceptance by the community. As the journal evolves, it is possible to seek for listing in databases, such as that of PubMed (also see Journal Selection for MEDLINE® Indexing at NLM).

Funding

It is clear that starting and maintaining a journal can be costly. There are many mechanism for sustainability of open access journals. The business models for such activities has been summarized in Open Access Journal Business Guides of the Budapest Open Access Initiative. A common practice employed by PLoS, BioMed Central, and Hindawi is the reflection of processing cost to the authors (see more on sustainability for these publishers here).

Initiatives exist to help authors who cannot afford publication charges. Funding agencies also support payment of publication charges. Nonetheless, some investigators may refrain from submitting to journals with publication charges, particularly when the aim of the journal is to support and encourage free and open dissemination. It may be possible to avoid author charges. In this case, the journal has to heavily rely on donations, advertisements, and more importantly continuing funding to accommodate operational costs. It should be remembered that the goal of the journal is providing a service to the community rather than do business. New mechanisms of funding (not relying on authors) may be necessary to accomplish this goal:

  • Can this journal be part of National Center for Biomedical Computing?
  • Can this journal be supported by an established non-profit organization?
  • Is it possible to support the journal by an existing company?
  • Is a new non-profit entity necessary?

Issues/Concerns

This sections lists some of the questions and concerns raised in e-mail discussions that are not necessarily discussed in this document yet. Efforts in launching a new journal should be able to address these and will be provided in various sections of this document.

  • What is the information dissemination system that will serve your community ?
  • What is the Journal, that you as a reader, will consider to be the ideal way of exchanging reports, data and software?
  • Will the journal have a lot of momentum?
  • Issue of standards for data, models, algorithms, tools, etc. Some standardization is useful for facilitated sharing and usage of data. To many constraints may be cumbersome and inhibit submissions. On the other hand, do we really need standards, if data, models, and software are disseminated freely and openly with adequate documentation to use them?

Issues raised by NIH IMAG group, conveyed to Ahmet Erdemir by Zohara Cohen on March 15, 2010:

  • What would be the domain of this journal? How would it be distinct from Insight Journal?

ErdemirA 16:55, 15 March 2010 (EDT) Insight Journal's focus is medical image processing and visualization. It's definitely a great system in that discipline. I am more inclined to have a journal promoting sharing data/models/software not only related to imaging but also including many different types of information we may need in a usable form to start new research, build upon it, etc. It can have medical imaging of the knee with mechanical data associated with it, as well as a finite element representation. My research domain is in biomechanics but a broad scientific view may be more useful. Maybe, something like a hybrid of PLOS One and Insight Journal with an open expert and public rating system, where ratings can be accessed through pubmed.

--CohenZ 18:06, 15 March 2010 (EDT) But there will be some constraints in terms of the domain the journal would cover, no? You couldn't cover the whole waterfront of biomedical research... or would you? Would it be limited to modeling studies or do you imagine publishing data+software for any sort of experimental study?

ErdemirA 18:23, 15 March 2010 (EDT) This seems to be an editorial decision. Such a journal can cover a widespread of topics, e.g. PLOS One, Annals of Biomedical Engineering, or it can be focused on categorical disseminations, e.g. Models OR Data OR Software, or it can be limited to a certain discipline, e.g. focus on biomechanics for example. Personally, I will lean on a discipline specific journal with a focus on sharing all categories; e.g. data, models and software in biomechanics. On the other hand, why should we reject anyone who is willing to disseminate, i.e. PLOS One example. The logistics will be more difficult of course, but may be attainable.

 

  • Do you think there’d be objections from the researchers’ host institutions?

ErdemirA 16:55, 15 March 2010 (EDT) Possibly, particularly when data/models/software is proprietary. Nonetheless, I don't think that sharing data/models etc will have significantly different objections and resistance when compared to sharing software. Sharing software is working but people may want to refrain sharing their data and models until they are published in the traditional system. A Journal for Dissemination may provide the award mechanism for the investigators that the host institutes and funding agencies may recognize, i.e. increased visibility.

  • Do you have a plan for addressing conflicts between authors (coauthors on a single paper) regarding data sharing and other legal issues?

ErdemirA 16:55, 15 March 2010 (EDT) When you publish in traditional journals, all the authors should agree content, right? Well, with this mechanism, all the authors should agree to disseminate. I don't see that as being a big problem, particularly when investigators are made aware of such possibility before and during the work. This does not seem much different than agreeing upon a data and resource sharing plan when writing a grant application.

  • Funding… everybody’s question, of course. I know you have a section dedicated to it and that the Open Access Initiative has published a feasible model for this.

ErdemirA 16:55, 15 March 2010 (EDT) There are feasible models indeed but subsidies may be necessary. I am not certain if mechanisms exist to federally fund a journal site. Nonetheless, I also think that these mechanisms can be implemented as part of infrastructure at National Centers for Biomedical Computing.

 

Comparison of Data/Model/Software Dissemination Methods

 
  Proposed Mechanism Traditional Publishing Open Access Publishing (Peer-Reviewed) Open Access Publishing (Rated by Readers) Self Dissemination Organized Repositories
Time to Dissemination Immediate or at the discretion of the investigator At the discretion of the investigator; commonly after publication is accepted At the discretion of the investigator; commonly after publication is accepted Immediate or at the discretion of the investigator Immediate or at the discretion of the investigator Immediate or at the discretion of the investigator
Time to Publication Immediate Depends on the review process Depends on the review process Immediate Depends on publication path Depends on publication path
Location of Data/Models/Software journal site journal site if the journal accepts data submission; or combined with self dissemination or organized repositories journal site if the journal accepts data submission; or combined with self dissemination or organized repositories journal site if the journal accepts data submission; or combined with self dissemination or organized repositories site of the investigator site of third-party repository
Ownership Investigator Journal (for article); investigator (for data) Investigator Investigator Investigator Investigator
Review Process Peer-review (expert rating) + rating by readers, reader rating has the potential to overwrite expert rating Peer-review Peer-review (+ rating by readers, not necessarily influencing peer review process) Rating by readers None; can be supported by traditional publishing None; can be supported by traditional publishing; some attempts are available in some repositories for rating by readers
Author Recognition Aimed for the efforts to be recognized by investigator's institute and funding agencies; Chance to get recognition by public; self-satisfaction; Efforts recognized by investigator's institute and funding agencies Efforts recognized by investigator's institute and funding agencies; chance to get recognition by public; self-satisfaction Chance to get recognition by public; self-satisfaction; potentially recognized by funding agencies Chance to get recognition by public; self-satisfaction; potentially recognized by funding agencies Chance to get recognition by public; self-satisfaction; potentially recognized by funding agencies
Reviewer Recognition            
Access Restrictions Free and open access Restricted Free and open access Free and open access Free and open access Free and open access
Distribution Online Online + print Online + print for revenue Online + print for revenue Online Online
Funding Grants/donations/company support to construct journal infrastructure Publication revenue Author charges + grants/donations/company support to construct journal infrastructure Grants/donations/company support to construct journal infrastructure Investigator funded Grants/donations/company support to construct repository infrastructure
Example            
This dissemination venue values the publication of data, source code and technical reports Yes Not common, depends on journal Sometimes, depends on journal Yes Yes Yes

 

Table sorting checkbox
Off