If your software has actual academic value, I see absolutely no reason why you wouldn't publish it in academic journals. After all, academic journals examine the scientific or technical value of your software and its underlying methodology/interface, which open source journals won't. Therefore some people might think that the academic contribution of your work has not been validated, which is relevant for your PhD and your academic success.

What I strongly recommend is, either try to publish a regular paper in regular machine learning venues, or try to aim for JMLR OSS track or the Journal of Statistical Software (if your work has a statistical learning flavor). Both are proper academic journals, but specialized tracks for OSS software, that I think your advisor (and hopefully yourself) would be happy with.

> If you get a ton of citations, who cares what journal it’s in?

Citations take a significant amount of time to accumulate (and not all citations are equal).

The way academic incentives work, people generally want to evaluate published work sooner than that, they don't want to wait for five or ten years to see how much the paper will get cited, and they do not want to judge the actual papers (since most papers in the world or even in computer science are outside of anyone's field of expertise), so the main proxy is the selectivity and impact factor of the publication venue, so people absolutely care about what journal it's in, because they will judge your publications primarily (or even solely) by where they're published, assuming that the journal's review and selectivity is more informative about the quality than whatever they can gleam from a quick skimming of your papers.

Generally there will be some sort of semi-objective criteria - e.g. they'll look at Scopus or Web of Science citation counts - ignoring any citations from anything else, and see whether that journal is in the top half of all the journals in that field (for example). And then some evaluation (e.g. for fulfilling research project goals, future grant proposals, or academic job evaluations - in essence whenever funding is involved, the funding officials defer to such criteria) the publications which fit those arbitrary criteria matter, and the rest are worthless.

The second half of the incentive issue (problem?) is that since you can't publish the same research twice (at least not in respectable venues), if you publish it someplace that "doesn't count" then that research work may be wasted for whatever metrics your institution or advisor is evaluated on, being worse than unpublished work that at least has the potential to become a "proper" publication. So my guess is that simply that "more scientific journal" fits those criteria better than the ones you mention, and your advisor is simply aware of the "metrics politics".

A nice middle ground might be creating a doi using Zenodo (github docs) so that your work is citable and trackable while you work towards some publication format or venue that excites your advisor.

What are you hoping to get out of this? Since you're not in academia anymore, why bother at all? Since for some reason you decided to do this, why not do it right. Your advisor seems to think that you have a chance to publish this at a good, community-relevant venue, which is heaps and bounds better than JOSS or JORS. Why, you ask? Well, a couple of reasons:

1. Discoverability: I don't know who your end users are going to be, but I can almost guarantee you that they won't be reading JOSS or JORS. But they'll likely read their community's journals. Maybe even the OSS variants of it. So if you want to tell the world "look, I made something useful", don't publish in JOSS/JORS because you'll reach way more potential users by publishing in a journal your end users are going to actually read.

2. Prestige: It will look so much better on every co-author's CV. You already have your PhD and don't need this right now, but your advisor likely cares because of this (and every other potential co-author). I mean, if you already have 10 NeurIPS publications, one JORS one might make you seem more well-rounded. Likewise if you're in Software Development now, it might actually be beneficial to demonstrate to employers that you're not just a theoretician. But in general, people in research will not take a JOSS publication as seriously.

3. Valuation of your work: Very related to the previous point, but JOSS/JORS aren't were good research ends up. Scientifically, I'd rank it as low tier publication where you publish stuff that wasn't good enough to make it into a big journal. I.e., my first line of thinking would be "okay, the authors created something that wasn't good enough to make it into the software-edition of the journal in his field" (IME most ML adjacent fields have this). YMMV, this is just my very subjective and biased impression. I never actually checked out JOSS/JORS, but this is how I would judge this, and how I would assume others would judge this.

As others have said: if you just need a citeable artefact, there are quicker ways (arxiv or Zenodo). I see JORS/JOSS as a sort of middle-ground. It's nicer and better than just putting it on arxiv, but definitely not as impactful as a "proper" scientific publication.

The advisor is giving you good advice?

You want to optimise for academic citations for your work.

Take the advisor's advice. Aim to publish in highly regarded journals, get the most academic citations.

The advisor is politely saying that if you "do it their way" you'll get a ton more citations.

I understand your point but there is a difference between publishing peer-review and posting a pre-print, and this is critical if your audience comes from the life sciences for example, whom have historically followed peer-reviewed literature. Since thankfully things are moving towards open-access, why don’t you publish your manuscript in arXiv and submit it to a peer-reviewed journal - common practice nowadays? Then everyone will be happy I guess?

Is it not possible to write a Methods or Protocols paper? DeepLabCut did both for example.

Your supervisor has a point.

Publishing in predatory, garbage journals hurts more than it helps. It's tempting to just get it out there and be done but some of these journals will really milk you.

There are plenty of open source mid- to high- quality scientific journals. I would listen to your advisor they have more experience in the field.

How about https://www.jmlr.org/mloss/mloss-info.html which was created especially for cases like yours.

Commenting to say I share all of the sentiment of the post!

I personally was nudged by someone I met at a conference to publish in JOSS which has a pleasant review process. Sure it’s not high academic prestige but I am very proud of the work and GitHub’s own metrics (as silly as stars ⭐️ sound!) not only provide me with satisfaction but I’m certain that outside of academia (when interviewing for internships or meeting new people) people who code are typically really impressed with a well packaged software that clearly has users - it’s almost always a topic of conversation and I am always delighted to talk about it.

I have my fingers crossed that this is a skill that is valued outside of academia 🤞

Yes, this is another very bad mentality of academic world. I believe what you want to do is better and more important.

Also I wondered your software could you reply me it's github repo

SoftwareX is also a nice alternative journal

As a JOSS reviewer and author of papers in academic journals, I’d like to offer my perspective. Please keep in mind I am in industry now, so there’s a bias there.

The first thing to point out is that JOSS has standards for software: tests, documentation, functionality, to name some of them. Reality is that academic journals do not care or ask for any standards and that’s why you end up finding crappy code in many of those repos. Code that is essentially useless because is not tested, not documented, and may be even irrelevant. (All of this you already mentioned)

Then the questions are: what do you want the publication for? To demonstrate skills? To simply get citations? To be a good citizen of the world and share the code? What is what you want to achieve?

IMO, If you are in industry, JOSS is way more valuable. Unless you are doing basic research in industry, your academic publications will have no impact in your career. If you would like to eventually join an open source project as a core developer (to give an example) or to apply to a company that is aligned with the open source philosophy/values, JOSS will be more interesting than other purely academic journals.

JOSS and the open source foundation have an ideal in the background: to make shit reproducible, even if it takes time. And partially, JOSS would like to change mindsets like the one your supervisor seems to have. Because it is by sharing the code that science and technology progress. We want to get out of that stupid loop of “in house software” used ten thousand times to publish ten different papers in one single research group. If the research is computational, then the code must be peer reviewed as well, and that’s what JOSS offers.

Just out of curiosity, have you submitted to Nature? Have you seen their form? I’m asking because even monsters as Nature or Science do not require code or data. It’s “optional”.

[deleted]

If you already graduated then do what what you value the most for the time involved and offer authorship to your adviser. Explain this is what makes sense for you and they can have a paper on their CV or not.

It’s clear they’re pushing for an alternative journal because that’s what valued by their field.

What’s the link to your repo?

you might get more users if and thus attention on scientific papers

The is always a bias amongst the academic research community that if it isn't about science it doesn't count. I would say now that you have graduated and afford fees to publish. Take your shot and add your advisor as a co-author - if they don't like it they will ask you to take their name off the paper - fair game then.

Go for JMLR

Publish in the academic journal, then follow up with a second publication in an open source journal. You have two different platforms with different goals. They also have different audiences.

The good news is that you seem to be doing good recognizable work. So focus on that. The academic paper will help you get some recognition and more importantly diffuse your ideas in the academic community. The open source journal may be helpful for a wider audience who finds your code on git hub, by helping explain a little more than the typical read me , and by writing it you may fill in some gaps in your current explanations and implementation and help gain a wider audience for your code.

Publishing takes time, but I have found that 5 or 10 years later I like having a paper I can go back to, or if I have to explain something to people, I can pull out an old paper. I also find that it helps me remember what I was thinking about at the time.

I have put software in JOSS for similar reasons you state.

Depending on how important your software is, I would consider nature methods paper. I might also try IEEE Access.

Although its been a few years so I don't remember all the details...

My only issue with JOSS is that it sometimes feels anti-academic, rather than just inclusive. Specifically, there are some issues with indexing (e.g. scopus, as I recall) that, as I read through threads about, seems the original organizers just didn't care about or think was important to the mission of JOSS.

Additionally, with the smaller page size, I think searching makes it show up less.

There are a few comments here about zenodo, but JOSS will have you use that anyway to make a doi, so i'm not sure thats really an alternative to what you are looking for.

I do support what JOSS has done and wants publishing to be. I think they have an important place. I only wish it was _also_ able to align inside of academia. From what it sounds like, you don't actually care about that though...

I published recently in JOSS and would agree with your thoughts. JOSS is exactly the right kind of journal for your software I guess - like me you probably don't want to waste time writing a long formal paper when you have already documented the hell out of everything. Many academics are still stuck in a traditional way of thinking and presume JOSS is not a "real journal" because it takes an unconventional approach. JOSS is also relatively new on the scene, hence some of the negative opinions here, but my experience was a thorough review process that actually digs into the code. Sure, I could have probably aimed for JSS or something like that, but frankly I'm also fed up with a lot of the bullshit that goes along with traditional academia, having been in the game for some years! I would try to talk your supervisor around, honestly.

I'm with you - if the publication venue gets good peer review, visibility, and citations, then it wouldn't matter to me.

I could maybe understand your advisor's perspective if they're still seeking tenure and the venue doesn't have great publication stats.

The advisor is not guided by good code, or even research value.

Professors like this just care about a citation in the best journal / conference as possible. The citations, h-index and i10-index.

They then use that to secure their tenure or secure funding.

Unless you go to a really good school, you cannot expect to do anything but cargo cult.

Just publish it wherever they want. Finish your PhD and get out.

Who cares what your adviser likes?