Submitted by t0t0t4t4 t3_10zzm18 in MachineLearning
As far as I know, the Transformer architecture is patented: https://patents.google.com/patent/US10452978B2/en. Since OpenAI has used the Transformer extensively (including GPT), I'm wondering if this can be considered as patent infringement.
If you know about legal stuffs please share your opinions.
The GPT family of models are a decoder-only architecture which is not covered by the patent.
Additionally, Google has released many open source repositories with transformers and appropriate licensing.
There's also a question of whether a patent of this type will hold up in court. Anything reminiscent of a software patent is on shaky footing.
Yeah, Alice Corp. v. CLS Bank significantly limited the scope of software patents. It ruled that adding "on a computer" to an abstract idea does not itself make it patentable.
I believe that true inventions of real algorithms (like movie codecs) are still patentable though.
I agree that they should be, but in my experience it's been difficult to get algorithms patented that lawyers felt would actually protect IP. Basically they're not that well tested in court so it's yet to be seen how much protection they'll provide.
Which then begs the question - what really is the purpose of filing such patents if they can be circumvented? Seems like a lot of effort for nothing (unless patents are part of some evaluation criteria to climb up the corporate ladder?).
It's an arms race. Do Russia and the US really know that each other's nukes work? Or would they rather just not find out. It's like that... will these patents hold up in court? Well if I have enough of them and it's my 20k patents vs your 15k patents, we're probably going to settle based on the likelihood that enough will hold up that we have proportional mutually assured financial destruction.
The crappy part here is that a patent is essentially useless and inaccessible for the individual inventor. A bigger entity will dwarf them. In the end it comes down to money, and it's just a matter of understanding the nature of the investment, which is not as concretely defined as the general public might think.
Schmidhuber prior art
Patenting stuff is so weird if you think about it, it feels so 19th century
I dunno if a company paid for r&d then they should be entitled to exclusively make money from it for some time. I think the problem is around the what and the when and not the why.
Or they should not do r&d if they cannot accept others people's human rights to use their brains whatever they like. This is like saying a thief had so much effort he should be allowed to keep the stolen good
No, it's not like that at all. IMO that analogy does not make any sense. First, r&d is not just thinking up stuff and then making them. In drug discovery it involves expensive trials. In other fields it may involve a lot of building and scraping things, sometimes from expensive material. Patent should be an incentive to do all that knowing that once that it's done it can monetized in a way that does not allow other companies just to copy and paste without effort. Should they be able to do it for everything and forever? Probably not and that is what I was referring to.
Do you know the ”swipe to write” feature that exists on iPhone and Android, where you can keep your finger down and “draw” the words?
There is some small company suing the big guys atm for this “feature” (imo I think a fraction actually uses it). When I heard it, I lost it as, how can you patent such a thing? I mean yea, it might not be the most simple software to write but it just feels so weird to be able to patent such a (useless) technique
Are you talking about swipe typing? It was a lot faster than peck typing back when phones were small enough to fit in one hand.
AFAIK the company that patented it (Swype) screwed up by making their patent more specific than they needed to and apple/google were able to work around it without licensing it. They eventually got acquired and killed a few years ago.
We're off-topic for this forum, but since we're here anyway...
Patents are tricky when it comes to stuff like this. To successfully patent something software related, you must be able to convince the patent office that what you're patenting counts as a "process", and not as an "idea", or "concept" or "principle" or "algorithm", all of which are explicitly not patentable. The nuances of how you draw the lines between these categories are fairly complex, but in general it often comes down to being able to patent engineering details of HOW you do something in the face of a bunch of real-world constraints, but not WHAT you are doing or any broad generalization of the bigger picture.
It's likely that Swype didn't just screw up and write their patent poorly, but rather wrote the only patent their legal team could succeed in getting approved. If it didn't apply to what other companies did later because they used a different "process" (for nuanced lawyer meanings of that word) to accomplish the same goal, that is an intentional feature of the patent system, not a failure by Swype.
Patent on transformer is like a patent on physics, its hillarious
Newton actually had patents on his laws
Wow, I'm really interested at this history, do you have more detailed literature about this?
It is a specific kind of architecture that is patented. If they implement a different kind, they can patent and use it. The problem with patenting something very general is that someone will always find a way to do it another way. Of course except cases like patenting pi. Then, still we would probably say that we invented a new number called as phi, obtain it via Taylor expansion and use it to compute the area of a circle by dividing the diameters square to the inverse of this number. Until we obtain a patent in this way, the other company may dominate the market because it found it first. But that is the idea of the patent.
My opinion is that patents are garbage and should be put in the dumpster and set fire to with gasoline.
Agree, it only blocks innovation and makes it very expensive.
If there is no patent system then every innovation by any individual will be copied and mass produced by big corporations within the day it’s invented.
Imagine you spend a few years designing a new motor. if there is no patent system, Toyota or Tesla will mass produce it the moment they understand how it works. And since they are far more resourceful, you will never be able to produce anything that can compete with them in quality or scale. At least now with patent system they will have to pay you a little to use your invention.
You may not care if you can benefit from your own innovation, but I still think a system that can protect individual ingenuity is somewhat useful
You don't have to publish how anything works. If I code with other 4 guys chatgpt and we bring it online, it will take time anyone to copy it, and it will be easier to buy us.
Secret and first to market beats patent. Specially in software you can add one "moronic attention" layer and claim that does something different.
Actually patents protect more big corporations than little players they may patent hundreds of random things just in case even if is something which they haven't productize.
WhatsApp for instance, they could have been copied by any company (somehow they were copied) but was worthless.
No every innovation can be materialized just by a handful of people like a software app, and not everyone who is involved in this process is your buddy and can be assumed to have good will.
In any hardware-related industry, you will need corporations to mass produce your innovations. If there is no patent system, the moment the manufacturer figures out how to produce it, the innovation is no longer yours. In fact, this is one of the major reasons there is this whole US-China trade war in the first place. Basically, local Chinese contract manufacturers have access to the manufacturing procedures of foreign companies who invent the products, so they just directly copy it and undercut their customers.
Patent also protects the interest of individual researchers who do RD for corporations. But that’s another topic.
I disagree about that. Imagine you invest millions of dollars then someone makes millions of it and you lose millions of dollars.
Imagine you just didn't invest those millions of dollars, then, and instead someone else developed the idea and didn't want to freeze the rest of the world out of using it.
Patents only makes sense if you assume that the alternative to you inventing something is no one inventing it. Experience shows that's very rarely the case; in general, when an idea's time has come (the base knowledge is there to understand it, the infrastructure is in place to use it effectively, etc.), there is a race between many parties to develop the idea. Applies to everything from machine learning models to the light bulb or telephone, both of which were famously being developed by multiple inventors simultaneously before one person got lucky, often by a matter of mere days, and was issued an exclusive license to the invention, while everyone else who had the same idea was out of luck.
Yes the same as owning trade routes. If you don't want others to use it then don't publish it or don't invest in the first place. Leave the market to good people that don't feel the need to restrict other human's freedoms.
Can google sue OpenAI for training on youtube video ?
That's another question. And the answer might be yes
currentscurrents t1_j865is4 wrote
There's some mutually assured destruction going on here. Microsoft/OpenAI also own patents that cover Google's products. If Google sued them over Transformers, they would sue right back for something else.
https://www.aeaweb.org/research/a-truce-in-the-patent-wars