Comments

You must log in or register to comment.

mongoosefist t1_iwz4hkk wrote

I'm not really impressed by this tbh. Besides the obvious small issues with temporal stability, it looks like it's rather naively fitting it to the geometry of the person.

It seems to have just applied the pattern of the sweater to the subjects arms, so it's not like they're 'wearing' it. It even just applies the exact same folds from the sweater to the person's arms because it can't tell what's a sweater pattern, and what's due to the geometry of the sweater itself.

168

Ok-Western2685 t1_iwzq0wl wrote

Note that such technology in real-time on device did not exist until now... Also this support any input garment image which is unique.

As far as I know they are the first to bring such a solution for AR creators.

45

mongoosefist t1_iwzv3e7 wrote

Your statements are only correct if you're being extremely specific to this use case.

Real-time AR 'clothing try on' tools have been around for a handful of years already. I've never seen an implementation where they support any garment, but I'd argue that due to the fact that this is not meaningfully different than SNAP's normal AR filters, this implementation has a long way to go to catch up to the state of the art.

25

Ok-Western2685 t1_ix0e3ua wrote

Seems you haven't tried it but relied on this example. If you haven't tried this yet on Lens Studio I highly recommend you do. I think you will understand what the fuss is all about when you do.

Said that, I do agree with you that they should improve the realism of this and I hope they do as this tool can be really amazing.

−6

cyancynic t1_ix03kwn wrote

You’re way behind the curve then. Meta/instagram has been around for awhile and even TicToc seems to be ahead of snap on this.

2

venustrapsflies t1_ix0hk1d wrote

yeah the biggest hurdle in figuring out what an article of clothing will actually look like on you is the fact that the vast majority of us aren't shaped like clothing models. the question is how it lays, not how it looks dynamically stretched out over a 2D image.

38

jhaluska t1_ix18e2u wrote

Yep, all this will tell you is if colors go well together...which really isn't that useful.

Now if it new the pattern and could augment it for a person for custom clothing, that would be really amazing.

5

smallfried t1_ix0qi3h wrote

All your points are valid, but I am still impressed that it runs on a phone.

5

MoreOfAnOvalJerk t1_ix2drmi wrote

Not to mention that you cant tell if the clothes actually fit properly as it doesnt look like its taking the actual length/amount of material into account.

You can end up with a shirt with sleeves that go past your hands.

1

partypoison43 t1_iwyyf6d wrote

Snapchat has a lot of good AR features and yet they're still not as famous as other social media apps.

I wanna use it but non of my friends use it.

23

infinitree t1_iwz5svs wrote

The Snapchat app is a disaster. Just garbage icons everywhere. Such a pain to use. I started trying to use it specifically because of the AR stuff. Too many times unexpected things are happening when I tap icons I think I understand the meaning of. You're right, though. Their snap filters are pretty amazing.

35

pennomi t1_iwz8fax wrote

Okay, and their cardinal sin is putting the back button on the right side of the screen as an arrow pointing right. That goes against every UX convention ever.

But yeah, the AR and AI they do is top of the line, and has been for years.

15

mudman13 t1_iwz0eg5 wrote

It's a very popular app but no doubt does the same data harvesting.

12

Thrannn t1_iwzusv0 wrote

Snapchat was much more popular than Instagram.

The app is a mess tho. Usability is pure shit

4

EmbarrassedHelp t1_iwzn3ur wrote

SnapChat is still lagging behind other apps on implementing proper end to end encryption.

1

3deal t1_iwzu57j wrote

In France it is a very popular app.

1

SpatialComputing OP t1_iwyusth wrote

>With the Garment Transfer Custom Component, Lens Developers can utilize an image of an upper body garment that is transferred in real-time on a person in AR. The Garment Transfer Template offers a quick way for you to get started with the Garment Transfer Custom Component and provides a starting point for photorealistic Try-On experiences.
>
>https://docs.snap.com/lens-studio/references/templates/object/try-on/garment-transfer

18

starstruckmon t1_iwz85dc wrote

Is this even ML? Seems simmilar to something like patch based style transfer...

10

Sirisian t1_ix1k40n wrote

There's probably a number of ways to do it. My naive solution would be to use segmentation masking first to simplify the problem without any background pixels. Mediapipe pose can do this. You can use pose to also segment regions, like arms and torso, to specialize the computation.

Can precompute the input image into such regions as well and mask out the person/background to just have the clothing.

That said the documentation says:

> Garment Transfer currently supports only one person in the target image. When using Garment Transfer with two or more people visible, the garment will attempt to apply to multiple targets, and place different parts of the garment on different people.

They have full pose tracking for multiple people, so this indicates this method is different and probably not relying on that previous research. It sounds more global to me which is strange as I'd expect that to be slower. Maybe their temporal pose tracking is more expensive than I'm thinking.

3

Auth0ritySong t1_iwzluon wrote

The hoodie doent move at all. They should be able to do better

8

seek_it t1_iwyyo7k wrote

Can someone explain how can it's inference be that fast that it can process realtime?

5

Ok-Alps-7918 t1_iwz05ql wrote

Using architecture optimised for mobile (mobilenet-style), compiling the ML model for mobile (e.g. AI accelerator chip for iOS devices), quantisation of the model, pruning, etc. I’d also imagine it’s being done locally on the device instead of the cloud.

10

seek_it t1_iwz0epu wrote

That's why I'm even more surprised. Such kind of models are usually GAN based that inference still require good compute power! On-device inference is even more astonishing!

1

pennomi t1_iwz8l6m wrote

It’s likely not a GAN.

7

Ok-Western2685 t1_iwzos8k wrote

It actually is, and indeed runs on the device.

They did amazing work in that regard!

3

simplypeta t1_iwzs2b6 wrote

My clueless dream finally come true!

3

Echowns t1_ix2sbxt wrote

Amazing technology

2

AtumTheCreator t1_iwz2uhj wrote

What if shes 600 pounds?

1

Caldoe t1_iwzj7m0 wrote

it's the resize the cloth? what did you expect

1

EuphoricPenguin22 t1_ix12g8e wrote

You could probably do this with SD's textual inversion and EBSynth. Not in real-time, but it's one way to go about things if you're making a video.

1

gobert22 t1_ix1yvb3 wrote

but how does this affect lebron's legacy

1

Marcus_111 t1_ix257pn wrote

Omg, I won't be surprised if the first AGI will be a Snapchat filter too!

1

merlinsbeers t1_ix2fr2h wrote

What if they actually had different bodies?

1

Willing-Ad7210 t1_ix2xqzp wrote

Does it really show if the shirt size is too big/small?

1

SkyEngineAI t1_ixqalwv wrote

pretty primitive. try this between a woman with bigger boobs and transfer the jumper on random guy. voila! magic happens and tits will grow on him LOL

1

SumitEduardo t1_ix1reol wrote

Same idea that I thought in 2015 and people called it shit.

−2