Comments
Puzzleheaded_Read959 t1_jc1irak wrote
Elon is going to buy Microsoft next so he can get GitHub to take down the scraper.
Elon logic.
Origonn t1_jc2a9ha wrote
Nah he's gonna make a joke about its share price and then be forced to buy at that price to avoid SEC.
3vi1 t1_jc35u3j wrote
Microsoft is valued at more than 15x Tesla. Elon couldn't buy a third of MS even if every lender forgot that Twitter's valuation has dropped by half since Musk took over.
SuperSpread t1_jc3x0wo wrote
This. Elon could suck a million cocks a day and charge $500 a pop and still not have enough money after 10 years to buy Microsoft. I'm serious he'd be a a few hundred billion short after sucking billions of cocks - what makes anyone think he could even be a controlling shareholder. Microsoft is worth $1.89 trillion last I checked.
Put another way if all 330 million Americans including the children each tossed in $5K, that still wouldn't enough to buy Microsoft.
DayTradingEdge t1_jc421gc wrote
Tesla is an AI company, a battery company, a solar company and Elon is 100x more intelligent then Gates. The growth rate of Tesla far surpasses the one of Microsoft. Gates could only dream of being ahead of Musk in the next 10 years. Mark my words. Not to mention he has the boring company, building space ships, phones, Twitter might be half the value now, but it will be 10x higher 10 years down the road (this is my Conservative valuation) 🤫😉
scofieldr t1_jc478z8 wrote
Apparently you are already sucking Elon's dick, so he can skip you
OHMG69420 t1_jc1o6kb wrote
How much does a tweet cost? $10? - Musk probably
DevAway22314 t1_jc1s793 wrote
10 Doge
I'm kinda kidding, kinda referencing the fact he actually suggested that
tyler1128 t1_jc1vmhb wrote
I do wonder the legality of offering a side market for buying twitter data scraped from what the website freely gives you. I'm sure those "hacker" forums still sell sock proxy lists on the daily. That plus beautiful soup and not being stupid in how you do it should be both a weekend prototype level project, and pretty cheap. It's been a while since I've done something like that, but socks proxies are a dime a dozen more or less. Now, you are probably utilizing hacked servers, but you aren't hacking them so pleading ignorance would probably do just fine. Plus, Twitter is hardly capable of keeping running now, not sure their scraper detection is exactly "state of the art".
FamousSuccess t1_jc27bzt wrote
I'm not sure if the data will be sold, rather than just tools to gather it.
Even still, from what I've seen in the past not much stands in the way of "ownership" of tweets/FB posts/Social media. It tends to fall in the public IP territory
tyler1128 t1_jc27nqp wrote
I'm personally thinking about writing a service to sell the data at something like 1/10,000th the cost twitter is charging or less. It'd cache most of the tweet data in LRU form up to a specific data limit in a central database, and dynamically grab new data in the case it isn't already there. There's also be a constantly running scraper for new data to throw it in the central DB cache. Only think stopping me is understanding the legal ramifications. On-demand access to historical data is too slow for large cohorts.
FamousSuccess t1_jc2ry05 wrote
Well. Keep in mind that google effectively sells advertising based on user data, and their services/users depend entirely on content and data of non google entities.
So I’d say if google can build a business on other entities public data, so can you.
Not a perfect parallel but a parallel nonetheless
dubiousadvocate t1_jc2jom8 wrote
I don’t think legality enters into it. At worst it’s a EULA violation. Like any public facing website. Grounds for banning the account but these would be throw away accounts to begin. Musk would whine about it but he’d probably also embrace the artificial user numbers at the same time.
One thing we’ve all learned about the man during this debacle is he’s self destructively impulsive and undisciplined.
Mr_ToDo t1_jc2lulp wrote
Well it doesn't use the API, and assuming that it doesn't use a login then it's probably not bound by the EULA since it would all be public data with no agreement to see it.
Could be a bit of fun if it removes the login prompt, but it's pretty random normally and if there isn't an actual hard limit to what you can load then removing it is likely just a technicality at best(It seems more concerned about how long I stare at old tweets then how far down I scroll. I know sometimes I've gone years down if I don't stop scrolling)
dubiousadvocate t1_jc2qc1a wrote
All good points.
haux_haux t1_jc3q8f1 wrote
Didn't LinkedIn sue an organisation for scraping a while back. Did that fly?
bobartig t1_jc42cxs wrote
There’s been a lot of misreporting regarding the recent HiQ v. LinkedIn case from the 9th Circuit. The best write up I've encountered is by an Internet and Web Scraping attorney, Kieran McCarthy
The key takeaway is that in the 9th Circuit (which has the most developed law in this area) web scraping a publicly available website doesn’t necessarily constitute a CFAA violation, but that doesn’t mean what you did was either legal, or that you won’t face legal liability.
dubiousadvocate t1_jc3uc8z wrote
I haven't heard about that. I'm curious too.
Of course anyone can file a SLAP lawsuit and hope to intimidate legal behavior through financial burden.
uglypatiofurniture t1_jc51g9j wrote
I've seen others say that scraping data would give roughly 0.01%. I think this is just a negotiating point for Elon. Hopefully, this blows up in his face like everything else he's done with Twitter.
bitsandbooks t1_jc1wq0k wrote
> Twitter did not respond to a request for comment.
Yeah, probably because they laid off whoever was doing PR.
TinyBig_Jar0fPickles t1_jc293lj wrote
Well they weren't also "coding" so they weren't needed /s
Intelligent-Use-7313 t1_jc5596u wrote
Worked for Tesla lol.
[deleted] t1_jc23wlf wrote
[deleted]
Hrmbee OP t1_jc0l14e wrote
>Twitter’s API is used by vast numbers of researchers. Since 2020, there have been more than 17,500 academic papers based on the platform’s data, giving strength to the argument that Twitter owner Elon Musk has long claimed, that the platform is the “de facto town square.” > >But new charges, included in documentation seen by WIRED, suggest that most organizations that have relied on API access to conduct research will now be priced out of using Twitter. > >It’s the end of a long, convoluted process. On February 2, Musk announced API access would go behind a paywall in a week. (Those producing “good” content would be exempted.) A week later, he delayed the decision to February 13. Unsurprisingly, that deadline also slipped by, as Twitter suffered a catastrophic outage. > >The company is now offering three levels of Enterprise Packages to its developer platform, according to a document sent by a Twitter rep to would-be academic customers in early March and passed on to WIRED. The cheapest, Small Package, gives access to 50 million tweets for $42,000 a month. Higher tiers give researchers or businesses access to larger volumes of tweets—100 million and 200 million tweets respectively—and cost $125,000 and $210,000 a month. WIRED confirmed the figures with other existing free API users, who have received emails saying that the new pricing plans will take effect within months. > >“I don’t know if there’s an academic on the planet who could afford $42,000 a month for Twitter,” says Jeremy Blackburn, assistant professor at Binghamton University in New York and a member of the iDRAMA Lab, which analyzes hate speech on social media—including on Twitter. > >Elissa M. Redmiles, a faculty member at the Max Planck Institute for Software Systems in Germany, says the new prices are eye-watering. “It’s probably outside of any academic budget I’ve ever heard of,” she says, adding that the price would put off any long-term analysis of user sentiment. “One month of Twitter data isn’t really going to work for the purposes people have,” she says. > >Kenneth Joseph, assistant professor at the University of Buffalo and one of the authors of a recent paper analyzing a day in the life of Twitter, says the new pricing effectively kills his career. “$42,000 is not something I can pay for a single month in any reasonable way,” says. “It totally destroys any opportunity to engage in research in this space, which I’ve in many respects built a career on.” > >The pricing documents were provided to WIRED by a researcher who asked for anonymity, since they are still accessing Twitter data through an existing API agreement and worry it could be terminated if they were identified. They say the new costs were “not viable for the academic community.” > >“No one can afford to pay that,” they say. “Even rich institutions can’t afford to pay half a million a year for a thimbleful of data.”
From a lay perspective, it looks like this kind of pricing scheme for API access is designed to eliminate the possibility of independent research on the platform more than it is to generate revenues for the company.
303Redirect t1_jc1jxuz wrote
I was thinking the same thing. He's gutting safety teams, saying safety is just as important as ever, and in the same motion making it harder for people to verify that.
[deleted] t1_jc14tju wrote
[deleted]
[deleted] t1_jc0vm9w wrote
[deleted]
[deleted] t1_jc0xuoj wrote
[removed]
moses420bush t1_jc0yb33 wrote
Wasn't twitters api free once?
__nickelbackfan__ t1_jc1jl4v wrote
yup
and lil ol' Musky now wants to charge half a million / year for this pathetic amount of data
web scrappers ftw
DevAway22314 t1_jc1scwh wrote
It sounds like the goal is to get rid of and/or discredit independent research
No one will pay it, so research will die down. Some woll use scrapers, but then he can just claim they're inaccurate because they aren't "official" results from the API
Mr_ToDo t1_jc2mzgd wrote
Don't worry, he said those publishing "good" content will be except.
It's a good thing he has that trust and safety council or it would be all up to him what good content is...
It is amusing that he's not denying access to "bad" content publishers, they just have to give him money first. That's just so wonderful of him isn't it ;)
canastrophee t1_jc2xg90 wrote
He must be pretty hard up for cash at this point, he's getting closer and closer to the Ed Edd and Eddy Jawbreaker pricing strategy.
cowvin t1_jc5frhf wrote
How much you want to bet that right-wingers will still have access but left-wingers will have to pay?
skilliard7 t1_jc2e1xo wrote
And it led to a lot of people avoiding the platform altogether via third party apps, circumventing ads.
whyreadthis2035 t1_jc1yog5 wrote
Stop. Using. Twitter.
aphelloworld t1_jc3964z wrote
One question... Why?
skilliard7 t1_jc2e62n wrote
There isn't really anything better. What are people going to use, Truthsocial? Mastadon isn't very easy for the average person to use.
Ok_Sir5926 t1_jc2hqey wrote
Why the incessant need to replace it? Just stop using it, and then touch grass. There's no biological requirement to tell the world about your thoughts, 140 chars at a time.
There's no reason you need something "better." If it's bad...stop. Don't look for a replacement for the bad....cuz it's just more bad.
MustacheInterpreter t1_jc2ih4d wrote
Just an aside: Ukraine uses Twitter to put its cause directly before the world. Nascent rebellions, such as in Iran, do also. Twitter is about far more than me showing you what I'm eating.
Ok_Sir5926 t1_jc2isn0 wrote
It's a method they choose to use because there's people on it.
Are you implying there's no way to talk to a large audience of people in 2023 without a specific free social media platform?
MustacheInterpreter t1_jc2jlsb wrote
Twitter for now is an unmediated source of direct communication to an audience without paywalls, tweaks by journalists or their employers. For now. That's lifeblood to these movements. I'm not here to argue with you. Ukraine has itself said the same thing.
Ok_Sir5926 t1_jc2jxfz wrote
I'm not here to argue with YOU. You decided to reply to me. I was talking to another person.
Have a great day.
skilliard7 t1_jc2iigb wrote
You do realize most people use it to keep up on things they follow, but don't post, right? It's a nice centralized location.
Sailor_Jets t1_jc2sauo wrote
Like reddit?
skilliard7 t1_jc2spxc wrote
Reddit has issues with freedom of expression that Twitter does better.
Ok_Sir5926 t1_jc2jkei wrote
You know it hasn't existed for very long? There were things before Twitter, there will be things after Twitter. People found out about shit long before tweets existed. I probably won't follow the goal posts on this one, so no need to move them again. I have a Twitter account. I've logged into it MAYBE 5 times. I still keep up on "things that I follow." I simply use other resources.
Oh, I also don't give a shit about the Musk drama. I find it funny people get so worked up about it. Doesn't mean the concept of the app is worth a shit.
legendofsteve t1_jc3io57 wrote
“But if I can’t use heroin what else can I use??”
Twitter is a drug. No less. People are addicted to it and are going to want to replace it. I agree they shouldn’t but there’s no logic in an addiction.
Ok_Sir5926 t1_jc3kgc7 wrote
Damn good analogy.
ISAMU13 t1_jc48h0f wrote
The Suboxone of Reddit is much better.
tundey_1 t1_jc2yyrk wrote
>Just stop using it, and then touch grass.
It's not mutually exclusive to use Twitter and to "touch grass".
Ok_Sir5926 t1_jc2zdzv wrote
Got it. Thanks.
757DrDuck t1_jc43w1e wrote
Ever been on Facebook? We’re better off on networks without average people.
skilliard7 t1_jc45rs8 wrote
You say this speaking on a platform full of average people... I don't understand your point
poop-machine t1_jc1c53q wrote
Musk is clueless about how the real world works.
DeathGPT t1_jc23yhd wrote
Okay poop-machine, pretty sure Elon is just another cog in the machine of capitalism that has led the brutality of corrupt success in America.
Don’t hate the player being Elon, hate the game being corporate America and the gov. for allowing people Like Elon to succeed.
blindedtrickster t1_jc29hqd wrote
xD
How many times has any variant of 'Don't hate the player, hate the game" been a reasonable argument?
It's way more valid to recognize, as you may have been trying to do, that responsibility doesn't lie with only one individual. You could have easily said that Elon's actions are rediculous and simultaneously lampooned American capitalism, and government, as being a massively influential in Elon's efforts.
But you didn't. You basically said that Elon's not the problem. That's not true. He's not the only problem. Hell, he's not the biggest problem either! But he's part of a compounding problem.
DeathGPT t1_jc2doq9 wrote
I completely understand where you're coming from, and I agree that it's important to recognize that responsibility doesn't lie with only one individual. When we say "don't hate the player, hate the game", what we mean is that we shouldn't blame individuals for the flaws in a system, but rather look at the systemic issues that are causing those flaws.
I think you all seem to forget, Elon pretty much Saved Ukraine on the battlefield with the release of Starlink. But ohh just forget about his good contributions like good little basement commies who repeat the same mainstream ideas on a daily. If 90% of your thoughts are someone else’s, or just ideas perpetuated from an assortment of Redditers and tweeters - become original.
blindedtrickster t1_jc2tsga wrote
> we shouldn't blame individuals for the flaws in a system
I still disagree with this mentality. If you recognize that it's a bad system, what reasonable basis is there to conclude that taking advantage of a bad aspect of a system doesn't assign some level of 'badness' to a person?
Elon's system has helped Ukraine, yes, but he's still looking at is as 'how profitable can this be for me?'. Profit as a primary motive is incredibly dangerous and I don't think it's a good thing. Profit isn't inherently bad, but when profit takes the form of price gouging, it clearly is bad. So there is some form of line in the sand where 'don't hate the player, hate the game' stops being a defendable perspective.
Elon didn't create the technology and he almost doubled the cost to Ukraine-based Starlink subscriptions. He saw an opportunity for more profit and had no problem with it. He didn't care about what negative effects that could/would have on Ukraine.
So no, 'don't hate the player, hate the game' isn't a good methodology. It's bad because bad players working to create a rigged game. If you're unwilling to look at the people creating a rigged game and simply blame the game, you won't ever fix the game.
Finally, it's not a game. It's entire economies and real people. It's regular people's livelihoods. It's important.
apostroangel t1_jc1srri wrote
It's like paying half a million dollars to study and insane asylum where everyone says the same thing they said yesterday.
joeefx t1_jc1uevx wrote
Twitter is descending into a racist, conspiracy content-filled cesspool.
Jaedos t1_jc23c1k wrote
So Parler with worse programming?
50years50cents t1_jc23n0w wrote
Anyone think this might be a deliberate method of preventing researchers from tracking and quantifying hate speech on the platform?
mecha_flake t1_jc26agy wrote
If that's true, it's a stupid way to attempt that. There is nothing stopping people from just scraping Twitter's front end. That means Twitter pays a ton more money because they have to serve much more data.
APIs are a much more efficient way to serve specific data sets.
UnkleRinkus t1_jc2hwiy wrote
They will rate limit the number of queries coming from a given IP address. Then someone will develop an agent that will support crowd sourcing this across thousands of users, and then either Twitter's servers will hopefully crumble under the load or their AWS bills will skyrocket, either of which will accelerate the fall.
Elon is about to discover the Streisand effect.
mecha_flake t1_jc2muq0 wrote
Exactly. Either give people a way to do what they are trying to do or they will find a much less desirable path forward.
tundey_1 t1_jc2yo43 wrote
Also, Elmo doesn't have that many developers to spare. He's starting a costly war that he doesn't have the resources to fight. Kinda like what a person who refuses to pay his bills and would rather be sued by the King of England!
Amaranthine t1_jc558ao wrote
I already see “Rate Limit Exceeded” in the official Twitter iOS app lmao. I’m sure it’s much easier to get around this using a normal browser, but it’s probably going to be made more and more annoying over time -_-
FamousSuccess t1_jc290co wrote
I was thinking the same. API being closed will now shift a huge proportion of querying to scraping. They'll definitely see an uptick in server loads.
Funny enough it would not shock me to see Twitter scale down their server/hardware to mirror their expected data demands as a result. But I could see this back firing pretty quickly when a couple github projects find their way to mainstream and near-plugin ready twitter scrapers are slamming the site 24/7
Going to be interesting to watch play out
mecha_flake t1_jc29ws6 wrote
I don't play the market, but watching Elmo do something stupid with Twitter and then watching how it affects $TSLA has become a new hobby.
tundey_1 t1_jc2ygj7 wrote
But Elmo is a stupid man who has loan interest payments coming up.
AmyCornyBarrett t1_jc0xd5m wrote
Elon is dooming his own investment one (plus 499,999) dollar at a time
mikey-likes_it t1_jc1wgby wrote
Musk trying to tank his own company
VapidRapidRabbit t1_jc25y02 wrote
But has he paid Twitter’s rent yet?
Clewis22 t1_jc1ojzw wrote
Looks like a way of removing access to the API entirely without formally doing it.
slashd t1_jc1z25i wrote
Maybe the real plan is to price anchor this and a few months later he lowers the price to 'just' $1000 and everyone will think its an good deal
londons_explorer t1_jc1g3ee wrote
There are many companies who would be willing to pay far more than this for the data.
Things like investment firms who want to know what is going on and react in real time.
The real solution is to have a delayed data feed - everything more than a week old is available for free. If you want data 15 mins delayed, pay $$. If you want data 1 second delayed, pay $$$$. If you want data immediately, pay $$$$$$$$.
dubiousadvocate t1_jc2iun3 wrote
The value of the data comes from the quality of data. What we’re seeing with twitter is the data is low quality, too much noise not enough signal.
This was true before Musk was forced to buy Twitter. In fact when Musk finally got around to doing basic due diligence after he locked himself into the deal he himself used the low quality as his reason to bail out. 🙄🤣
noobgolang t1_jc1jesg wrote
Hey this is good idea
Westfakia t1_jc1r8lo wrote
Good for whom?
Giving preferential treatment to financial companies based on means alone isn’t going to benefit anyone except the people at the top, who will continue to work to widen the gap between them and everyone else.
yukeake t1_jc1vmxy wrote
Any sort of social research that isn't extremely time-sensitive could make use of week-old data. Academia is the big one, though, as they're always cash-strapped and there's no way they could afford thousands of dollars a month.
FamousSuccess t1_jc27xaw wrote
Currently the situation does more for what you're describing than what was just proposed.
As-is, researchers are cut off entirely. Only those with the means to do say may access the API now.
Free tiered access of some level would level the playing field MORE than less
0pimo t1_jc1xi9l wrote
>Giving preferential treatment to financial companies based on means alone isn’t going to benefit anyone except the people at the top
What planet are you on? This is how the world works. People with money spend it to gain benefits and value.
Westfakia t1_jc2fet3 wrote
Last time I checked this was earth.
I’ve been here long enough to know that leaving capitalism unchecked is not the best course for the majority of society, and that we have a system of laws in place to prevent abuses as well as a tax system that helps redress systemic inequity.
Reall though, are you going to deny that letting oligarchs like musk run rampant is how twitter got fucked up in the first place?
0pimo t1_jc2i0zz wrote
TIL that having to pay for API data from a privately held social media company constitutes "abuse and systemic inequity".
tundey_1 t1_jc2zbxc wrote
>Things like investment firms who want to know what is going on and react in real time.
Were these investments firms using the API when it was free?
[deleted] t1_jc32s54 wrote
[removed]
[deleted] t1_jc12h5b wrote
[deleted]
jj_888_ t1_jc1mjbj wrote
I support these prices. Hell - make it 1M/day. I look forward to the day twitter ceases to exists.
SuperSpread t1_jc3wl50 wrote
These are Ferrari prices for a Pinto quality bottom-of-the-barrel product. .3% of tweets is worthless on a platform mainly inhabited by bots.
[deleted] t1_jc0vzur wrote
[deleted]
[deleted] t1_jc22mo3 wrote
Left Twitter and haven’t missed a second of it and I’m not alone. Bad news. In fact, you don’t hear much about it anywhere.
Yard-of-Bricks1911 t1_jc23op2 wrote
Pretty sure that's the whole point.
mymar101 t1_jc2wqub wrote
Told ya. Musk wants to follow the Adobe model for things like AEM.
Individual-Result777 t1_jc358pl wrote
Would putting the API on the top shelf keep bots and bs off the platform like hiding the cookies from the kids?
Akul_Tesla t1_jc3auaq wrote
My understanding is that Elon is going with the move fast and break thing strategy that works really well for tech stuff obviously this price is too high to attract the majority of customers he'll lower it as capitalism supply and demand works itself out
pa_forge t1_jc3yy2u wrote
It’s going to be a cat and mouse game of people trying to scrape content and Elon trying to change things around just enough to break it. Just ask LinkedIn what happened
Due-Resident-4588 t1_jc35698 wrote
I stopped using Twitter long before elon took over. It’s just a pool of bull shit where everyone hurls insults at each other instead of having a civilized conversation about real issues.
SecSpec080 t1_jc1zcvn wrote
Why do people care what twitter does? Seriously?
pilat909 t1_jc0z885 wrote
This will motivate researchers to web scrape to circumvent these restrictions. Twint can scrape tweets and it supports proxies. It can also be multi threaded. A huge hassle and it's prone to breaking when the site changes, but at least there are alternative means to get around this stupid decision.