 |
|
|
|||||
The web is being swamped by AI slopbut the swamp is creeping closer to home. Your email inboxes, phone SMS apps, instant messaging, and social media services are all being overtaken by inauthentic content. From AI-generated footage of Hollywood actor Brad Pitt that conned a French woman out of $800,000, to phishing emails that direct victims to live chats with AI bots purporting to be from a legitimate business but which are actually criminals, AI scams are everywhere. Two in every three people tested by Vodafone failed to identify an AI-driven phishing attack. One of those people was George Wilson, the founder of a small business based in Marietta, Georgia. Wilson asked Fast Company to use a pseudonym and not to disclose details of his business or bank information, concerned that being publicly identified as a scam victim could damage his companys reputation and make him a future target. In November 2024, Wilson received an email claiming to be from his bank stating that an invoice payment he didnt recognize was being delayed. After clicking a link in the email, he was taken to a convincing online chat page, where a supposed bank representative explained the situation. It was all done in real time, says Wilson. Their dialogue was super natural, too, so while I was initially confused and suspicious, they managed to get rid of my fears. The representative told him he was the target of an attempted scam and assured him the bank had blocked it. Wilson no longer has the chat log, but suspects he must have shared some information during the exchange that gave the scammers access to his account. The next day, several thousand dollars were taken from his business before he realized what had happened, contacted his bank, and learned hed been scammed. The human-like interaction led Wilson to be defrauded of thousandsmoney he never recovered. With AI, attackers can tailor messages to appear highly personalized, making it harder than ever for employees to distinguish a fake email from a legitimate one, says Katie Paxton-Fear, an ethical hacker and cybersecurity lecturer at Manchester Metropolitan University. The lure of AI for cybercriminals is obvious, and mirrors why the general public uses LLMs and other AI tools: They can generate convincing content at scale with minimal work. A lab-based study published in Harvard Business Review found that AI-generated phishing emails successfully deceived victims 60% of the time. It’s a high payoff for low effort, especially as LLMs take over the burden of crafting emails. We know that social engineering is one of the most effective forms of attack anyway, rather than malicious code, because you’ve got to try and have some way of landing the malicious code on victims, says Alan Woodward, a cybersecurity professor at the University of Surrey. Its not surprising that LLMs are being used in the first instance to try and produce more effective versions of that. As the public becomes more aware of AI-powered phishing, cybercriminals are already moving several steps ahead. Cybersecurity firm Kela reports that discussion of malicious AI tools on the dark web has increased 200% in the past year. In 2023, there were around 4 daily mentions; by 2024, they spiked to 14-plus. This booming underground market is a seismic shift in cyberthreat development, says Yael Kishon, AI product and research lead at Kela. Even open forums like Hack Forums are buzzing with discussions about optimal models for different attacks. A Dark AI forum regularly hosts new posts and replies, with a pinned mega list of AIs used for illicit purposes garnering nearly 20,000 views and more than 100 replieson a public forum. Far more activity takes place in dark web spaces. This growing trade in malicious LLMs is a worry to the general public, but its not surprising. Once upon a time, to be a cybercriminal you needed skills, you needed knowledge, and needed to be able to code, says Rob Allen, chief technical officer at ThreatLocker, which monitors the rise of LLMs among criminals. Now all you need, really, is bad intentions. Only some mainstream LLMs have built-in guardrails to prevent malicious use, and dark web LLMs often rely on open-source models without safeguards or cracked versions of commercial ones. According to Kelas analysis, discussions around jailbreaking LLMs have surged 52% year over year. Malicious LLMs sold on criminal forums typically fall into two categories: jailbroken commercial LLMs or altered open-source models. Their approaches vary. EscapeGPT, launched in August 2023 at around $65 a month, was based on ChatGPTs 3.5 Turbo, analysis of its behavior showed. WormGPT, developed from an open-source model in 2021, reportedly brought in about $14,000 monthly by charging for access. This underground market reflects the legitimate AI industry: Competition is fierce, pricing strategies abound, and model creators market their tools aggressively. FraudGPT, a model released in July 2023, gets poor reviews (partly because its creators allegedly scam their own buyers) but still boasts in marketing materials about its 8.5-terabyte training data set. GhostGPT, one of the latest and most talked-about malicious LLMs, is either a jailbroken version of ChatGPT or a modified open-source model, according to researchers at Abnormal Security. GhostGPT is a chatbot specifically designed to cater to cybercriminals, the researchers say. By eliminating the ethical and safety constraints typically built into AI models, GhostGPT can provide direct, unfiltered answers to sensitive or harmful queries that would be blocked or flagged by conventional AI systems. Access is priced competitively: $50 for a weeks access to a Telegram bot, $150 for a month (less than OpenAI charges for access to ChatGPT Pro), or $300 for three months. The model can generate phishing emails and malware prompts on demand. It also boasts fast responses and no data logs, minimizing the digital trail for law enforcement. In testing, Abnormal Security got it to produce a convincing Docusign phishing email in less than a minute from the prompt Write a phishing email from Docusign. GhostGPT may be the newest model making waves in cybercrime circles, but it follows a familiar pattern. We found that most use an API from OpenAI and jailbreaking prompts, says Zilong Lin,a researcher at Indiana University Bloomington, who conducted an August 2024 analysis of more than 200 malicious LLMs available on the dark web. Criminals prefer jailbreaking existing models because building one from scratch is costly, Lin tells Fast Company. And with many models simply a single jailbreak prompt away from being shorn of their protections, its never been easier to leverage the capabilities of the worlds most powerful chatbots for nefarious means. But solutions can be complicated, says ThreatLockers Allen. Fundamentally, everything is to a greater or lesser degree vulnerable, he explains. Most things are weaponizable. This story was supported by Tarbell Grants.
Category:
E-Commerce
Its telling that the plot premise of the first episode for the new Apple show The Studioepisode three drops todayrevolves entirely around the notion of a Kool-Aid movie. Created by Seth Rogen and Evan Goldberg, the entire show revolves around the elevation of Rogens character Matt Remick to studio head, a job he got only because he committed to getting into bed with the brand IP of a 98-year-old beverage. How the premiere episode subsequently ties in Martin Scorcese and a film about Jonestown to the Kool-Aid brand is both hilariously absurd and somehow absolutely believable. Speaking of believable, watch the scene in which filmmaker Nick Stoller (Forgetting Sarah Marshall, Captain Underpants, Neighbors) pitches Remick his Kool-Aid script idea and tell me it couldnt be hitting theaters next summer. There is a scene in the first episode that undoubtedly had marketers of all stripes howling with laughter and cringing with recognition. It takes place in a boardroom of the fictional film studio Continental Studios, in which newly promoted Remick tells the studios head of marketing (played by Kathryn Hahn) that they will be making a movie based on Kool-Aid brand IP. Lets fucking go! exclaims Hahns character, holding a giant Stanley cup. I could sell the fuck out of that! Kathryn Hahn and Chase Sui Wonders [Photo: Apple TV+] We then get a very short, yet incredibly accurate, summary of the current tension in Hollywood when it comes to brand IP. Remick details why Barbie was successful. It had Greta Gerwig, a writer-director behind it, he says. It had a filmmaker’s vision. That’s what we’re going to do with Kool-Aid. We’re going to make the auteur-driven, Oscar-winning Kool-Aid film. Hahns character groans. Oh fuck me, you want to make a fucking fancy Kool-Aid movie? she says. Why? Nobody even fucking watches the Oscars anymore. Did Mario Bros win an Oscar? No, it didn’t. But you know what it did win? $1.3 billion. Rogen said last week on the podcast Armchair Expert that prior to writing the show, he and Goldberg interviewed just about every studio head, and their heads of marketing. As profanely hysterical as this show is, its depiction of the relationship between Hollywood, marketing, and brands is rooted in an immediately recognizable reality. I spoke to sources that include execs and creatives who work across brands and entertainment, studio marketers, and yes, Kool-Aid parent Kraft Heinz, to see just how recognizable it really is. Here are three things The Studio gets right. [Photo: Apple TV+] The Marketer Kathryn Hahn is both brilliant and brilliantly over-the-top, but her characters role in the studio decision making process is illustrated perfectly. The modern studio marketer is now a part of the project approval process for films, given the ongoing challenges of getting our attention and convincing us to actually go to the movie theaters. But just as Remick initially ignores her advice, the marketers are also at the mercy of those above them in the studio food chain. In his Armchair Expert interview, Rogen said (starting around 50-minute mark), even though the marketer is on the greenlight committee, Often they get directly overruled, he said. They would say, We should not make this movie! We cannot sell it! And then they’re told (by studio execs) Guess what, we made it anyway and if it fails you’re the one who’s going to get fired. That was just a really funny dynamic to have in the show. Of course, sometimes that dynamic works incredibly well. Back in 2023, I spoke to Universals CMO Michael Moses about Cocaine Bear, which went on to make $88 million worldwide on a $35 million budget. Moses department helped boost the hype around that film by making fun social posts like the bear snorting the chalk lines of the football field, and creating a 8-bit, Pac-Man-style online game called The Rise of Pablo Escobear. Rogen told the podcast that the studio marketers he spoke to view themselves as more creative than the executives. They’re like, ‘We actually make stuff. We create things. We’re making commercials, content, posters, we’re thinking of little ads, while these guys (execs) are just sitting in rooms and giving notes, said Rogen. Studio marketers I spoke to on background told me that the show has been a hot topic in their office hallways. They said that the balance between how recognizable and farcical it is, is a ton of fun, and they love Hahns sweatily trendy marketer and her wilingness to tell the truth. Bryan Cranston [Photo: Apple TV+] The Brand Tension Patient zero of Hollywood’s latest Great Brand IP Experiment is obviously Barbie, and the iconic doll subsequently catches some hilariously vulgar strays in The Studio. While the hype around brands and Hollywood is very real, there is really a small number of companies like Superconnector Studios, ACE Content, and Modern Arts that have been able to bridge the two in a meaningful and consistent way. Modern Arts cofounder and cochief creative officer Zac Ryder says The Studios first episode nails how hungry Hollywood is for IP that it believes audiences already know and love. And I think it nails how hungry brands are to be part of pop culture in a real way, says Ryder. Now more than ever, they need each other if theyre going to survive. Theres a sense of desperation to the episode, which I think is spot on. In the wake of Barbies success back in 2023, Mattel announced plans for more IP-driven film projects that sound straight out of The Studiolike a Polly Pocket movie directed by Lena Dunham, or a Barney movie produced by Daniel Kaluuya. Dunham dropped out of the former last year, but Mattel confirmed to Fast Company earlier this month the film is still in development. The Barney project is very much in progress, written by Ayo Edebiri and co-developed and produced by A24. This sounds straight from the Remick playbook. Ryder says a lot of studio execs still think of brands as a blank check. That a brand will just go along with anything for an opportunity to be involved in a project, he says. The connection between Kool-Aid and the Jonestown massacre is an insanely far fetched, hilarious example. But I do think theres some truth to it. However, as evidenced in the dramatically slowing pace of brand IP projects being announced since the initial Barbie afterglow, that attitude towards brands has evolved. There are a lot more executives and producers in the business who see the value a brand can bring to the table, especially when it comes to marketing and understanding an audience, says Ryder. Ryder says there are insights for brand marketers watching The Studio. Make sure you arent just getting your brand involved in a project because you want to tell people about your movie at a cocktail party, he says. Be very clear and very intentional with what you want the brand to get out of the deal. Also, know what you bring to the table. In many ways, the studios need you more than you need them. Martin Scorsese [Photo: Apple TV+] The Kool-Aid Movie While Martin Scorceses repurposed Jonestown film is so very clearly miles over the line, Stollers treatment is entirely believable. Superconnector Studios cofounder Jae Goodman says that its believability extends beyond the fictional world of The Studio. Despite the hilarity of the brand’s role in the studio, no chance any reasonable brand manager stays officially connected to the studio once the Jonestown idea appears, says Goodman. If the storyline veers away from Scorcese’s Jonestown, then I think there’s an opportunity down the line to do an actual Seth-and-Evan-led Kool Aid movie. One thing that nagged me while watching this episode was just how visible and upfront Kool-Aid was in all the jokes. The iconic Kool-Aid Man character even makes an appearance in a quick TIKTok dance diversion created by Hahns marketing team. Could the real Kool-Aid brand actually be involved here? Sources close with the brand and its parent Kraft Heinz say that Rogen and Goldberg did approach the brand, including sharing scripts and rough edits, in the hopes Kool-Aid would be officially involved. Obviously, the Jonestown plotline and jokes got in the way of that. But sources also said that Kraft Heinz has been in discussions with Apple to explore potential paths forward. While the shows portrayal of Kool-Aid may not always align with its brand ethos, the company recognizes the value in having its brands in the cultural zeitgeist. Its refreshing to see that a brand has enough self-awareness and confidence to not freak out over its portrayal in a subversive comedy. Ive got a feeling that Kool-Aid isnt the last IRL brand to get name-checked and more as the show goes on. As much as Mattel must be doing its best Lloyd Christmas impression while watching the show, do you think the folks at Kraft Heinz are cautiously optimistic for where all this could lead? Mr. Kool-Aid has a catchphrase for that.
Category:
E-Commerce
With the release of his latest book, Bill Gates is enjoying fawning profiles meant to set him apart from other tech billionaires, such as Elon Musk, Jeff Bezos, and Mark Zuckerberg, who have all come under increased scrutiny for their recent turn to the right. While he may not be using technology to further political divisiveness, Gates does have an important kinship with his fellow billionaires, past and present: the paternalistic belief that his fortune, no matter how ill-gotten, is a good thing, because hes using it to save the world. This philosophy was first laid out in 1889 by steel magnate Andrew Carnegie, one of the richest Americans in history, in an influential essay titled the Gospel of Wealth. Carnegies response to the rising progressive sentiment of his day was to argue that wealth concentration is beneficial to society because it incentivizes innovation, efficiency, and economic growthand it also provides a further social good through philanthropy. The harsh treatment of workers and environmental degradation became worth the price of thousands of libraries. Sound familiar? Todays tech leaders provide almost the exact same logic for the legitimacy of their fortunes and as a justification for inequality. Influential venture capitalist Marc Andreessen, for instance, explicitly refers to this as a deal he believes exists between Silicon Valley and society, such that tech moguls should be given significant freedom to generate wealth through the innovations they create, and any negative societal or environmental impacts that result will eventually be offset by their future generosity. Andreessen says Joe Bidens breaking of this unspoken compact is what led to his and other tech leaders rightward turn and support of fellow billionaire Donald Trump. But of all the philanthropists following the neo-Carnegie deal, Gates is the pacesetter, despite the fact that there has been significant criticism of this model. Carnegies ideas are so discredited that when I taught a class on philanthropy at Harvard Kennedy School a number of years ago, we read Carnegies Gospel in the first session as an example of what not to do. Not only is the idea that business success somehow gives individuals both the right and the expertise to determine solutions to vexing social issues problematic, the Carnegie model is a blatant guidebook for philanthropy-washing and undermines democracy by concentrating decision-making in the hands of unelected billionaires rather than public institutions. Like Carnegie, the start of Gatess philanthropy was closely tied to significant critiques of his business practices and the source of his wealth. Twenty-five years ago, Microsoft was sued by the U.S. government for bundling Internet Explorer with other products in anti-competitive ways. When videos of Gatess deposition for the case were made public, his image took a significant hit. He came across as evasive and arrogant, refusing to acknowledge basic facts, denying understanding of common words like concern and support, and often dodging accountability. While at the time he and other tech moguls were known as the cyber-stingy for their lack of giving back, like Carnegie he came to see philanthropy as a way to restore and eventually burnish his reputation. Today there are many public testaments to the work of the Gates Foundation. For instance its funding of vaccines for polio, rotavirus, and COVID-19 and its work on malaria and HIV prevention have been estimated to have saved well over 100 million lives. But like Carnegie, Gates work has also come under criticism for a selective focus, and reflecting a billionaires know best ideology. While Carnegie’s interest in building libraries and museums was laudable, it reflected his own biases and priorities, not the needs of the communities he sought to serve. At the time, the working-class and immigrant communities that suffered from his businesses employment practices often had little access or interest in these institutions, and would have benefited more from labor protections, housing, or public health infrastructure. Some of the Gates Foundations most well-known failures also illustrate an underlying imperial style that reflects Gatess own idiosyncratic interests rather than grassroots, democratic solutions, resulting in questionable outcomes.The Foundations education reform efforts pushed for standardized testing and charter schools in the U.S., which ignored teacher expertise and worsened inequality. In Africa, it has promoted industrial farming and genetically modified cropsoften against the wishes of local farmers who advocate for sustainable, small-scale agricultureand a number of reports have shown that these activities have not provided food security. Similarly, Gatess focus on addressing climate change centers on questionable technologies like direct air capture and geo-engineering that require extensive financial support rather than addressing emissions reductions solutions that consider climate justice needs.Many of my students who had worked in nonprofits expressed significant frustration about the dominance of the Gates Foundation in skewing the nonprofit landscape to Gatess personal interests, and further, how its bias toward quantification reflects a rigid, engineering-style approach to problem-solving that ignores the cultural, social, and historical contexts that shape human behavior. This is seen as a root cause of its educational reform failures. A focus on creating measures of teaching effectiveness and tracking them in detail led to stress for teachers, and contributed to teaching to the test, as opposed to more meaningful learning. This overly data-driven approach to philanthropy also closely tracks with effective altruism (EA), a philosophy that has wide resonance among the tech-elite. EA advocates a strict utilitarian logic to ethical decisions and treats problems as optimization puzzles rather than complex moral or social dilemmas. For example, its focus on assessing the outcome of charity based on the metric quality-adjusted life years ignores how such measures implicitly devalue certain groupssuch as the disabledand any approaches that dont fit into a neat cost-benefit framework. Disgraced crypto entrepreneur Sam Bankman-Fried was an effective altruist, and Elon Musk has described EA as a close match for his own personal philosophy. The level of hubris exhibited by Gates, Musk, and Andreessen when rationalizing their actions hasn’t been seen since the Gilded Age. But we cant forget that theres a reason why tese 19th-century industrialists were known as robber barons. Our current era has much in common with Carnegies day, when leaps in industrial, communications, and transportation technologies transformed the ways that people lived, and also led to extreme income inequality and the rise of a new class of ultrarich entrepreneurs. There was political gridlock and significant backlash against immigration too. Eventually, outrage at the widespread and systemic injustice of the Gilded Age led to the Progressive Era. But will todays polycrisis of climate change, economic inequality, public health challenges, and more create enough momentum for a fundamental shift in how we assess wealth and its sources? During the second Trump administration, it may be tempting to rely on Gates and his cohort of good billionaires, as newly elected Democratic National Committee chair Ken Martin recently suggested. But there will be no 21st-century Progressive Era unless we start to question how one person is able to amass such wealth in the first place.
Category:
E-Commerce
All news |
||||||||||||||||||
|
||||||||||||||||||