Author: DarkOwl Content Team

August 14, 2024

The speed and scale at which large language models (LLMs) have captured the attention of investors, the public, and tech startups cannot be overstated. This technology will undoubtedly revolutionize not only our personal interactions with technology but also business, analysis, medicine, and nearly every industry in some capacity. However, there is a dark side to this revolutionary technology. As the founder of Linux, Linus Torvalds, often stated, “With great power comes great responsibility.”

Bad actors have already begun leveraging these LLMs for nefarious purposes, and cybersecurity professionals have been highlighting proof-of-concepts to warn of various ways these models can be exploited. This blog will point out some of the early examples and known vulnerabilities of these LLMs and speculate on where they could lead us in the not-so-distant future.

Types of LLM Exploitation

Prompt Injection – Exploitation of LLMs for nefarious purposes has manifested in various forms. One prevalent method is prompt injection, which ranges from straightforward to sophisticated techniques. At its basic level, malicious users attempt to bypass security filters by presenting prompts in a manner that deceives the LLM into providing unintended responses. For instance, they might present multiple prompts where one is benign and the other malicious, hoping the model responds to the malicious one. This creates a continual challenge for developers and security experts who must constantly adapt to new tactics. On the more advanced end of prompt injection, techniques like encoding malicious questions in base64 can evade security measures by using encoded prompts to evade detection of harmful content. Developers are made aware of these types of obfuscations and appear quick to mitigate them in later releases.

Jailbreaking – Jailbreaking GPT models and prompt injection share significant similarities. When bad actors successfully devise a sophisticated set of prompts that circumvent multiple security measures, such as crafting prompts to generate malware, and the LLM consistently responds, it qualifies as a “jailbroken LLM.” In this compromised state, the model retains these malicious prompts, enabling users to interact with it normally while evading security filters. These jailbroken models are actively traded on the dark web for various illicit purposes.

One notorious example is FraudGPT, prominently featured on the dark web. It purports to execute a wide range of malicious activities, including generating phishing emails, creating keyloggers, producing malware in multiple programming languages, obfuscating malware to evade detection, scanning websites for vulnerabilities, crafting phishing pages, and more. The below image was extracted from a dark web site selling subscriptions of their version of a jailbroken LLM. If you want to learn more about Jailbreak GPTs you should check our a previously written DarkOwl blog that dives deeper into these GPTs.

Training Data Poisoning – Emerging as a significant concern for cybersecurity professionals and LLM engineers, in this method, threat actors and black hat hackers introduce malicious data during the model training process. This tainted training data becomes embedded in the model’s algorithms, eliminating the need for prompt injection. Consequently, malicious or unsafe responses are ingrained in the model’s core functionalities. Depending on the nature of the maliciously infused data, this could potentially enable outputs ranging from the generation of malware to the production of deepfakes and dissemination of misinformation directly from the model’s core algorithm.

Leakage – Leakage refers to various methods that enable LLMs to return sensitive data inadvertently captured during training, which was not intended for redistribution to users. This includes access tokens, personally identifiable information, cookies, and other data types assimilated during the model’s training phase. Such leaks can happen through prompt injection or more advanced techniques. Below is an example posted on X of a user whose cell phone number was captured and used in the output of an OpenAI ChatGPT response. As these models get access to more and more user data, you can imagine the impact of these leakages becoming even more concerning.

AI Agents – This represents a slightly more sophisticated form of exploitation compared to our previous examples. With the rise of AI integration in programming and its accessibility via APIs, there is a burgeoning interest in “AI Agents.” These agents operate autonomously and sometimes possess special privileges on the host computer. For instance, a program could scan files, read data, copy logs, inspect system defenses like Windows Defender, and relay this information to an LLM. Each “agent” is tasked with retrieving specific information—such as scanning logs for leaked passwords or identifying vulnerabilities in a WordPress instance running on a server—using the LLM model. Finally, another agent might execute commands on the host computer based on the information gathered. These agents perform autonomous actions, resembling a sophisticated virus operating intelligently within your environment.

Summary

As we explore the realm of Large Language Models, their rapid advancement offers promising potential across diverse industries—from streamlining business processes to advancing medical diagnostics. However, alongside these opportunities, there are significant challenges. Malicious actors exploit vulnerabilities such as prompt injection and training data poisoning, utilizing these powerful tools for cyber threats and manipulation. It’s crucial to remain vigilant and aware of potential misuse of these tools and mitigate the risk—from potential data breaches to orchestrated misinformation or even AI agent malware.

Curious about trends on the darknet? Don’t miss any of our research! Follow us on LinkedIn.

August 07, 2024

As hands-free, low/no-contact trends exploded in popularity during the pandemic, QR code technology became more prevalent. So, too, do the ways to take advantage of the technology and turn a QR code into a phishing operation, or worse. QR codes are appearing in public places such as parking areas, restaurants, and hospitals. Their convenience is a no-touch way to pay for or order a service. However, the accessibility of QR codes extends not just to patrons looking for a simple, germ-free way to get things done. Unfortunately, malicious actors are taking advantage of QR codes in public places, as well as sending them via phishing campaigns via email and SMS messages.

At the end of 2023, the Federal Trade Commission published a warning about an uptick and tactics used by scammers and fraudsters to disseminate QR codes that stole personally identifiable information (PII) or directed unsuspecting victims to fraudulent websites that would do so. QR codes can also install malware onto personal devices, such as laptops and mobile phones. The dark web and its adjacent platforms, such as Telegram, offer tutorials and services to empower cyber criminals to steal not only information but in some cases, finances of victims, using QR codes:

Figures 1 and 2: On an onion forum, malicious actors discuss QR code fraud sales and cashing out on them using cryptocurrency, as well as possibly accessing Discord; Source: DarkOwl Vision

The easiest method to spread QR code fraud is simply placing a sticker over a QR code located in an open, public place. Criminals can do this outside of the range of security cameras in many instances. These cover-up QR codes can send victims to fraudulent websites.

Alternatively, if QR codes are sent via email, embedding them as an image in the email does not trigger security or scanning software, so the malicious link of the embedded QR code will function and lure victims to the malicious website. This tactic is called “Quishing” – a portmanteau of QR code and phishing.

Both of the above scenarios rely on people using personal devices as they travel out and about, running errands. Personal devices often see lower security protections as opposed to a corporate or employee-sponsored device. Criminals also take advantage of the fact that people are often in a hurry when conducting errands or going to a leisure event, so they don’t take the time to inspect URLs, ensuring no typos or suspicious looking links. To maximize their financial gain, online tutorials offer QR code fraud guides of all types:

Figure 3: A Telegram user advertises for all kinds of malicious services, including QR code fraud; Source: DarkOwl Vision

How to Protect Yourself

Since QR code fraud is similar to phishing operations, the same protective measures apply:

• Always investigate URLs closely, and ensure there aren’t typos, or a possible misdirection located in the code, or the URL provided with the code.
  • This includes ensuring the URL provided uses a secure HTTPS protocol, and not just HTTP.

• Do not click on or scan QR codes from strangers, only open QR codes from trusted sources.

• Don’t download any files from a QR code or permit auto-downloads from any websites related to QR code use.

• Ask employees in places where QR codes are located publicly to verify the website the code takes you to, so that no fraud or information stealing occurs.

Questions for our analyst team of darknet experts? Contact us.

August 06, 2024

In DarkOwl’s Darknet Marketplace Snapshot blog series, our researchers provide short-form insight into a variety of darknet marketplaces: looking for trends, exploring new marketplaces, examining admin and vendor activities, and offering a host of insights into this transient and often criminal corner of the internet. This edition features Ares market.

Don’t forget to subscribe to our blog at the bottom of this page to be notified as new blogs are published.

Dark web marketplaces are synonymous with the dark web where users can buy and sell illicit goods. It began with Farm Market, followed by the more prolific Silk Road. Ever since Silk Road was taken down by law enforcement, different markets have jostled for supremacy. As such, dark web markets are perhaps one of the more recognized things to appear on the dark web and they operate just like surface web marketplaces with reviews, escrow services and reputations.  

However, in recent years law enforcement have become more and more successful at shutting down these marketplaces, meaning that the vendors have to move to new areas. There have also been a number of exit scams from marketplaces with the admins closing down the site and taking the funds in escrow. 

DarkOwl analysts will write a series of blogs reviewing the most popular marketplaces of today after recent seizures of once popular markets like Kingdom, Incognito, and Bohemia Marketplaces. We will explore the various sorts of products regularly sold and well as how much the prices of products can vary within or between product categories.  

Ares Market

The first market we will explore is Ares Marketplace. Originally established in 2021, it is a well-known marketplace that offers a variety of products, from illicit substances and pharmaceutical substances to digital fraud products ranging from credit card fraud, cryptocurrency fraud, malware source code as well as a robust variety of counterfeit products like currency and IDs. Below is a screenshot of the homepage, which is what one would see after a successful log in:

Figures 1 and 2: Ares Market Home Page 

Drugs  

Cocaine and ketamine seem to be the most popular drug products boasting over 1600 listings. Pricing varies considerably listing by listing and vendor by vendor. It’s a challenge to determine which vendors might be legitimate or which vendors could be scammers. Although vendors work on the principle of reputation, and purchasers will quickly leave reviews if they think something is a scam. DarkOwl analysts regularly see ketamine sold by the gram on Ares Market with prices varying drastically: 26 USD for 1 gram all the way to 482 USD for 25 grams: 

Figure 3: Ketamine for sale on Ares Market 

As stated above, reputation is very important for dark web market vendors. The vendors will have profiles on the markets which provide details of how long they have been on the site, how many successful sales they have had and details of the reviews they have received. The below screenshot shows a product listing from a vendor who seemingly has a high reputation, has been a member since 2021, and allegedly has successfully completed 216 sales:  

Figure 4: Profile of Ketamine seller on Ares Market 

Financial Fraud 

Credit card fraud, aka carding, is also a popular product category on Ares with well over 500 listings. Again, prices range dramatically as well as the types of products offered.  

 While Visa, Mastercard, and Amex tend to be the most popular credit card company targets on this site, it is also common to see Credit Unions (CUs) because threat actors consider CUs to be easy targets with the assumption that they don’t always have the same budget to combat fraud. The below screenshot is a good example of a well-known carding threat actor, johnnywalker1, selling bank accounts with active balances from Robins Credit Union, which is a Georgia based credit union. The user is selling these accounts for roughly 136 USD and allegedly will gain full access to an account with an active balance ranging from 3 – 5,000 USD in addition to relevant personal identifiable information (PII) to access the account online: credentials, SSN, DOB, address, etc.  

Figure 5: Credit Union Credit cards for sale 

 Johnnywalker also regularly sells accounts and cards affiliated with larger banking/credit card companies like Amex. This user is allegedly selling one American Express account for roughly 13.50 USD, which is significantly cheaper than the above example of the credit union:  

Figure 6: American Express Credit Card for sale 

The seller does not make clear how they are obtaining these cards, but threat actors are known to clone cards, or access banking information from stole credentials, particularly through Stealer Logs.  

Counterfeiting 

Counterfeiting is also a popular section on Ares Market. The two most popular product categories are counterfeit currency follower by counterfeit IDs.  

 The below examples are from the currency category. The user, CounterKing, seems to have a verified reputation of 5 stars, level 9, and over 120 sales since they first registered in March 2023.  

CounterKing is selling 20,000 Euros of counterfeit currency for roughly 2,284 USD. The post goes into excruciating detail of the product description as well as their Terms & Conditions. Counterfeit cash products are expensive, and it is common to see a price range anywhere from 300 USD to above 3,000 USD.  

  Figure 7: Counterfeit Cash for Sale 

 Administrators 

Marketplaces are operated by admins, who ensure that the market is used in the way that they want and that the rules are followed. Some admins will also manage escrow services and a responsible for banning members who do not follow the rules. The admins of Ares Market are do a decent job of quality control on these listings because they are all related to credit card fraud. It is not uncommon to see less quality control and random products listed under the wrong categories on other marketplaces that are less reputable.  

Subscribe to email to receive the latest research directly into your inbox every Thursday and don’t miss our next Darknet Marketplace Snapshot.

 Setting up DarkOwl Vision alerts on Ares Market

1. Select Ares Market from the lexicon 

2. Then add in the vendor/username you are interested in monitoring, for this example I chose “counterking,” which returned 354 results related to this user 

3. Next let’s create a monitor on future posts from this actor:  

1. Simply go over and click the star highlighted in blue on the right-hand side of the search bar.  

4. Followed by entering in the information to save your alert, choose your alert frequency and alert criticality and then clicking the box to receive email notifications and finally hitting the save button.  

  Figure 8: Ares Alert in DarkOwl Vision

August 01, 2024

Our analyst team shares a few articles each week in our email newsletter which goes every Thursday. Make sure to register! This blog highlights those articles in order of what was the most popular in our newsletter – what our readers found the most intriguing. Stay tuned for a recap every month. We hope sharing these resources and news articles emphasizes the importance of cybersecurity and sheds light on the latest in threat intelligence.

1. AT&T Confirms Data Breach Affecting Nearly All Wireless Customers – The Hacker News

On July 12th, AT&T confirmed that it had suffered a data breach affecting “nearly all” of its wireless customers between April 14th and April 25th, 2024. The leaked files contain records of customers’ calls and texts which occurred on January 2nd, 2023, and between approximately May 1st and October 31st of 2022. The leak also included customers of mobile virtual network operators (MVNOs). The data was stolen from the company’s workspace and on a cloud platform. This data does not appear to have been made publicly available at this time. Full article here.

2. GuardZoo Malware Targets Over 450 Middle Eastern Military Personnel – The Hacker News

Military personnel in the Middle East have been targeted by GuardZoo malware, an Android data-gathering tool. Over 450 victims across Egypt, Oman, Qatar, Saudi Arabia, Turkey, the U.A.E., and Yemen have been impacted by the surveillanceware operation, with the majority of victims located in Yemen. GuardZoo is a modification of Dendroid RAT malware which targets Android OS and was first discovered in 2014. Read more.

3. 4 FIN9-linked Vietnamese Hackers Indicted in $71M U.S. Cybercrime Spree – The Hacker News

Four Vietnamese Fin9 actors were indicted for cybercrime activity between May 2018 and October 2021. They conducted phishing campaigns, social engineering and supply chain attacks that resulted in data theft. In some instances, FIN9 used personally identifiable information (PII) to create fake accounts “tied” to victims from the first stage of their operations, conducting cybercrime from assumed identities.. Article here.

4. US disrupts AI-powered bot farm pushing Russian propaganda on X – BleepingComputer

On July 9th, the U.S. Department of Justice announced the disruption of a Russian, AI-powered information operation devised to spread Russian propaganda in the United States and abroad. The DOJ operation involved the seizure of two domains used to issue emails for the bot accounts, as well as the search of nearly 1,000 social media bot accounts which were subsequently suspended on X (formerly Twitter). According to the DOJ press release, the bot farm was developed by the deputy editor-in-chief of RT (formerly Russia Today), the state-controlled news organization. Court documents also reveal the use of artificial intelligence to enhance the Russian bot farm, reflecting the increasingly normalized use of AI in disinformation operations. Read article.

5. Fake CrowdStrike fixes target companies with malware, data wipers – BleepingComputer

On July 19, the cybersecurity company CrowdStrike distributed a faulty software update to its customers; the update affected devices running Windows, and an estimated 8.5 million computers worldwide were disabled. The incident—which grounded thousands of flights and affected a variety of industries, including the healthcare sector—is believed to be one of the worst cyber incidents of all time. Threat actors quickly took advantage of the worldwide disruptions by impersonating CrowdStrike in phishing emails to distribute malware. Full article here.

6. LockBit lied: Stolen data is from a bank, not US Federal Reserve – BleepingComputer

At the end of June 2024, LockBit ransomware group claimed they hacked the US Federal Reserve. However, further analysis of the data, which LockBit published on their website, proved that in reality, LockBit hacked Evolve Bank and Trust, an entity not at all tied to the US Federal Reserve. When approached, Evolve Bank and Trust admitted they were investigating a cybersecurity incident, but provided no additional details or confirmation. Full article.

7. Ukrainian Institutions Targeted Using HATVIBE and CHERRYSPY Malware – The Hacker News

Ukraine’s Computer Emergency Response Team (CERT-UA) revealed that a Ukrainian research institution has been targeted by HATVIBE and CHERRYSPY malware distributed in a spear-phishing campaign. CERT-UA has attributed the attack to UAC-0063, which it previously identified as targeting state bodies in Ukraine. CERT-UA shared that it is aware of multiple cases of HATVIBE infections. According to previous research, the threat actor UAC-0063 has been linked with moderate confidence to APT28, the Russian GRU-backed threat actor. Read more.

8. U.S. indicts Russian GRU hacker, offers $10 million reward – BleepingComputer

The U.S. indicted 22-year-old Russian national Amin Timovich Stigal for allegedly assisting Russia’s military intelligence service’s “WhisperGate” cyberattack by distributing malware to Ukrainian government computer networks a month prior to the invasion of Ukraine. Stigal targeted non-military systems and attempted to sow doubt in the Ukrainian government by publishing citizen data. According to the federal indictment, Stigal also targeted countries that supported Ukraine, including the United States. The U.S. Department of State’s Rewards for Justice program is offering $10 million to locate the GRU hacker, who remains at large. Read here.

9. Microsoft links Scattered Spider hackers to Qilin ransomware attacks – BleepingComputer

According to Microsoft, the cybercrime group Scattered Spider has added RansomHub and Qilin ransomware to its arsenal and has begun utilizing them in its attacks. Scattered Spider was identified in early 2022 and is also known as Octo Tempest, UNC3944, and 0ktapus. RansomHub ransomware was first observed in February of 2024 and is believed to be a rebrand of the ransomware strain “Knight.” Qilin ransomware, meanwhile, first emerged in August of 2022 and was initially referred to as “Agenda.” Read more.

Make sure to register for our weekly newsletter to get access to what our analysts are reading on a weekly basis.

July 23, 2024

While the concept of humans being the weakest link in cybersecurity is undeniable, it is not always YOU or your employees that are that weakest link. Often, it’s your family or your employees’ families. The mother-in-law who uses your personal or work email to sign you up for a special deal. The grandfather who passes along your contact information to help you reconnect with an old school pal. The significant other who overshares on social media. The friend who wants to help you land a great job. Or worst of all, the child who likes to play video games and watches YouTube. 

It’s no secret that children and the elderly are often targeted in cyberattacks, primarily due to their lack of awareness and education in cybersecurity safety. One alarming trend in 2024 is the exploitation of popular platforms like YouTube to deceive and steal from unsuspecting users. 

While the videos themselves may appear harmless, the real danger lies in the links embedded in descriptions or comments, which can lead to malware downloads or phishing attempts. YouTube has emerged as a hotspot for malicious software such as Vidar, Lumma Stealer, Redline, and Racoon. For more information on StealerLogs and the dangers they pose to your system, as well as how they function, check out our previous blog posts, here and here.

A notable investigation conducted by ProofPoint in April 2024 uncovered multiple compromised YouTube channels. These channels, although appearing legitimate, were used as conduits for distributing malware or collecting sensitive information from viewers. 

So, how are these threats specifically targeting children and youth? Cybercriminals often exploit children’s trust and curiosity by embedding malware in content related to popular games or offering seemingly enticing freebies like game upgrades or cracks. Children, eager for new games and unaware of online risks, are more likely to fall victim to these deceptive tactics. 

Even if your child’s device does not store personal information directly, it still poses a significant risk if connected to the same network as other devices that do. Malware infiltrated through one device can potentially compromise the entire network, putting all connected devices— including those containing payment information or personal identifying information (PII)— at risk. 

Not only can your children compromise your personal network, but they can also inadvertently jeopardize your business or the business you work for. If young children who don’t have their own devices play on your phone and accidentally compromise it or your home network, the consequences can extend to your workplace. Bringing compromised devices to work or accessing corporate networks remotely could unwittingly upload malicious files, endangering sensitive corporate data or infrastructure. 

Parents and professionals alike must remain vigilant and educate themselves and their children about cybersecurity best practices. Establishing safe browsing habits, monitoring online activities, blocking click-through links, and restricting unsupervised access to platforms like YouTube can significantly mitigate these risks. 

So, remember, the next time you’re out to dinner in that crowded restaurant with a fussy child, YOU can easily become the weakest link in cybersecurity by queuing up that favorite YouTube video and handing your phone over to your child just to entertain them. Your kid’s entertainment could also be the entertainment of threat actor.